Novastacks Field Note

How to REALLY stop AI slop in your writing: the engineering fix (copy-paste prompt included)

A rule in your skill file is a suggestion your writing agent may or may not implement. The same rule in a YAML file, enforced by a script wired into your build, is deterministic. It works 100% of the time, before the AI ever passes the draft to your eyes.

How to REALLY stop AI slop in your writing: the engineering fix (copy-paste prompt included)
Share

Key Takeaways

  • A writing rule kept in a skill file or system prompt has no enforcement, because the model that drops it is also the model that grades whether it was followed.
  • Move the rules into a machine-readable file, let a script return the verdict as an exit code, and fire that script from your editor or CI rather than from the prompt.
  • Block at the point of no return. Editor hooks report; only the publish or deploy step can refuse.
  • Graphite found 86% of articles ranking in Google and 82% of articles cited by ChatGPT and Perplexity are human-written, so the quality bar is doing commercial work.
  • This article failed its own checker three times before passing, once for quoting the sentence that created the rule.

I had a rooftop dinner with friends last night, and people are still complaining about em dashes and "this is X, not Y" as the tell for AI writing.

I get why people still run into this. Every one of those patterns is public by now, identified and shared a hundred times over. Knowing them has not been enough.

Fixing that takes engineering.

None of my writing rules live in a prompt, hoping to be remembered. They sit in a YAML file. A Python script reads that file and returns pass or fail. My editor runs the script on everything I write, whether I ask it to or not. The script decides, and the model doesn't get a vote.

Below is the prompt that builds the whole thing, then the reasoning behind it.

The cheat sheet: build it yourself

This is the prompt I would hand a colleague. Paste it into Claude Code, Codex or Copilot. It interviews you on your own rules first, then builds the checker, the tests, the hook wiring, the publish-time block, and the loop that turns every miss you catch into a new rule.

Do not skip the interview. It is the step that decides whether the checker catches anything you actually care about.

build-rules-gate.prompt
I want you to build me a deterministic "house rules" gate for my written
output. A system that enforces my writing rules mechanically instead of
relying on you to remember them. Work through this in order and stop at
each checkpoint.

## The problem I'm solving

I have writing rules: banned words, banned phrases, structural patterns
I dislike. When those rules live only in a skill file or system prompt,
they get ignored. You weigh them against everything else in context and
quietly drop them. I want the rules moved out of prose and into a program
with an exit code, so compliance stops being a judgment call.

You must play NO part in the verdict. A plain text match decides.

## STEP 1 - Interview me first. Do not write any code yet.

Ask me, one topic at a time:
1. What are my banned words? Single words that are always wrong.
2. What are my banned phrases? Exact strings.
3. What are my banned patterns? Structural ones, for example
   "It isn't X. It's Y." Ask me for two real examples of each so you can
   write an accurate regex.
4. What are my mechanical limits? Max em dashes, max exclamation marks,
   average sentence length.
5. What TYPES of content do I produce, and how long does each run?
6. For each type, absolute counts or rates per 1,000 words? Short pieces
   get absolute caps. Long pieces get rates, otherwise length alone
   trips the checker.
7. Can I give you samples? One piece I am happy with, one I am not.

Summarise back what you heard. STOP and wait for my approval.

## STEP 2 - Build the rulebook

Create rules.yaml as the single source of truth. Every rule gets:
  id, kind (banned_word | banned_phrase | banned_pattern | mechanical),
  regex, severity (fail | warn), applies_to, note (what to write instead)

Add a profiles: section, one block per content type, holding that type's
thresholds and whether the mode is absolute or per 1,000 words.

Rules live in this file and nowhere else. If I later ask for a readable
summary of my rules, generate it FROM this file. Two hand-maintained
lists will drift, and once they disagree neither is authoritative.

## STEP 3 - Build the checker

Create check.py:
- Usage: python3 check.py <file> [--type <content-type>]
- Guess type from path or extension. --type overrides.
- Read rules.yaml. Do not hardcode any rule in the script.
- Accept .md, .txt and .html. Strip HTML tags before scanning.
- Print every violation with line number, matching text, rule id, and
  the note explaining what to do instead.
- Exit 0 if clean, exit 1 if any fail-severity rule matched or a
  threshold was exceeded. The exit code is the whole point.
- Add --list-manual for rules I marked human-check-only, so the ones
  too subtle for regex stay visible instead of being dropped.
- Standard library only. No pip installs.

## STEP 4 - Test the rules themselves

Create fixtures/ and run-tests.sh. Put at least three files in fixtures:
a piece I approved that must PASS, a piece I rejected that must FAIL, and
one you write that violates every rule and must FAIL.

run-tests.sh runs the checker against each, compares to the expected
verdict, prints a table, and exits 1 if any expectation is missed.

Run it now. If my approved sample FAILS, the rule is too aggressive. Tell
me which rule and propose a narrower pattern. Do not loosen it silently.
A checker that flags good writing gets ignored within a week.

STOP and show me the table.

## STEP 5 - Make it run automatically

Ask which environment I use.
If Claude Code: add a PostToolUse hook to settings.json matching
Edit|Write that runs check.py on any written content file. Show me the
exact JSON and where it goes.
If Codex, Copilot, or anything without hooks: install a git pre-commit
hook that blocks the commit on exit 1, plus a CI workflow that does the
same on push.
Either way, the check must fire without me asking for it.

## STEP 6 - Block at the point of no return

Ask me where my writing actually leaves my control: publishing,
deploying, sending, exporting. Wire check.py into each so a non-zero exit
stops the action. Check the FINAL artifact that ships, after any
conversion, not the draft upstream of it.

Add this to my project instructions, word for word:
  "The agent that produced a piece of work never issues its own pass.
  Run check.py and show the real output. If the checker does not fit the
  artifact, report the mismatch. Never substitute your own judgment."

## STEP 7 - Write down what this cannot catch

Create LIMITS.md: anything I write outside these tools, any language the
regex does not cover, and tone and register generally. State plainly that
a green check means "no banned patterns found", not "this is good."

## STEP 8 - Set up the learning loop

Write this into LEARNING.md and follow it every time I report a miss:
1. Find the PATTERN, not the instance. Ask me for a second example.
2. Add the rule to rules.yaml with a note.
3. Add the draft that slipped into fixtures/ as a must-FAIL case. This is
   the step people skip, and it is what stops a future rule edit from
   quietly re-opening the same hole.
4. Re-run run-tests.sh. Confirm the new fixture fails and every existing
   fixture holds its old verdict.
5. Never fix only the draft. The draft is the symptom.
Do the same in reverse when I approve something: offer to save it as a
must-PASS fixture, so the checker learns what my good writing looks like.

## STEP 9 - Hand it over

Write a short README: how to run it by hand, what the exit codes mean,
where rules get edited (rules.yaml, nowhere else), and a pointer to
LEARNING.md. Then show me the file tree and one real run.

Everything below is the reasoning. If you only wanted the tool, you already have it.

Why your AI ignores the rules you wrote in your skill file

The rules you wrote in your SKILL.md are competing with everything else in the model's context, and they hold no special standing there. Sooner or later they get dropped. Then the same model that dropped them tells you the draft is clean, because nothing outside the model ever read the output.

Your rules are advice, and the thing taking the advice is also the thing grading it.

That is why longer banned-word lists keep failing. You can write the rule perfectly and still watch it get ignored on a long session, because writing it down was never the part that binds.

This matters commercially, not just aesthetically. Graphite analysed 65,000 URLs published between 2020 and 2025 and found that 86% of articles ranking in Google Search are human-written, and 82% of articles cited by ChatGPT and Perplexity are human-written. AI-generated articles briefly passed human ones in November 2024, then settled back to roughly level, and they tend to rank lower when they do appear. The engines are still disproportionately citing work that reads like a person wrote it.

The five controls

These follow the life of one rule, from writing it down to it blocking a deploy.

1. Put every rule in one machine-readable file. Every rule gets an ID, a pattern, a severity, and the content types it applies to. Mine holds about 111 of them. Any readable summary is generated from that file rather than maintained beside it. The moment two lists disagree, the model stops treating either as authoritative and follows whichever it reads first.

2. Make the checker a program, and make it accurate. The script returns pass or fail as an exit code, and reads the rule file rather than hardcoding anything. Calibration matters as much as coverage. Thresholds are tuned per content type: hard caps for a short post, rates per thousand words for long articles, per-slide scoring for decks. Before I calibrated mine it threw 80 restricted-word violations at a 14,000-word audit document that was mostly quoting the client's own copy. That is how a tool teaches people to ignore it.

3. Let the environment run it, not the model. A PostToolUse hook fires the checker on every content file written. Without hooks, a git pre-commit hook plus a CI job cover the same ground. Running the check stops being something you have to remember when the session gets long. This is the single biggest lever against being ignored, because you removed the choice.

4. Block at the point of no return. Editor hooks can report, but they cannot stop anything. So the real refusal lives where work leaves your control: publish, deploy, send, export. One standing rule sits next to it. Whatever produced the work never issues its own pass. I learned that one the hard way, when an agent ran a post-scoped checker against a 31-slide deck, noticed the mismatch, and passed itself anyway on a targeted read. The slop shipped.

5. Make it survive other people. The rules get their own tests: saved files with known verdicts, one you approved, one you rejected, one deliberately bad. Re-run them after every rule edit, so tightening one pattern cannot quietly break another. The kit installs into any project with one script, so teammates get updates by pulling rather than by reading a doc.

What almost everyone else is missing: it's the enforcement layer

The most popular answer to AI slop is a skill file called stop-slop, in the screenshot below. It carries 15,500 stars and 1,100 forks, which makes it the most widely shared anti-slop artifact on GitHub. The pattern library inside it is genuinely sharp.

It also contains no executable code. The repository is markdown: SKILL.md plus reference files for phrases, structures and examples. No releases published. The model reads the rules, scores its own draft on five dimensions from 1 to 10, and the instruction is "below 35/50: revise".

The stop-slop GitHub repository, showing 15.5k stars, 1.1k forks, 12 commits, only markdown files (SKILL.md, README.md, CHANGELOG.md, LICENSE and a references folder), and no releases published.
The most-starred answer to AI slop. Every file in it is markdown.

That gap is worth sitting with. What gets starred, forked and shared is not what a working content practitioner actually runs. What is being distributed is a suggested rulebook.

Don't get me wrong, that is a good rulebook to minimise AI slop. It has no way to enforce itself, because the thing being graded is doing the grading.

An enforcement layer is the part that sits outside the model: it reads the finished output, applies the rules without being able to negotiate, and can stop the next step from happening. Run any rulebook against the five controls and you can see precisely where it stops.

ControlRulebook aloneWhat the enforcement layer adds
1. Rules as machine-readable dataWritten as prose for a model to read.A format a program can execute, so the rules apply the same way twice.
2. A program returns the verdictThe model scores itself 1 to 10 across five dimensions.An exit code from something that was not involved in the writing.
3. The environment fires itApplies when the model chooses to invoke it.A hook or CI job, so it runs when a long session makes it most necessary.
4. Block at the point of no returnNothing to block on.Publish and deploy refuse to run on a failure.
5. Survives other peopleRules can be weakened with nothing to notice.Fixtures with known verdicts, so a rule edit cannot quietly break another.

What the gate caught on this article

I ran this piece through the same checker while writing it. It failed three times before the draft ever reached my eyes.

That is the part that changes your day. Three rounds of slop were caught and fixed before I read a word of it. I never had to tell the model it had used a banned reveal, wait for a rewrite, then read the rewrite to see whether it had introduced something new. That loop, spotting the pattern, naming it, asking for a fix, checking the fix, is the work I used to do by hand on every draft. It now runs without me, and I only see what is left.

The first fired on a banned buzzword I had quoted as an example of a banned buzzword. The checker cannot tell the difference between using a word and citing one.

The second is the good one. To explain how the system learns, I quoted the exact sentence that created one of my rules, and the rule fired on it. The article was flagged for quoting its own origin story.

$ python3 slop-check.py why-ai-ignores-your-writing-rules.html --type blog

FAIL — 1 violation(s) found:

  [BANNED_PATTERN] Line 344: 'X wasn't A. It was B'
                   past-tense trite reframing / reveal
    Rule: wasnt-itwas
    Text: "the sharpest takeaway wasn't about the tools.
           It was about who actually gets to use them."
The checker flagging this article for quoting the sentence that created the rule.

I repunctuated the example rather than loosening the rule, because a failing check does not get argued with inside the deliverable.

Then the human read caught two more things no regex can see: a balanced aphorism in the closing line, and a three-item list inside a sentence. Both are written in my rules. Neither is detectable by pattern matching, which is why a green check is the first gate and never the whole gate.

$ python3 slop-check.py why-ai-ignores-your-writing-rules.html --type blog

PASS — No fail-level violations found.

Em dashes:           2   (0.57/1k, fail over 3.0)
Restricted words:    0   (0.00/1k, fail over 8.0)
Robotic transitions: 0   (0.00/1k, fail over 2.0)
Exclamation marks:   0   (0.00/1k, fail over 0.5)
Manual-only rules:   4   (voice read still required)
The run that let this article ship. A green check is the first gate, never the whole gate.

Every run gets logged. On this file that is 16 so far, three of them failures. The log answers "was this actually checked", which is a question no amount of good intentions can settle.

What this system will not do for you

Write these down when you build yours. A green check is worth much less if people read it as approval.

It cannot see tone. Pattern matching catches banned words and structures. Whether a paragraph sounds like a person still needs your eyes.

It is language-bound. English regex does nothing for your Chinese content.

It starts weak. Version one catches your obvious rules. It gets good by absorbing every miss, which takes a few months of real use.

That is a narrower promise than it sounds, and still worth having. Build the first version this week with the prompt above, then add a rule every time you catch something it missed. If you want help wiring this into a content operation rather than a personal workflow, that is the kind of system Novastacks builds for clients, and the same thinking shows up in our study of what makes a page win across engines.

Frequently asked questions

Why does AI ignore my instructions?

Because an instruction in a prompt or skill file has no special standing. It competes with everything else in the model's context and gets dropped as the session grows. The same model then assesses its own compliance, so nothing outside the model ever checks the output. The fix is to move the rule into a program that returns a pass or fail verdict independently.

Why is ChatGPT ignoring my custom instructions?

Custom instructions are context, not enforcement. They are weighed against the current task, the conversation history and any other instructions in play, and long sessions make it worse. If a rule genuinely must hold every time, it needs to be checked by something outside the model, such as a script wired into your editor or your publishing step.

How do I stop AI slop in my writing?

Keep your banned words, phrases and structural patterns in one machine-readable rules file. Have a small script scan each draft against that file and exit non-zero when it finds a violation. Fire the script automatically on every file you write, and make your publish step refuse to run on a failure. Then add a rule every time you catch something the script missed.

Do AI content detectors solve this?

No, and a rules checker is a different thing. A detector guesses whether a whole document was machine-written and produces false positives at rates that make it unsafe to act on. A deterministic checker makes no authorship claim. It reports which named rule matched at which line, so every finding can be checked by eye in a second.

Sources

  1. Graphite, AI Content in Search and LLMs, analysis of 65,000 URLs published 2020 to 2025.
  2. Axios, AI-written web pages haven't overwhelmed human-authored content, study finds, 14 October 2025.
  3. Charlie Guo, The Field Guide to AI Slop, on detector reliability.
  4. Momentic, 34 types of AI slop you should avoid in your content, on tell co-occurrence.
  5. hardikpandya, stop-slop, a skill file that self-rates against a 35 of 50 threshold.
  6. petergyang, no-ai-slop, whose evaluation checks are answered by the model itself.

Want this wired into your content operation?

We design and run AI search systems for brands that need their quality bar enforced, not just documented.