The Guardrails That Stop My Publishing Agent Before It Ships

An agent that opens pull requests on its own has to be able to fail on its own too.

Search for “AI agent guardrails” and you get policy language: principles, governance frameworks, risk matrices. Fine for a slide. No help when your agent is a few seconds away from git push.

This site has an editor agent. It researches post ideas, drafts the ones I approve and opens a pull request for each. This post came out of it. What follows are the guardrails between that agent and the internet, in code, from the script that runs it.

The short version: two human decisions, one leak scan, one build check and a cost line. Everything else runs unattended.

Two gates, not a review queue

The script has two main commands. ideas researches topics and opens one GitHub issue per idea. run drafts every issue labelled approved, opens a PR for each, and revises any PR I’ve commented on. Merging schedules the post, and a daily publish job puts it live.

So a person decides exactly twice: add a label, merge a PR. There’s no dashboard or approval queue. GitHub is already where I review things, so the gates live there.

The first gate is just a list query:

for (const issue of ghJson('issue', 'list', '--label', 'approved', '--state', 'open', '--json', 'number,title')) {

The check next to it matters more than it looks. The drafter refuses any issue I didn’t open, and only my own comments reach the writer as guidance:

if (issue.author.login !== OWNER) throw new Error(`#${number} wasn't opened by ${OWNER}; not drafting it`);
const guidance = issue.comments.filter((c) => c.author.login === OWNER).map((c) => `- ${c.body}`).join('\n');

The repository is public, so strangers can comment. Without that filter, every comment is a prompt. The revise step filters PR comments and review comments the same way.

The agent itself runs as a headless Claude Code session with --allowedTools set to reading, editing and web research. It has no shell, so it can’t push, open PRs or run anything. The script does all of that, and only after the checks below pass.

The leak check: a flat list, grepped before anything is pushed

I write this site alongside a day job, and the house style bans employer names, clients, private project names and ticket IDs. The agent reads my notes and recent PR titles to find ideas, so it knows some of those words. The question isn’t whether it will ever write one. It’s whether that draft can leave my machine.

The list lives in editor/.private-terms, one term per line. It’s git-ignored, because a published list of things you never publish defeats the point. The script uses it three ways:

1. Every prompt ends with “Never mention any of these, in any form”, followed by the list. That’s the polite request.
2. Every proposed idea is scanned before its issue is created. A hit skips the idea.
3. Before anything is committed, verify() scans the text of every changed file. This one doesn’t rely on the model listening.

const leaks = (text) => privateTerms.filter((t) => text.toLowerCase().includes(t.toLowerCase()));

const verify = (dir) => {
  const changed = sh('git', ['-C', dir, 'status', '--porcelain', '--untracked-files=all'])
    .split('\n').map((l) => l.slice(3)).filter(Boolean);
  const text = changed.filter((f) => /\.(md|txt)$/.test(f))
    .map((f) => readFileSync(path.join(dir, f), 'utf-8')).join('\n');
  const found = leaks(text);
  if (found.length) throw new Error(`stopped: the draft mentions private terms (${found.join(', ')}). Nothing was pushed; worktree kept at ${dir}`);
  // then astro check, below
};

It’s a case-insensitive substring match. It won’t catch a paraphrase, and it will flag an innocent word that happens to contain a term. Both are fine: a false positive costs me a minute, a miss costs more.

That --untracked-files=all flag is there because drafting this post found a hole. By default, git status shows a brand-new directory as one entry, not the files inside it. A new post is a new folder, so src/content/posts/<slug>/ failed the .md filter and the post body was never scanned. The social copy was, because those folders already existed. Scan the files you’ll push, not the lines git chooses to show you.

Why astro check runs in the agent’s worktree

Every draft gets its own linked git worktree: a clean checkout of a fresh branch in a temp folder, sharing node_modules with the main checkout by symlink. The agent writes there and nowhere else, so a half-finished draft can’t touch my working copy or another draft.

After the leak scan, verify() runs astro check in that folder. It runs diagnostics across the project and exits with code 1 if it finds errors, which the script turns into a stop:

const check = spawnSync('npx', ['astro', 'check'], { cwd: dir, encoding: 'utf-8' });
if (check.status !== 0) throw new Error(`stopped: astro check failed. Worktree kept at ${dir}\n${(check.stdout + check.stderr).slice(-2000)}`);

Why here, and not in CI after the PR opens? Because on this site CI only runs on pushes to main, and it’s the same job that builds and deploys every morning. A broken draft that gets merged doesn’t fail one post; it stops everything from publishing. Checking inside the worktree means a broken draft never becomes a PR at all.

Both failures keep the worktree. The error message says where it is, so I can open the draft, see what went wrong and fix the prompt or the post. Only a successful push removes it.

Cost per run, in the PR body

Headless Claude Code with --output-format json returns a total_cost_usd field on every call. The docs are clear that it’s a client-side estimate, not your bill, but it’s the right order of magnitude and it’s free to collect. The script adds it up:

spent += out.total_cost_usd ?? 0;

That total goes into the PR description as a line, “Research and writing cost: $X.XX”. A revision posts its own cost as a PR comment, and the terminal prints the run total even when a run fails.

The smallest spend dashboard is a number next to the thing it paid for. When I merge a post, I can see what it cost. When a revision costs more than the draft did, that tells me my feedback was vague.

Two limits worth knowing. It’s visibility, not a cap: nothing stops a run that goes over a budget. And it counts the agent calls only, not the cover image. If you want a hard stop, the same number is right there to compare against a limit before the next call. For a weekly blog, seeing it has been enough. For a bigger fleet, I’d want the cost guards and circuit breakers that post covers.

What still needs a human

The two gates aren’t there for show. They’re the two decisions the pipeline genuinely can’t make:

• Is this worth writing? The agent can find what people search for and what the site doesn’t cover. It can’t tell whether I have something true and first-hand to say. That’s the approved label.
• Is this true, and is it me? The checks prove the draft builds and doesn’t name anything private. They can’t prove it’s accurate, or that it sounds like me. That’s the merge.

Everything else, from research and drafting to branch names, scheduling and commit messages, is cheap to undo. So it isn’t gated. The same rule drives my code review agents: automate the checks, keep the judgement.

If you’re building your own

• Put the gates where you already work. A label and a merge beat a new dashboard nobody opens.
• Filter who can steer. On a public repo, only trust comments and issues from people you expect.
• Take the dangerous tools away. Let the agent edit files; let your script push, after checks the agent can’t skip.
• Scan what you’ll ship, not what the model promised. A plain list and a substring match catch what a polite prompt misses. Then test it with a planted term.
• Fail before the PR exists. Run the build check in the agent’s own worktree, and keep the worktree when it fails.
• Print the cost where you decide. One dollar figure in the PR body is a dashboard you’ll actually read.

More in the Working with AI agents series.