Leak doubts now go to a judge that reads the written rules, not to a pile in the inbox

The engine behind this site writes drafts from our repositories and runs every draft through a set of gates. One of them is the leak gate. It looks for anything that should not be public. It is told “if in doubt, flag it”, and it does exactly that.

That instruction is right for a gate. It also means the gate flags things the rules allow. When we measured the gates on 3 October 2026, the leak gate had 302 findings across 269 runs. In about half of them, the passage it quoted was still in the text afterwards. We used that as a rough proxy for “this was not a real problem”. The same measurement showed that 22 % of model calls hit the usage limit. Our task list from that day says nothing would be published autonomously yet, and that the next step was to make the leak gate calmer without making it blind.

This post describes what we built instead of making the gate itself calmer.

Where things stood on the morning of 4 October

From 4 October, by Henning’s decision, an autopilot publishes drafts that come out green on two independent gate runs. In the first version it ran once a week, on Monday. Any leak doubt was enough to keep a draft off the green list: “A leak doubt is never waved through by a model; that is what Henning is for.” Leak doubts, stops and anything that was not a plain count or date doubt went to Henning’s inbox.

The first run published four posts in a single commit at 08:02 UTC that morning. The commit message reads: “every gate green on two independent runs, remaining fact doubts judged harmless, no duplicate.”

Later the same day Henning changed the brief. In the words of the commit that followed: the app should handle its own doubts and keep the site fresh, not leave a pile in the inbox and publish in weekly batches. The project’s ground rules now put it like this: the app is to handle the doubt itself.

The judge

Leak doubts no longer block a draft automatically, and they don’t go to Henning either. They go to a separate model call, Opus according to the commit, which has one job. It gets the list of doubts and the visible text of the draft, along with the same written rules the leak gate works from: the main publishing rules, the rules on weaknesses, and the rules on financial advice. Its instructions tell it to read “the rules, not your caution”.

For each doubt it gives one of three verdicts, with a one-sentence reason:

• allowed: a rule explicitly permits the passage, and the gate only flagged it to be careful. The instructions give examples: the mechanics of an app, a weakness described as fixed without an address or anything reusable, a single inline file name, class name, flag or error message, a statistic about an app’s own method that gives no buy or sell signal, and a screenshot caption on an app page (screenshots are approved separately).
• fix: the rules forbid the passage, or the judge is unsure. The instructions say plainly that removing a passage is cheap, so when in doubt the answer is fix.
• topic: the post can’t exist without the passage, because the subject itself breaks a rule. A post whose point is what to buy is one example. This verdict is only for cases where removing the passage would leave no story.

If the call fails for any reason other than quota or authentication, every doubt counts as fix. Quota and authentication errors stop the run, so it can be retried later.

Only fix goes to the fixer

The important part is what each verdict leads to. An “allowed” doubt is dropped. A “fix” doubt goes to the fixer step, which makes the smallest edit that removes the passage. So when the judge agrees that a doubt is real, the passage is cut from the text. It is never waved through.

Then the gates run again on the edited text. A draft has to be green twice, on two independent runs of every gate from scratch, and the judge rules again on the second run. It doesn’t carry over its first verdicts.

A “topic” verdict means there is nothing to fix. If the judge returns topic on two gate runs, the draft moves to the rejected folder. Before, it would have sat in the inbox waiting for Henning.

What is never judged

Two kinds of findings bypass the judge entirely. Stops, the findings a gate marks as blocking, are never judged, only fixed. The same goes for hits from the leak gate’s pattern layer, the part that matches known forms of secrets and identifiers. The judge only weighs doubts. It can’t overrule a hard finding.

Fact doubts already had a similar but narrower check. A separate call (Sonnet) has to judge every one of them harmless, meaning a count, a rounding, a date or wording. Anything about who did what, about Claude, security, money or advice counts as a real doubt. Doubts from the form gate still block, except for word count. So does a model gate that couldn’t run.

The autopilot tries at most three rounds of fixes and gates. Whatever is still not green after that goes to Henning’s inbox with the reason. The autopilot won’t try the same text again until it has changed. It tracks this with a hash of the text.

One a day instead of a Monday pile

The same commit changed how green posts go out. A post that is green twice is no longer published on the spot. It is scheduled. The plan:

• One post a day at about 09:00 Oslo time, or two a day when more than ten are waiting.
• New posts are queued from the day after the last one already scheduled, so they never jump ahead or bunch up.
• A scheduler checks every ten minutes, publishes a post when its time comes, and sets its approval time to that moment.
• Scheduled posts count as published in the duplicate check. If two green drafts cover the same commits, only the first goes out.

The autopilot now runs on Monday and Thursday at 04:00 UTC instead of only on Monday. If a run fails, for example on quota, it tries again three hours later and carries on from where it stopped.

The About page, the /checks page and the colophon on each post were updated in the same commit to say what changed and from when. Each post still says whether it went out automatically.

What we don’t know yet

This post was written on the day the change landed. We have no measurement yet of how often the judge says allowed, fix or topic, or how often a fix verdict removes something that should have stayed. The baseline is the 3 October measurement above.

The design rests on one assumption: removing a doubtful passage costs less than publishing it. The judge’s instructions say so, and so do our rules. If the judge turns out to cut too much, posts will get thinner, and you will see that before we do.