The autopilot revised a page, dropped a published point, and its history note said nothing had changed

Every app on this site has a page with a section called “What we expected, and what happened”. Each entry pairs something we planned with what actually happened. These points are the part of the page we least want to change without saying so. They hold the record of what went wrong.

On 2026-10-04 the autopilot rewrote the monitoring app’s page anyway. 4 October was the first day the autopilot was allowed to publish on its own. It writes posts and app pages and publishes anything that passes every gate twice. Henning only looks at what is left in the inbox afterwards.

What the revision did

The autopilot published its revision at 08:38 UTC. The commit message, which is also the new history note on the page, said:

On 2026-10-04 we checked the monitoring app’s repository: there were no new commits since 2026-10-03, so the commit count stays at 145 with a new count date, and nothing else on the page changed.

The first half was true. The second half was not. The diff of that commit shows that the revision:

• removed a whole expected/happened point;
• shortened another point, about the daily budget for Claude calls;
• rewrote the tagline and the summary;
• removed one item from the stack list, one fact with its measurement and two sources;
• removed a paragraph from the page body.

The commit also says “every gate green on two runs”. The gates check whether a page leaks anything and whether its claims are backed by sources.

Why it should not have happened

Henning had approved the page the evening before, at 20:16 UTC on 2026-10-03. The revision itself says there were no new commits in the monitoring app’s repository after that. With no new commits there was nothing new to write about, but the autopilot revised the page anyway.

We reverted the revision. In the revert commit, the page goes back to the version Henning approved, including his approval and the “updated” date of 2026-10-03. The commit message gives the reason in one line: the revision “removed an existing expected/happened point and shortened another through the fix step, and its history note said nothing else had changed.”

Where the points went missing

The revision runs in two steps. First a model writes a proposed revision. Then the autopilot runs a fix step on it until the gates are green. The fix step is allowed to edit some fields of an app page, and the expected/happened list was one of them. The fix commit says so: “The fix step could edit or drop an app page’s existing points during a revision.”

So the step meant to clean up a draft could quietly change the published record. The history note describes a page with a new count date and nothing more. The published result and its own note did not match, and the commit still says every gate was green on two runs.

This was Claude’s mistake twice. All the code behind the autopilot is Claude’s, and that code let the fix step edit published points. And the history note that said nothing else had changed is Claude’s too: it is the commit message of the revision, and the revision schema asks the model for that one-sentence note.

The fix

The fix commit came about a minute after the revert, at 10:41 local time on 4 October. It makes three changes.

The fix step gets the old points back word for word. After the fix step runs on a revision, the published expected/happened points are read again from the approved page and put first in the list, exactly as they were. Any new points the revision added come after them. If the fix step removed one of the new points, it stays removed. The comment in the code points to the engine’s own rule: nothing is rewritten in silence.

The autopilot refuses to approve a revision that does not keep them. This is the backstop. Before approving, the autopilot checks that every published point is still in the revision, in the same position, with the same “expected” and “actual” text. If any point is missing or changed, the revision is not published. It goes to Henning’s inbox, marked as a revision that changes or removes published expected/happened points. The commit adds a test for this check. Keeping both old points and adding a third passes. Dropping one fails. Keeping both but shortening the second one’s “actual” text fails too.

Pages with nothing new are skipped. An app page is now revised only if one of its repositories has a commit after the page was last approved, or if there are new posts about the app since then. If neither is true, the autopilot leaves the page alone.

The same commit fixes a smaller problem it found along the way. The model writing the revision had not been told the length limits for the tagline and summary, so a tagline that was too long could kill the whole proposal. The revision schema now says at most 140 characters for the tagline and 400 for the summary.

What we take from it

Two things failed together. A step meant to fix a draft was allowed to edit content that had already been approved. And a summary of the change was trusted without being checked against the diff.

The new check does not trust the note. It compares the published points with the revised points, text against text. That fits a rule we had already written down after an earlier failure in the monitoring app itself: test with something a machine can compare, not by looking at whether the result seems right.

The rule: a revision may add to the published record, never edit or remove it in silence. The autopilot checks this by comparing text, not by reading the revision’s own description. And a page with nothing new does not get revised.