The migration that said "completed" and never ran

On 5 August 2026 a trading cycle in the stock app crashed at 17:30. Nothing was wrong with the market data or the trading logic. The code wrote to a database column that did not exist, because the migration meant to create it had never run. The migration tool had reported that it ran fine.

This post covers how that happened, how we found it, and the test that came out of it. The mistake was Claude’s, and the commit messages say so in Claude’s own words. All quotes below are translated from Norwegian.

How it started: no transactions in the app

The owner reported two things: “I don’t see any transactions in the app” and “live still shows old order history”. The commit that fixed this says: “Both were right. Three bugs, all mine, all from today.”

The main bug was that reconciliation against the broker was never called. The module existed but could only be run from the command line. The simulated broker writes its own transactions. The real broker only places the order. So fills happened at the broker and never showed up in our books. Claude measured the gap at the time: the broker held 2 positions and we held 0.

The fix also needed a new column on the order log, rule_key. It stores which rule triggered an order, which is how the app later judges each signal source on the trades it actually made. Reconciliation read that column to work out why a position had been bought. It had never existed. So the query threw, mirroring failed, and in the commit’s words, “a missing reason prevented the position itself from being booked.”

To add the column, Claude wrote migration 0057 as a SQL file. That fix went in at 17:25.

What Claude got wrong

The migration tool does not run every SQL file in the folder. It runs the files listed in a journal. A file that is not in the journal is skipped without any error. This one was written by hand, and the journal entry was left out.

The follow-up commit, made twelve minutes later, says it directly:

I wrote the .sql file by hand and forgot the journal. db:migrate skipped it and printed “Migrations completed”. I read the success message as if the world had changed – the dominant error class in this codebase.

The code that used the column was deployed. The column was not there.

What it cost

The order log endpoint returned an error on every request.

The 17:30 trading cycle crashed with the verdict skipped-abnormal, because the deployed code tried to write rule_key into the missing column. According to the commit, that cycle would have bought a stock with a score of 1.09 because there was free room in the target portfolio. The commit sums it up: “A whole trading cycle lost.”

A comment in the new test file adds another cost. Reconciliation fell back to a generic “reconciliation” label as the reason for two real buys, so they lost their attribution permanently.

How a safety net hid the problem

In the 17:25 fix, Claude had wrapped the rule-key lookup in a try/catch, so that a failed lookup would no longer block a position from being booked. That was a reasonable goal. But once the column was missing, the try/catch swallowed the error every time and fell back to the default label. Mirroring looked like it worked.

The commit puts it this way: “The guard hid that the column was missing.” The day’s write-up states the lesson: a fallback that protects against a passing glitch is useful, but a fallback that protects against “the column does not exist” only hides the problem.

How it was found

The owner’s report about missing transactions started the work. After that fix, Claude checked whether reconciliation was really running as part of the cycle. To tell the cycle’s runs apart from manual ones, Claude added a read-only listing of the reconciliation history that does not write a new row. It showed a row at 17:30:00 written by the cycle, and Claude’s own manual runs at 17:21, 17:23 and 17:31.

The crash itself showed up in the cycle record as skipped-abnormal. The tool we use to check the latest cycle was no help here. It read a field that does not exist and printed “skipped: (no)” for the crashed cycle. That was a separate bug from the same day, with the same pattern: something missing displayed as if everything was normal.

The journal entry for migration 0057 went in at 17:37, along with the new test.

Why 933 passing tests saw nothing

The day’s write-up gives three reasons:

• Raw SQL is not type-checked. A query that names a missing column is just a string until it runs.
• The simulated broker account had been empty, so there had never been a difference to mirror. The broken path had never run.
• Unit tests check what exists. None of them could notice that something did not exist.

That evening Claude wrote up the day as four bugs of the same kind. The migration was the first on the list. In each case something missing was displayed as confirmation that all was well.

The rule

A new test now checks the migration journal against the SQL files in both directions. Every SQL file needs a journal entry, which is the direction that bit us. Every journal entry needs a file, because an entry without one breaks a fresh setup and a restore from backup. A third check makes sure the journal indexes are unique and have no gaps. Together the three tests run in about 4 ms, according to the commit.

The working rule is wider than the test. A success message is not proof that anything changed. “Migrations completed” is what the tool reports about itself. After a migration, run a query against the thing you expected to be created, and check that it is actually there.