Guards and dials that were built but never ran in the path that mattered

The stock app runs a fleet of simulations plus one live account with real money. In late July all of them moved to a newer engine, the target-portfolio engine. The older candidate-loop path stayed in the code as a legacy reference.

Over about a week, between 30 July and 7 August 2026, we found four places where Claude had built a safeguard or a setting that did nothing where the trading actually happened. The test suite was green each time: 538 tests on 30 July, 678 on 31 July, 976 on 7 August. None of the four was caught by a unit test. All of them were caught by watching what the running system did.

1. The swap mechanism could never trigger (30 July)

Henning asked a simple question: what happens if something during the day calls for a buy, but all the cash is already spent? The commit message for the fix says the answer was “worse than the question suggested” (translated from Norwegian).

The engine was configured with at most 10 positions and at most 15 % of capital per position. That adds up to 150 % of capital. With a 5 % cash buffer, only 6 positions could ever be funded. The engine treated the portfolio as “full” at 10 positions, so that condition never became true, and the whole swap mechanism in the target-portfolio engine was code that could not be reached.

The measured effect on the real fleet: a US stock with a top score of 1.00 became tradable at 15:30 and got a position of 5,804 NOK. The European names bought that morning got 75,000 NOK each. The day’s best signal got the smallest position only because its exchange opened last.

While checking the fix, Claude found a second cause. The mapping from the general config to the engine’s config was written by hand in two places. When the swap flag was added, it silently dropped out of both. The config warning reported the flag as on while the plan behaved as if it were off. Since then there has been a single mapping function.

2. The minimum hold time never reached the engine (31 July)

Before a deploy, Henning asked for an acid test. The planned checks passed. Then the monkey test, which compares the app’s trades against random picks, showed that a fleet only two hours old already had 8 round trips, all on the same day. The transactions showed buys at 20:30 sold at 21:30 as rotation exits.

The setting minHoldDaysBeforeRotate exists in all three risk profiles. It had never been passed to the target-portfolio engine:

Profile	Setting	Actual hold time
Passive	5 days	1.0 hour
Balanced (×2)	3 days	1.0 hour
Aggressive	0 days	1.0 hour (correct)

Passive and balanced behaved exactly like aggressive on the one dimension the setting exists to separate them on. They paid double trading friction on decisions that lasted an hour. Friction had already been measured at 71 % of the underperformance. The commit also notes a side effect: the monkey test cannot evaluate same-day trades at all, so these trades could never be judged on stock selection.

The commit calls this the third time the same class of bug had bitten. Claude then mapped all 31 config fields in a design doc: 12 reach the engine through the mapping function, 14 reach the cycle by other routes, and 5 have no effect on this engine. Some of those 5 are inactive on purpose and documented as such. The doc says plainly that unit tests do not catch this class of bug: “the pure function can be perfect while the engine never calls it with parameters that give a different answer” (translated).

3. The duplicate-order guard sat in the legacy path (5 August)

At 18:00 on 5 August the live cycle placed a buy order for 899 shares of MOH. A buy for 903 shares from the 17:00 cycle was still open at the broker. Both were marked “Working”. If both had filled, the account would have owned the name twice.

There was a guard against this. The commit message explains where it was: in the candidate loop, “which is the LEGACY path” (translated). There were zero references to the open-orders list anywhere in the target-portfolio branch, and “the whole fleet plus LIVE run target_portfolio”. The guard did nothing in the path that mattered.

Idempotency did not help either. The two orders had different client order IDs, so the broker correctly saw them as two separate orders.

The fix moved the check into the one function every buy goes through, so a new code branch gets the guard without anyone having to remember it. Claude also made the check compare against the app’s own order reference, not only the broker’s symbol lookup. That lookup can fail and return a placeholder symbol, and then the check would have let the order through without a word. The order just placed is now added to the list of open orders, so the guard also catches two buys of the same name in one cycle. The duplicate was cancelled, and the open-orders tool now exits with an error when it finds duplicates instead of only listing them.

The same commit corrects an overreaction of Claude’s own. A new cycle checker had flagged traded, a normal outcome, as abnormal. In the commit Claude writes “I whitelisted only ‘ok’ and ‘ran’” and adds that a watchdog that cries wolf over normal operation gets ignored, which makes it worse than no watchdog.

4. The startup ramp was set in zero simulations (7 August)

Henning asked for a forward-looking verification sweep, “so that we don’t assume anything and then find out we have to wait another period” (translated). All 19 invariants passed. But a startup ramp, a cap on new positions per day, had been built that same day, unit-tested and checked through the engine with the impact tool. It was then set in zero simulations. The commit names the same pattern from earlier: a sector cap had sat unused for six weeks.

The ramp only matters while a portfolio is being built up, and the fleet was fully invested, so a live A/B test would have given n=1. Claude added the ramp to the backtest engine instead and measured it over four historical drawdown episodes. It included a sensitivity row, so that “looks like the control” could not be mistaken for dead code:

Variant	Avg return	Downside capture	Upside capture
Balanced	+6.5 %	0.67	0.98
+ ramp 3/day	+4.1 %	0.77	0.69
Sensitivity: 1/day	+4.7 %	0.70	0.57

The mechanism works, and it does harm: it does worse on both downside and upside capture. It stays off, this time because of a measurement and not by accident. The docs point out that the two tools answered different questions. The impact tool said “not dead”, and the stress test said “not good”. The first answer on its own reads like an approval.

The same sweep found a sell rule that had never fired in any simulation. Its variant ran the aggressive profile, which holds positions for 3.4 days on average, while the rule needs a position to survive a 5-day window. Only 2 of 11 closed positions did. “Identical to the control” had been reported every day. It could mean two very different things: nothing triggered the rule, or the rule could never be reached.

The rule

Verify that a change actually changes behaviour in the running code. For example, run a consequence tool against real data and check that the plan with the setting differs from the plan without it. Green unit tests prove that a function works, not that the engine calls it.

Put a guard in the one place every action goes through, not in the branch that happens to be in front of you.

When a setting might never bind, measure it with a sensitivity row. Otherwise “looks like the control” gets mistaken for dead code, or the other way round.