What broke
The CI check only checked syntax and could not catch a broken server
CI for one of our apps only ran node --check, which parses the code but never starts the server. The old regex-based SPA fallback broke under Express 5, and that check could not have caught it. We added a smoke test that boots the server and hits real endpoints.
The short version
What went wrongCI only ran node --check, which parses the server file but never starts it. It could not catch the SPA fallback regex route breaking under Express 5.
How we found itA code review during the Express 5 upgrade showed the old CI would not have caught the breakage from path-to-regexp v8.
The rule nowIf a check cannot fail on the errors you fear, it is not a check; CI now boots the server and tests real endpoints, and you should ask what change would turn a green step red.
This post documents something Claude got wrong. The claim rests on a commit or a file, not on memory. All of Claude's mistakes
One of our apps is a React front end served by an Express server. Its CI pipeline had a backend step that looked reassuring. In practice it could not fail on the kind of breakage we were most likely to hit.
This post covers what that step did, how we noticed the gap during a code review on 2 July 2026, and what replaced it.
What the check did
The backend step in CI ran node --check on the server’s entry file. The workflow comment above it called it a light sanity check on the backend, and added that there was no test suite yet. node --check parses a file and reports syntax errors. It does not run the code, load the dependencies or register routes, and it never answers a request.
So the step proved one thing: the server file was valid JavaScript. It said nothing about whether the server could start or whether its routes did what they should.
The trap that was waiting
The app ran Express 4. Dependabot had opened a pull request to move it to Express 5. Our progress notes for that day call the PR “a standing trap”, and the reason is specific.
The app is a single-page app, so the server needs a fallback. Any path that is not an API endpoint or a static file should get the front end’s index page, and the client-side router takes it from there. That fallback was written as a regex route. Express 5 ships a new major version of its path-matching library (path-to-regexp v8), and the notes say the regex route broke under it.
If the Dependabot PR had been merged as it was, CI would have gone green. The server file would still parse fine. The fallback would have stopped working, and nothing in the pipeline started the server to find out. From the progress notes: CI “only ran node --check and would not have caught that the regex route broke on path-to-regexp v8.”
How it was found
We found the gap during a code review pass on the app, recorded in a single commit. The review took the Express major upgrade, and the notes record that the old CI would not have caught what it broke.
The commit and notes say plainly what the old check could not do. They do not say who wrote it or when, so we don’t assign it to Henning or to Claude here. The lesson does not depend on authorship. The step was there, it was green, and it gave a confidence it had not earned.
What we changed
The commit has two parts.
The fallback was rewritten. The SPA fallback stopped being a regex route and became a plain middleware. According to the notes, this works the same on Express 4 and Express 5, which made the major upgrade safe to take. The lockfile moved from Express 4 to Express 5, and the commit message says this neutralises the Dependabot trap.
CI got a smoke test. A new step runs after the syntax check. It boots the actual server against a temporary cache directory and then sends it requests. The notes list what it checks: the health endpoint, the SPA fallback, a round trip on the stored preferences, and more. The commit message counts eight checks. The comment in the workflow states the purpose directly: it catches routing breakage that node --check does not see, for example Express major bumps.
We kept the syntax check. It is cheap and harmless. It is just no longer the only thing between a dependency bump and a merge.
How it was verified
The commit message lists what was run before it went in:
- the new smoke test, 8 checks
- the Playwright end-to-end suite, 14 checks
- a separate Playwright sync scenario, 5 checks
- a production build
- Express 5 run locally
These numbers come from the commit message of that day.
The rest of that review
The same commit fixed several unrelated things, which shows how much a single “light sanity check” had let through:
- a preferences sync bug, where an item deleted on one device came back from another device’s local storage
- timeouts on every call to the local language model backends, after a hung model server left a scan request hanging forever
- atomic JSON writes (write to a temp file, then rename), so a crash mid-write can no longer corrupt files on the volume
- reading that tolerates a UTF-8 byte order mark
The syntax-only CI step would have caught none of these. That isn’t a criticism of node --check. It does what it says. The mistake was letting it stand in for a test.
The rule
If a check cannot fail on the kind of error you are afraid of, it is not a check. A syntax check guards against syntax errors. For a server, the errors we fear are a route that no longer matches, a fallback that no longer falls back, and an endpoint that throws on startup. To catch those, CI has to start the service and test what it actually does, by booting it, sending real requests and checking the answers. Before trusting a green step, ask what change would make it red. If you can’t name a realistic one, the step is decoration.