Saraan Asim
A missing @UseGuards line on a NestJS controller passes every test nobody thought to write. The AI reviewer in our pipeline usually won’t flag it either, because nothing in the diff says the guard should be there.
My team runs AI code review in CI, with CodeRabbit. I also wrote rules for reviewing AI-generated code. Those rules only held once something in the pipeline checked them on every pull request, and the AI reviewer is part of that check.
What changed
An AI reviewer judges a change against what the change appears to be trying to do. It is good at that. It reads every line, it does not get tired on the fortieth file, and it catches the slips a person skims past.
What it cannot know is what the change should have done and didn’t. That knowledge lives in the requirement and in the heads of the people who know the system. So the job of a human reviewer moved. Tests written from the requirement now carry more of the load than they used to, and a person reads closely where tests are usually thin.
The four places a person still reads
Auth and permission checks. The missing guard above is the typical case. The code that is there looks right. The problem is the line that isn’t.
Multi-tenant queries. Most of our backends run on PostgreSQL. A query that forgets its tenant_id filter looks fine on its own, until you remember the table holds every customer’s rows.
Migrations. They are hard to undo once they have run against production data. A ten-line Drizzle migration won’t tell you whether it touches 200 rows or 20 million. (Those two numbers are an illustration, not a measurement.)
Concurrency and queues. Background jobs, retries and anything two workers can touch at once. A retry that isn’t idempotent or a race between two jobs reads as correct line by line. The bug only exists in the timing.
One that got through
In Vectadiff, my own product, two people can’t edit the same bid at once, so each edit takes a lock. The lock code had been through AI review and was merged. A few days later a browser test in CI failed on an unrelated pull request.
Reading the run’s trace showed the problem. When a page loads, it asks the database who holds the lock. If the user made an edit in the first moment after load, the edit’s lock request and the page’s check overlapped. The check could see the row before the edit’s lock was saved, answer after it, and report the bid as free. The client then dropped the lock it had just taken and refused the edit. The window was about 100 ms.
Every line of that code was reasonable on its own. The bug was in the timing between two requests, which is exactly what a diff doesn’t show. The fix was small: ignore a lock check if the lock changed while the check was in flight. It shipped with a unit test that delays the check’s answer and a browser test that replays the overlap. Both fail without the fix.
Everywhere else
Everywhere else I lean on the tests and the AI reviewer. Picking which changes still get a person’s full attention is now one of the calls I make as the lead.