AI writes the code. Fine. But who checks it?
Writing code is no longer the problem: AI does it fast and well. What matters now is checking and securing that code before it reaches production!
Late 2025: AI writes, humans review
Claude Code has radically and permanently changed the way we code. An agent reads the code, writes the fix, runs the tests, TDD-style (tests first), and opens a Pull Request (PR), in a few dozen minutes. With several agents running in parallel, PRs arrive by the dozen. But the old paradigm still holds: nothing goes to production without a human review. Skip it? “Sacrilege!” the most reluctant would say.
For now, AI writes, humans review.
But humans can’t keep up: a single PR can take hours to review, and reviewing everything becomes impossible. The result: burnout spreads. What to do? Slow AI down to human speed, or approve without really reading: two bad choices!
Review has to scale up… with AI!
2026: AIs review each other
AI already codes better than humans. It also reviews better: it reads every line, never gets tired, and can add a systematic security analysis on top, something humans rarely do, and usually after the fact, in an audit.
So we decided to hand our code review over to AIs, in a controlled way, through a skill called /full-review. Several AIs take turns reviewing the PR. Each problem found is fixed right away, then the review starts again with another reviewer. And the skill improves itself from PR to PR, based on the problems it runs into.
/full-review skillAfter two bug-free reviews in a row, by two different AIs, the PR leaves the loop.
Our experience
The context: maintaining an existing 300,000-line application. The goal: fix the many bugs found in production, then build the new features the client asks for, without ever breaking in production what already works…
For nine months, every fix or change has gone through a PR written by AI. More than 600 PRs have gone through our /full-review code review skill. Here is what the numbers taught us.
Nine months in four phases
- February to April: one AI, one pass136 PRs, small and merged fast. About one in two is reviewed once, by a single AI.
- May to July: the break-in periodThe
/full-reviewskill is born. The rules change every week, reviewers keep failing. - July to September: three vendorsThe skill moves to v2, then Grok (xAI) joins Claude (Anthropic) and Codex (OpenAI).
- October: the loop converges3 passes per PR (median), down from 7 in August.
What the numbers show
Since the end of the break-in period, on July 13:
A PR takes 5 passes (median). Only 11 out of 376 shipped exactly as the AI first wrote them.
In May, the second pass found nothing seven times out of ten. The code wasn’t better: the review was less demanding. Tightened in June and July, it finds two to four times more issues from the first pass.
August is the peak: PRs double in size, close to 1,000 lines (median), and everything climbs with them. In October, the first pass still finds six issues, but later passes almost none: the second pass is clean one time in two, compared with one in ten in August.
before the skill
In April, with no process, a PR was merged in 1 hour 20 minutes. During the break-in period, time to merge soars: close to 15 hours in June. Once the process is stable, it comes back to 4 to 6 hours. That is the price of validation, and you can read it right off this chart.
See the month-by-month data
| Month | PRs | Full-review (%) | Median size | Median passes | Issues 1st pass | Issues later passes | Clean 2nd pass (%) | Median time to merge (h) |
|---|---|---|---|---|---|---|---|---|
| Feb. | 4 | 0 | 712 | – | – | – | – | 0 |
| Mar. | 43 | 9 | 255 | – | – | – | – | 13 |
| Apr. | 89 | 1 | 201 | – | – | – | – | 1.3 |
| May | 108 | 58 | 285 | 5 | 3.4 | 0.8 | 70 | 7 |
| June | 165 | 75 | 317 | 5 | 3.0 | 1.7 | 29 | 14.6 |
| July | 134 | 83 | 476 | 5 | 5.9 | 2.5 | 28 | 3.6 |
| Aug. | 117 | 86 | 961 | 7 | 12.7 | 4.6 | 9 | 5.8 |
| Sept. | 144 | 83 | 564 | 4 | 9.4 | 3.0 | 37 | 6 |
| Oct. | 76 | 91 | 471 | 3 | 6.4 | 0.9 | 50 | 3.9 |
Method: figures taken from the summary lines left by each review pass on over 600 of the project’s PRs, from February 18 to November 11, 2026. Months are those in which the PR was opened. November, incomplete, is left out of the charts. The writing-to-review ratio is measured on 7 sessions in October and November, active time only.
What we learned
Every rule in the process comes from a lesson learned in the field. We have logged more than a hundred of them. Here are the lessons that most changed the way we validate.
1. Pit the vendors against each other
Having an AI review the code it just wrote adds little. Changing its effort level, or switching to another model from the same vendor, helps a bit. The best option is competition: Codex (OpenAI) reviews code written by Claude Code (Anthropic), or the other way round. Each one hunts down the slightest detail in the other’s work.
The numbers confirm it: in 38% of PRs, one AI judged the code clean, then another vendor’s AI found a problem in it. Same code, different eyes. That is why we use three vendors.
A real case
A PR replaced an error with a fallback value when an external source sent corrupted data. The first review saw only a documentation detail. The second described it as an improvement, “safer”.
On the fourth pass, another vendor’s model followed that value to where it was used: it fed into the calculation of real amounts. Issue rated critical, change reverted.
A very human trait, by the way. When an AI is shown the bugs found in its own code, it almost always accepts them. But it is quick to play them down: “OK, it’s a bug, but it’s less serious than you say.” An exchange between two AIs, with no human in the loop.
2. Validating costs five to ten times more than writing
In the sessions where we could measure both phases, review takes five times longer than writing (median), and up to ten times. Writing takes a few dozen minutes; validating takes hours. Review also burns far more tokens, with three reviewers per PR.
Yet this is where AI made the most progress in 2026. It has become much better than humans at validating code, finding security flaws and fixing them before release. No human reviewer keeps the same attention from the first review to the hundredth.
3. Tune the settings so the loop converges
Two settings decide when the loop stops. First, the bar: each reviewer rates what it finds on its own scale, from a blocking bug to a typo in the docs, and the skill sets the level from which an issue restarts the loop. Second, the number of bug-free reviews in a row required to merge: two by default, a parameter of the skill.
Badly tuned, the loop never ends. Counting every rerun of the skill on the same PR, 34 PRs reached 20 passes, and seven went past 30, up to 41. A style detail in a test restarted the whole loop, just like a production bug.
So we set the bar according to the stakes: high on executed code and scripts, lower on tests and documentation. Less demanding on details, the review needs fewer passes. It is one of the changes behind October’s convergence.
4. Document every PR to keep the project’s memory
Every PR comes with a Markdown file versioned in the repository: context, root cause, fix, files touched and test plan. More than 600 PRs are documented this way. It is the project’s memory: an AI picking up a topic months later doesn’t have to rediscover everything; it reads this file, as a new developer would. The file is reviewed too, just like the code.
5. Bugs hide in edge cases
Reviewers don’t stop at obvious mistakes. The issues they raise are mostly about edge conditions, rare cases and security flaws: an empty value, a negative number, an external source returning corrupted data. Before, these cases were handled only partly, and often poorly. That is where AI review adds the most.
6. One problem found hides others
Fixing only the flagged line is not enough: the same flaw often sits elsewhere in the code. Before each fix, the AI searches the whole repository for the same pattern, and fixes every occurrence, with a test for each.
7. The process corrects itself
At the end of each review, the AI suggests improvements to its own rules. We review them and merge them in batches, several times a month. October’s process is no longer July’s, and that is by design.
8. Orchestrate the reviewers
Automating a review across several AIs takes a conductor: launch each reviewer on the PR, collect its verdict, relaunch the one that stops responding. Several tools do this. We chose Orca, which coordinates each vendor’s agents while keeping their original command-line tool: Claude Code, Codex and Grok Build run as is, with no adapter layer.
And also
9. A failed review is not a clean review
A reviewer that crashes reports no issues. Counted as “clean”, it could ship a PR nobody had read. Now a failed review is ignored, never counted.
10. No single reviewer should be able to close the loop
Two clean reviews in a row are not enough if they come from the same reviewer: it would close the loop alone, with its own blind spots. This happens whenever a reviewer stops responding, quota reached or service down: since July, 56 passes failed this way, across 40 PRs. Now a PR leaves the loop only once validated by at least two different reviewers, on its final version.
11. Check every suspicion before fixing
AIs raise many false positives. So each suspicion is checked in the code before being fixed. Since September, 8,504 false alarms have been dismissed, and passed on to the next reviewers so they don’t come back to them.
12. A rule nobody reads is not applied
The process instructions had grown to 91,000 tokens, reread in full at every pass. We split them: each step loads only what it needs, and each pass runs in a fresh agent, free of the noise of the previous ones.
For developers
- Reviewers: Claude (Anthropic), Codex (OpenAI) and Grok (xAI), in a fixed order.
- Every fix follows TDD: a failing test first, then the fix. 2,979 fixes proven this way since July 13.
- Keep PRs small: in August, when their median size doubled, passes and issues climbed with it.
- Each pass leaves a summary line on the PR: reviewer, issues, false alarms, fixes. All the numbers in this article come from these lines.
Conclusion
Nine months and more than 600 PRs later, the verdict is clear: writing code costs almost nothing now; validating it costs more. A single AI reviewing is not enough. Several AIs from different vendors, governed by a strict process, review with a level of attention no human reviewer can sustain over hundreds of reviews.
This process comes at a price: more time and more tokens per PR, and settings to fine-tune continuously. Yet this is the kind of process that lets you hand the evolution of your production software over to AI with peace of mind.
Humans don’t disappear, though. They decide what to build, set the bar, and write and adjust the validation rules, while keeping control from end to end.
Delegate execution to AI, but keep the understanding!
P.S. Related work:
- “here’s how i shipped 2,500 PRs last month to production”, tweet by @poteto from the Grok Bot team (4+M views);
- “No AI Slop: A Manifesto”, written by the speakers and attendees of the AIE Code Summit 2026 (350 signatories).
We can set up this same approach with your dev teams, on your code and with your tools.
See the approach