Adversarial multi-agent debate
We had AI agents critique each other’s plans and code in loops. They agreed too fast, and the debate ended worse than the strongest model’s first answer. Dropped.
Multi-agent debate is a popular idea: one agent proposes, another attacks, they iterate, and the answer that survives should be better than either first draft.
We tried it on our own work: our platform and the internal projects where AI writes the code. The targets were the decisions that matter most, plans and code.
Step by step.
- 01
One agent wrote a plan or a change. Another agent critiqued it. The first answered the critique and revised, and the loop repeated.
- 02
Some pairings put a strong model against a weaker one, to see whether a second opinion added anything.
- 03
We compared the final answer with the strongest model's first answer on its own.
- 04
The loop made it worse. The answers that came out of the debate were worse than the strongest model’s first answer on its own.
Built with AI agents reviewing each other (dropped) · Independent AI reviews
Agents agree too easily. The loop ended on consensus, and consensus is not the same as a correct answer.
Mixed strength makes it worse. A strong model debating a weaker one usually ended worse than the strong model's first answer.
Every round adds tokens and time. The quality did not rise with them.
What we decided.
Dropped. Each decision now goes to the most capable model, and we follow it. Reviews still happen, but as independent passes: each reviewer checks a slice against the code and the plan, without negotiating with the author. The findings go back to the strongest model, which decides what to fix.
You don't pay for agents talking each other into agreement. Your project's design calls are made by the strongest model available, checked by independent reviews against the actual code, and verified by a person. Fewer tokens go to conversation, more to checks that find real defects.
- This verdict is about loops where agents answer each other. Independent one-pass reviews stay.
- A reviewer gets a defined slice and a checklist, not an open debate.
- Disagreement is settled by evidence in the code, or by a person.
We keep independent reviews and measure them by what they catch that tests missed.
Does multi-agent debate improve AI code quality?
In our tests, no. Agents converged on agreement fast, and mixing a strong and a weaker model ended worse than the strong model alone.
What works better than agents critiquing each other?
The most capable model decides, independent reviewers check slices of the work against the code, and a person verifies.