diffox architecture: 17 specialist agents, four stages, one review
diffox splits an AI code review across 17 specialist agents. Every finding comes with a quote from the code, and code, not a model, checks that quote.
Summary. diffox is an AI code review tool that reviews pull requests on GitHub. A review goes through four stages, and this post walks through what happens at each one and why. The first two posts measured the results: diffox ultra ranks first among 23 tools on Martian’s benchmark, and a diffox review costs $0.41.
- Context & routing. diffox gathers the change, its files before and after, and the rules you have written. The kind of change and its risk decide which specialist agents read it.
- Specialist review. The team has 17 specialist agents. Each one looks for one thing and backs up what it finds with a verbatim quote from the code.
- Evidence checks. Code, not a model, checks each finding. Is the quoted code really in that file? Is the finding about something this pull request changed? Was the same problem reported twice?
- Select & publish. Every finding has a class: Critical, Warning or Suggestion. The pull request gets what you need to fix: critical findings and warnings. Suggestions are not lost; they wait on the Findings page. A review never blocks a merge.
How a review works
The quickest way to review a change is to ask a model: “Is there a bug in this?” diffox does not do that. It has separate specialist agents read the same change: one looks for logic errors, one for concurrency problems, one for tests that do not really test anything. Each agent’s job is narrow and clearly defined. Two more steps follow. Checks tie each finding to the file itself rather than to the model’s word, and a selection step decides what gets posted.
The first post measured what this design delivers: on Martian’s benchmark of 50 pull requests, diffox ultra caught 119 of the 158 known bugs and diffox caught 106. The second post covers what it costs. Here are the four stages, in order.
What the agents see, and which ones run
A review starts without you asking: when a pull request is opened, and on every new push. diffox first posts a comment on the pull request saying the review has started; when the review ends, that same comment turns into the result. Before the specialist agents see the change, two things are settled: what to put in front of them, and which of them to call.
Each of the four parts answers a different question. The diff says what changed. The files, before and after, show where the change sits. The title, the description and the commit messages say what the author meant to do. And AGENTS.md or CLAUDE.md, if your repository has one, says which rules your team follows.
The rules file is read from the branch the pull request will merge into, not from the pull request’s own branch. That way a pull request cannot change the rules of its own review: a change to the rules file takes effect once it is merged. Lock files, third-party code such as vendor/ and node_modules/, and minified files are not reviewed.
Then comes routing. diffox works out what kind of change it is (a bug fix, a new feature, a refactor, a migration, a test…) and looks at its risk: how many files it touches, whether it affects other files, whether tests come with it. Some agents read every change; others run only when their subject comes up. If no line was deleted, the agent that looks for removed behaviour has nothing to do. If no test was touched, neither does the agent that looks for empty tests.
When a new push arrives on the same pull request, diffox reads only the new change.
Each specialist agent looks for one thing
In the second stage, the specialist agents read the change. The team has 17 agents, of two kinds depending on what they look for.
12 agents look for bugs: code that gives the wrong result when it runs. 5 look for contradictions: places where the code disagrees with the comment, test or document next to it. Each agent has its own instructions, and all of them read the same change and the same context.
To count, a finding has to say three things: what it claims, what goes wrong and when, and the code the claim is based on, quoted verbatim. A contradiction finding carries two quotes: the code, and the text that contradicts it. If any of these is missing, the finding does not count.
Agents do not give line numbers. Language models are unreliable at counting lines, so diffox asks the model for the quote instead, and finds the line itself by searching the file for that quote. That means whether a finding lands under the right line does not depend on what the model says.
Rule-based detectors run alongside the specialist agents. They use no model and search with exact rules: a credential leaked into the code, a method defined twice in the same class, a branch that can never be reached.
The whole team, and what each agent looks for:
| Agent | Kind | Looks for |
|---|---|---|
| ➗ Logic | Bug | Hunts off-by-one errors, boundary conditions, inverted conditions, wrong operators and violated function contracts. |
| ⭕ Null Path | Bug | Hunts a value that can be absent where the code reads it as present: a None, a null or an empty result used as if it held something. |
| 🔀 Race | Bug | Hunts race conditions, deadlocks, non-atomic read-modify-write, unsafe shared state and lock ordering hazards. |
| 🔇 Silent Failure | Bug | Hunts swallowed exceptions, overly broad catches, ignored return values and fake fallbacks that mask errors. |
| 💾 Data Integrity | Bug | Hunts schema mismatches, backward-incompatible migrations, lost writes and non-idempotent operations. |
| 🔓 Security | Bug | Hunts injection, broken authentication and authorisation, secret leakage, SSRF and unsafe deserialisation or path handling. |
| 💥 Blast Radius | Bug | Hunts broken callers, changed API surfaces and signature or return-type changes that break other files in the change graph. |
| 🐛 Regression | Bug | Hunts the re-introduction of bug patterns previously fixed in these files. |
| 🥅 Safety Net | Bug | Hunts a contract break between the change and the tests that certify it: behaviour a test asserts that this diff changed without updating the test. |
| 🔥 Hot Path | Bug | Hunts N+1 queries, needless allocations in hot paths and algorithmic regressions. |
| 🏷️ Naming | Bug | Hunts four demonstrable classes only: a typo in an identifier that is wrong against a sibling in the same diff, two spellings of one key, a label that names the wrong thing, a test named for something it never runs. |
| 📐 House Rules | Bug | Hunts lines in this diff that violate the review rules the repository wrote down itself in AGENTS.md or CLAUDE.md, when it has them. |
| 💬 Broken Promise | Contradiction | Hunts text that makes a claim the adjacent code does not honour: a comment, docstring or README that says something the code does not do. |
| 🪦 Dead Code | Contradiction | Hunts code whose effect cannot be observed: unreachable after this change, or duplicated. |
| 📜 Stale Contract | Contradiction | Hunts a declaration that changed here while something in view still speaks the old shape. |
| 🗑️ Lost Behaviour | Contradiction | Hunts behaviour this change deleted without replacing, and asks whether anyone still depends on it. |
| 📭 Empty Test | Contradiction | Hunts a test that does not exercise what it claims: it would pass whatever the code did. |
Each finding’s comment carries the mark and the name of the agent that found it, for example ➗ Logic Agent.
How each finding is checked against the code
A language model can write something convincing and wrong. It quotes a line that is not in the file, objects to code this pull request did not touch, or raises the same problem three times in three different sentences. The third stage ties every finding to the file itself, not to the model’s word.
These checks are plain code, not a model call: the same input gets the same answer every time.
- Anchor Verifier. The finding’s quote is searched for in the file. A quote that cannot be found has no line to point to. If the line it is found on is outside what this pull request changed, the finding is out of scope.
- Second Witness. Findings that raise the same objection at the same place are merged into one, so you are not told about the same problem twice. And when more than one source points at a place, that counts in the finding’s favour.
- Verbatim Matcher. A contradiction finding’s second quote is also searched for word for word: in the change, the files, the pull request’s description, the commit messages and your rules file. If the model claims a comment says something it does not, this is where it shows.
There is one more check: a finding that says a value may be empty is refuted when the value’s type, or a check a few lines above, already rules that out.
Which findings are posted?
After the checks there is a list of candidates. The fourth stage decides which of them are posted on the pull request.
Attention Ranker answers one question about each candidate: how likely is this finding to be a real bug? The score takes in the finding’s severity and kind, the result of the evidence checks, and whether other findings point at the same place. Among findings that fall on nearby lines, only the strongest is posted.
On the pull request you see only what needs fixing: critical findings and warnings. Suggestions wait on the Findings page, and there is no setting to tune for it.
We measured the numbers below on the same benchmark at three sensitivity settings. That setting was a threshold on a score. High caught the most bugs. Medium and Low raised the threshold: diffox posted fewer findings, and was surer of the ones it posted.
| Sensitivity | diffox | diffox ultra | ||
|---|---|---|---|---|
| Real bugs | False alarms | Real bugs | False alarms | |
| High | 106 | 132 | 119 | 144 |
| Medium | 89 | 55 | 98 | 51 |
| Low | 59 | 19 | 70 | 19 |
Martian’s benchmark: 50 pull requests, 158 known bugs, judged by Claude Opus 4.5. High sensitivity: our public run, 7 October 2026. Medium and Low: our internal measurement on the same 50 pull requests, the average of 5 runs for diffox ultra and 2 for diffox.
What is not posted on the pull request is not lost: it waits on the Findings page in your console for 30 days, and you can publish any of it from there.
Publishing has two parts: one summary comment on the pull request and, for each finding, a comment right under the line it is about. When a finding can be fixed, it carries a suggested fix that GitHub applies in one click, and the commit is made in your name. Titles and suggested fixes are written after the selection, only for the findings that will be posted. The review arrives in your team’s language. The last line of the summary shows the review’s price, how long it took and the commit it read.
diffox’s status on a pull request never turns red: whether to merge is your team’s call. With every 👍 and 👎 on its comments, diffox gets to know you and your repository better, and its reviews fit your team more closely over time.
What diffox ultra does differently
diffox ultra goes through the same four stages; the difference is in the second and the fourth. Two separate models read the change, each running its own review from start to finish. The two reviews run side by side and are published as one. In the selection step a third, small decision model also runs: it writes no text and answers yes-or-no questions about each finding. The selection scores the findings of both reviews together with those answers. When both models point at the same place, that counts in the finding’s favour. After the selection, the decision model finds the findings that trace back to the same root cause and merges them into one.
The results are in the first two posts: diffox ultra caught 119 of the 158 known bugs and diffox caught 106; a diffox ultra review costs $1.64 on average, and a diffox review $0.41.
Frequently asked questions
Why does diffox use specialist agents?
“Is there a bug in this?” is a broad question, and it gets a broad answer. Each specialist agent looks for one thing and has to back up what it finds with a quote from the code. So you can see which agent every finding came from.
Can I give diffox my own review rules?
Yes. The rules you write in AGENTS.md or CLAUDE.md at the root of your repository are read in every review, and a separate agent looks for lines that break them. The rules come from the branch the pull request will merge into, so a change to the rules file takes effect once it is merged.
What do 👍 and 👎 do?
With every reaction, diffox gets to know you and your repository better: it learns what matters to you and what is noise. Its reviews fit your team more closely over time.
Does diffox block a merge?
No. diffox’s status on a pull request never turns red and it is not a required check; your team merges whenever it wants.
Is the whole pull request reviewed again on every push?
No. diffox reviews every push automatically but reads only the new change. After the first 5 pushes it reviews when you ask; you can change that number in the console.
Why are suggestions not posted on the pull request?
In our measurements, posting suggestions too added comments to the pull request and lowered the share of real bugs among them. So the pull request gets critical findings and warnings. Suggestions are not lost: they wait on the Findings page for 30 days, and you can publish any of them from there.
Does diffox keep my code?
No. diffox does not keep a copy of the code it reviews; the copy of the repository is deleted when the review ends. The services involved are listed on our privacy page.
The decision model, explained
The fourth post describes the decision model diffox ultra used in the fourth stage at the time, and the rule we drew from our measurements: How we use Jev for two jobs in AI code review.