How to review AI-generated code

Coding agents write more code, and review takes longer

These four numbers come from a single study. Two Harvard University researchers followed 718 firms from January 2021 to March 2026, using data from an engineering analytics platform (the August 2026 version of the paper). A coding agent is a tool that takes a task, writes code across several files, runs the tests and keeps going until the work is done. At firms that adopted these tools, lines of code added rose by 30%, commits by 20% and pull requests by 23%.

Finished work did not grow at the same pace: the rise in resolved issues is small and not statistically significant. The same study explains the gap. The time from opening a pull request to merging it grew by 49%, the share of pull requests where changes were requested nearly doubled, and review comments per pull request rose by 35%. The authors’ conclusion: “code review becomes the bottleneck.”

Other measurements point to the same place. In LinearB’s 2026 report (more than 8.1 million pull requests), AI-generated pull requests wait 4.6 times longer before anyone starts reviewing them, and are reviewed twice as fast once picked up. In Sonar’s survey of more than 1,100 developers, 96% of respondents don’t fully trust that AI-generated code is correct; 48% say they always check it before they commit.

We build diffox, a tool that reviews pull requests. The list below comes out of that work and out of our own repository’s rules, and you can apply it by hand, without any tool.

AI-generated code sounds equally sure of every line

When you review a colleague’s code, you know the author. You can guess what they know well and where they rushed, and you can ask about the parts you’re unsure of. When the author is a language model, none of that exists.

A study JetBrains Research published in October 2026 describes the problem this way. A model presents every line it generates with the same confidence; nothing tells you the authentication logic was easy and the database migration was hard. Reading every line doesn’t scale as the changes agents produce get bigger. According to the researchers, the issue is not reading the diff but trust calibration: being able to spread review effort in proportion to each part’s risk. The study rests on four workshops with 17 practitioners and a survey of 43 software professionals.

The usual warning signs don’t work either. In code a person wrote, a messy function or a half-finished comment tells you “look here”. AI-generated code usually compiles, its names are tidy, and it comes with comments and tests. In Sonar’s survey, 53% of developers list exactly this among AI’s negative effects on technical debt: code that looks correct but isn’t reliable.

What should a code review checklist include?

The list suggests an order. You start answering the first two questions before you open the code, and they tell you where to look; the third tests the evidence; the last two each need a reading of their own. The JetBrains study points the same way: the overview first, then the files, the lines last.

The list was put together for this post. Its sources: GitHub’s “Review AI-generated code” guide, Google’s code review guide, and the rules we apply in our own repository.

Read the description, the comments and the file list before the code

For the first question, read the pull request’s description as a list of claims. If it says “retries a failed request three times”, find the three attempts in the diff. Read the comments the same way: a comment may describe what the code should do, not what it does. An example:

def fetch_prices(symbols):
    """Fetches prices; retries three times on a network error."""
    try:
        return client.get_prices(symbols)
    except Exception:
        return []

This function compiles and passes a happy-path test. It has two problems. The docstring promises three attempts; the code tries once. And the error is swallowed: even if the service is down, the caller gets an empty list and reads it as “no prices”.

For the second question, look at the file list before you open the code. If the task is to add a field to an endpoint, the description should say why an unrelated module changed. Then read the deleted lines: an added line catches your eye, a deleted validation or log line doesn’t. For every behaviour that was removed, check whether anything still depends on it.

The third check is the path an error takes. The rule we keep in our own repository: a caught error must reach the caller. A broad except, a fallback that returns a default value, or an ignored return value doesn’t fix the error; it hides it.

A test that passes isn’t necessarily testing anything

A coding agent runs the tests itself and keeps going until they pass. So in an agent’s pull request, “the tests pass” is the starting point, not proof on its own. Google’s code review guide asks the same thing: “Will the tests actually fail when the code is broken?” An example:

def test_discount_is_applied():
    order = make_order(total=100)
    apply_discount(order, code="WELCOME10")
    assert order is not None

This test passes even if the discount is never applied. Its name is about the discount; its assertion only checks that the order exists. The fix is one line: assert order.total == 90.

The first check is called mutation verification: you revert the fix and watch the test fail. For the second, one distinction helps: a search is not an assertion. A test that says “the error message is not in the output” also passes when the message is never produced. It needs a control case that shows the search can find the message at all.

Test data is part of the same problem. An example made up by hand tests how the author imagines the data. Take the example from a real record, or compare it with one.

Code that compiles still needs a security read

According to Veracode’s July 2026 report, code generated without security guidance almost always compiles, but its average security pass rate across more than 100 models is 56%. The rate varies by language: 63% in Python, 30% in Java. The report tests the models on their own, not an agent with tools, guardrails or human review around it. The conclusion is still clear: security needs a reading of its own.

In that reading, look in three places. Does outside input reach a query, a shell command or a file path without being sanitised? Does a new endpoint carry the same authorisation check as the ones next to it, and has a secret been written into the code or a log? Last, open every new dependency, one by one.

A study accepted at USENIX Security 2025 looked at 576,000 code samples generated with 16 models. On average, at least 5.2% of the packages suggested by commercial models and 21.7% of those suggested by open-source models did not exist. That is why GitHub’s guide suggests checking whether a suggested dependency exists, whether it is still maintained, and what its licence is.

Review the change against the repository’s written rules

The fifth question doesn’t ask whether the code is correct, but whether it belongs in this repository. If your rules are written in a file (AGENTS.md, CLAUDE.md or CONTRIBUTING.md), keep that file open while you review, and read the change against it.

The first of the other two checks is duplication: someone may have written a new helper when the repository already has one that does the same job. In Sonar’s survey, 40% of developers also list unnecessary and duplicative code among the negative effects. When you see a new helper function, search the repository for an existing one.

The second is that both halves of an operation come from the same value. If a function checks validity against one list and acts on another, or reads the clock twice in two places, each half looks right on its own; the bug is in how they relate. Our own rule: resolve the value once and pass it to both halves.

Keep pull requests small and spend attention where the risk is

Fifteen checks are hard to apply to a thousand-line pull request. Google’s guide considers about 100 lines a reasonable size and 1,000 lines usually too large, and lets a reviewer turn down a change for its size alone. The Harvard study found no significant change in pull request size after coding agents; the baseline mean was about 1,168 added lines per pull request. Asking the agent for small pull requests that each do one thing is the first step of review.

Even in a small pull request, not every file needs the same attention. The order the JetBrains study proposes is to sort the files by risk before you read a single line.

Some of the list can be left to a tool, and some can’t:

• Leave it to a tool. The build, the tests, the linter, static analysis, secret scanning, dependency and licence checks.
• Let a tool flag it, and decide yourself. A comment that contradicts the code, a swallowed error, an empty test, a line that breaks the rules file. Whether what was found is really a bug is up to someone who knows the context.
• Keep it for yourself. Whether the change solves the right problem, whether the design fits the repository, and the merge decision.

diffox works at the second of these three levels. It has the pull request read by specialist agents, each looking for one thing. Four items on the list show up in comments under the names of four agents: Broken Promise Agent for a comment that says something the code doesn’t do, Silent Failure Agent for a swallowed error, Empty Test Agent for a test that passes whatever the code does, and House Rules Agent for a line that breaks your rules file. diffox doesn’t block a merge; the merge decision is your team’s (a separate post explains how it works).