Why AI Code Review Tools Can't Replace Senior Engineers (Yet)

The AI code review category has exploded in the last two years. CodeRabbit, Greptile, Codium, Diamond, Bito, Ellipsis, and at least a dozen others now offer some flavor of "AI reviews your PRs automatically." The pitch is intoxicating: senior-engineer-quality review on every commit, no waiting, no scheduling, no expensive humans.

We use several of these tools internally. They are genuinely useful for what they are. But they cannot do what their pitch decks claim — and the gap between marketing and reality is creating a class of bug that gets through review and lands in production.

This is a long-form take on what AI reviewers actually do well, what they structurally can't do, and how to make decisions about which kind of review belongs to which kind of PR.

What AI code reviewers genuinely do well

Let's start with credit where it's due. Modern AI code review tools are excellent at:

Style consistency and readability flagging. Catching inconsistent naming conventions, suggesting clearer variable names, identifying overly complex expressions that could be split up. This is high-volume, low-stakes feedback that humans don't enjoy giving and now don't have to.

Obvious bugs and code smells. Null pointer risks, unhandled error paths, dead code, unreachable branches, off-by-one errors in well-trodden patterns. The class of bug that a junior engineer might miss and a senior engineer catches in 30 seconds — AI reviewers catch most of these.

Documentation suggestions. Identifying functions without docstrings, suggesting comment text, flagging public API changes that should be noted in changelogs. Useful, low-cost, easy to ignore when wrong.

Test coverage observations. Flagging new functions without corresponding test cases, suggesting test scenarios, identifying when assertions look weak.

Pattern-matching against your own codebase. Tools like Greptile that index your codebase can flag when new code deviates from existing patterns — useful for keeping a codebase coherent.

If your engineering team isn't using at least one of these tools, you're leaving easy wins on the table. Most teams should adopt one tomorrow.

What AI reviewers structurally cannot do

The harder honesty is what these tools can't do — and not "can't do yet, but soon." Can't do for structural reasons that won't be resolved by a larger model or more training data.

1. They cannot understand business context they weren't told

An AI reviewer reading a PR sees the diff. It can see the surrounding code in the repo. It cannot know that this codebase processes healthcare claims under HIPAA, that the user_id field is also the medical record identifier, that the export endpoint is the one auditors flagged last quarter, or that the engineering team has an unwritten rule that PII never appears in error responses.

Some of this context can be encoded as rules. Most of it can't. The relevant context for a high-stakes review is what's not in the PR — the institutional history, the regulatory posture, the prior incidents, the engineering culture's hard-won lessons. Humans accumulate this context by being there. Models don't.

2. They cannot exercise judgment about trade-offs

Real code review is mostly judgment calls. This abstraction is leaky but the alternative requires a refactor we don't have time for; ship it but file a ticket. This query is technically n+1 but it only runs on admin endpoints and the data is small; not worth optimizing. This auth pattern is unusual but it matches the rest of this service which has its own reasons.

An AI reviewer cannot make these calls because it does not know which trade-offs are acceptable in this codebase, on this team, at this stage of company maturity. It can flag the technical issue. It cannot tell you whether to care.

3. They cannot take accountability

This is the structural one. When a Fairy-verified PR ships and something breaks in production, a human reviewer at Fairy stood behind it. We refund the review and help fix the incident. There is an accountable party. There is, in principle, recourse.

When an AI reviewer signs off and something breaks, there is no accountable party. The vendor's terms of service explicitly disclaim liability. The reviewer is a stochastic process. You can't sue a probability distribution.

For low-stakes code, this doesn't matter. For high-stakes code — anything touching money, identity, health, safety, or legal exposure — it matters enormously. The "trust anchor of last resort" cannot be a model. It has to be a human or an organization that can be held responsible. This is not a temporary limitation. It is foundational to how trust works.

4. They cannot verify the model's own work

The deepest problem: when you use Claude Code or Cursor to write code and then use a different AI tool to review it, you are asking one statistical pattern-matcher to grade the work of another. They share failure modes. They share blind spots. They both treat plausible-looking code as correct code.

There is reason to think AI reviewers catch a meaningful subset of AI-author bugs. There is no reason to think they catch most of them. The errors that one model is most likely to make are exactly the errors another model is least likely to flag, because both are sampling from overlapping distributions of what "good code" looks like.

A human reviewer steps outside that distribution. That's the whole point.

The framework: which review belongs to which kind of code

We recommend engineering teams adopt a tiered review model that uses AI tools where they shine and humans where they're irreplaceable.

Tier 1 (every PR): Automated tools

Style, formatting, linting, type checks, test coverage, common code smells, dependency vulnerabilities. Run this on 100% of PRs as CI. Block merges on failures. There is no judgment here; if the rules are clear, the machine should enforce them.

Tier 2 (most PRs): AI code reviewer

Suggestions, readability, obvious bugs, pattern consistency with the existing codebase. Treat the AI reviewer's comments the way you treat junior-engineer comments: useful, frequently right, sometimes wrong, never the final word.

Tier 3 (medium-risk PRs): One internal senior engineer

The reviewer reads the diff with attention. They confirm the business logic, the choice of abstractions, the trade-offs. They override the AI tool when its suggestions are misguided in context. This is the work that internal staff engineers should be spending their review time on — not the work AI tools already do.

Tier 4 (high-risk PRs): Specialized senior or external verification

For PRs that touch authentication, authorization, payments, data export, regulatory compliance, or other domains where mistakes have material consequences, the review needs to be done by someone with deep specialty in that area, with explicit accountability for the sign-off. This is where Fairy fits.

This framework is not novel. It's the structure most well-run engineering organizations had before AI tools existed, with the new tools slotted into Tiers 1 and 2 where they belong. The mistake we see most often is teams using Tier 2 tools and assuming they've covered Tiers 3 and 4.

How to evaluate AI code review tools

If you're choosing between AI code reviewers — or evaluating one you already use — the questions that actually matter:

1. Does it index your specific codebase, or does it review each PR in isolation? Tools that don't index your codebase can't flag inconsistency with existing patterns. They're working blind.

2. What's the false positive rate, and how easy is it to teach the tool to stop suggesting something? A reviewer that floods every PR with noise gets ignored, which means the valid findings get ignored too.

3. Does the tool know what to ignore in test files vs. production code? Reviewing test code with the same rigor as production code wastes engineer time.

4. Can it learn from team feedback? When you mark a suggestion as "not applicable here," does it generalize that lesson?

5. What's the latency? A review that arrives 15 minutes after the PR is merged is worthless.

The current generation of leading tools — Greptile, CodeRabbit, Codium — answer most of these well. The differences between them are real but smaller than the difference between using any of them and using none.

When to use Fairy vs. an AI tool

Use an AI code reviewer (Tier 2) when: you're shipping a medium-velocity codebase, the cost of a bug is recoverable, your internal seniors handle the high-stakes reviews, and you mostly want to catch the obvious-in-hindsight class of issue automatically.

Use Fairy (Tier 4) when: you're shipping code that touches data integrity, security, regulatory exposure, or money; your internal seniors don't have capacity for the volume; you need accountability for the sign-off; or you've had a recent incident that traced back to insufficient review.

Use both. They solve different problems.

Submit a PR for verification →

---

Related reading: Vibe Coding to Production: A CTO's Guide covers the broader framework. The AI-Generated Code Security Checklist covers what to verify on high-risk PRs.