Guide · AI coding ROI
How to Measure Claude Code ROI (When You Can't Tell If the Spend Is Working)
You bought Claude Code seats, the team says it feels faster, and now finance wants a number before renewal. Here is how to measure whether the spend is actually working, using signals you can trust instead of a vendor's acceptance rate.
The short version
- Acceptance rate measures typing, not ROI. METR found developers felt about 20% faster while measured about 19% slower, so instrument outcomes, not usage.
- Six numbers decide it: idle seats, PR throughput versus baseline, review-time trend, change-failure/revert rate, PR size, and spend versus value by team.
- Claude Code's admin and usage telemetry only supplies the adoption and idle-seat layer. The ROI-deciding metrics come from your own version control, CI/CD, and incident tools, so the method is identical for any tool.
- If your team cannot state its pre-AI baseline for review time and change-failure rate, that gap is the finding: instrument first, then scale, then renew on evidence.
Start with the question renewal is really asking.
The question is not "do engineers like Claude Code?" They almost always do. The question your CFO is asking is narrower and harder: for the dollars you are spending on seats, what attributable delivery gain came back, and by which teams?
That is a measurement problem, not a tool problem. And it has a specific starting point: your pre-AI baseline. If you cannot state, from memory or from a dashboard, your review time and change-failure rate from before the rollout, you cannot claim a delta. Most orgs cannot. That gap is not a failure of the engagement. It is the finding.
The good news: the method below is identical for GitHub Copilot, Cursor, Windsurf, or Claude Code. The tool only changes where the adoption signal lives. Everything that determines ROI happens downstream of the tool, in your own delivery system.
Why acceptance rate is the trap, not the metric.
The number every tool shows you first is a usage or acceptance signal: how often suggestions get taken, how active a seat is. It is comforting and it is nearly worthless as an ROI proxy. High acceptance tells you people are typing tab, not that your organization ships more or ships safer.
The reason is the productivity paradox, and it is measured, not anecdotal. In a 2025 randomized controlled trial, METR found experienced open-source developers were about 19% slower with early-2025 AI tools, while those same developers believed they were about 20% faster, even after the fact. Perceived speed and measured speed pointed in opposite directions.
So the first discipline is to distrust the feeling and the acceptance chart alike, and to instrument the outcomes those charts are supposed to predict.
The metrics that actually decide ROI.
Anchor on outcomes, then layer in AI-specific signals. Six numbers do almost all the work, and each maps to a source you can read without touching source code:
- Idle seats: licensed seats versus 30-day active seats. This is the fastest hard-dollar finding. Every dormant seat is money you can recover at renewal, and it usually hides in the admin console and your identity provider.
- Pull-request throughput versus a pre-AI baseline: merged PRs per developer per week, now compared to before. This is the real "is it working" number, and it is meaningless without the baseline.
- Review-time trend: how long code waits in review. This is where the value most often leaks. Independent-of-you but vendor-sourced telemetry from Faros AI (a measurement vendor, so treat the magnitude cautiously) has clocked review time rising 91% and, in a later study, roughly 5x as adoption deepens.
- Change-failure and revert rate: does more code mean more rollbacks? DORA's multi-year research shows throughput can improve while stability degrades, so watch this one closely.
- Spend versus attributable value, per team: seats times unit price on one axis, the throughput delta on the other. This single cross-tab is what a finance leader renews or cuts on.
- PR size trend: rising PR size plus rising review time means AI is producing batches too large to review safely, a workflow flag, not a tool win.
Where Claude Code's numbers live (and why the tool is the easy part).
For the adoption layer, Claude Code exposes the signal through its admin and usage telemetry: seats provisioned, seats actually active, and usage depth by user or team. Cross-reference that against your identity provider (Okta, Entra) to find licensed-but-dormant users, and against your actual invoice for seat count, unit price, and renewal date.
That is the entire tool-specific portion of the work. Everything that follows comes from systems you already own: pull-request volume, size, and review time from GitHub or GitLab insights; change-failure and revert rate from your incident tooling and version control; lead time and deploy frequency from CI/CD.
This is why the method is genuinely tool-agnostic. Swap Claude Code for any competitor and only the first data source changes. The downstream measurement, and the ROI answer, is identical.
Measure the perception gap on purpose.
The most persuasive slide in any ROI readout is not a throughput chart. It is the gap between what the team believes and what the data shows.
Run a two-minute pulse across the team: "How much faster do you feel with AI, as a percentage?" and "How much of that speedup survives review and rework?" Then put the answers next to your measured throughput and review-time trends.
When the felt number is +30% and the measured number is flat or negative, you have not embarrassed anyone. You have located the leak. The speed is real at the moment of writing code and it is being eaten downstream, in review, verification, and integration. That is a system to fix, not a tool to blame. One independent academic study backs the mechanism: Carnegie Mellon University, across 807 repositories, found Cursor adoption raised code complexity by roughly 41%. A vendor's own telemetry points the same way: CodeRabbit, an AI code-review vendor, reviewed 470 open-source pull requests and found the AI-coauthored ones carried about 1.7x more issues. More code arrives faster than the org's capacity to review it grows.
If you cannot state your baseline, that is the answer.
The honest landing point of any Claude Code ROI measurement is one of two outcomes. Either you can now show an attributable, baseline-anchored delta by team, and you know whether to keep, right-size, or expand. Or you discover you were never instrumented to answer the question in the first place.
The second outcome is not a dead end. It is the sharpest possible finding, because it means every renewal decision so far has been made on vibes. The fix is small: instrument review time, change-failure rate, throughput, and seat activity before you scale any tool further. Then the next renewal conversation runs on evidence.
Whichever outcome you land in, keep the source discipline that makes the number trustworthy: anchor every benchmark to independent research (METR, DORA, Carnegie Mellon University), label any measurement-vendor figure as vendor telemetry, and never repeat a 10x claim as fact. The honesty is not a nicety. It is the only thing that makes the ROI number worth acting on.
See where you stand.
A free thirty-minute call, or a self-check you can run in a minute. Either way you leave with the next honest move on your AI coding spend, even when that move is not hiring me.
Straight answers
Questions leaders ask.
Isn't Claude Code's acceptance rate a good enough ROI signal?
No. Acceptance and usage rates tell you people are using the tool, not that your organization ships more or ships safer. METR's 2025 randomized trial found developers who felt about 20% faster were measured about 19% slower, so a usage chart can move in the opposite direction from real delivery. Measure outcomes (throughput versus baseline, review time, change-failure rate), and use the admin telemetry only for the adoption and idle-seat layer.
We never recorded a baseline before rolling out Claude Code. Can we still measure ROI?
Partially, and the gap itself is the most valuable finding. Without a pre-AI baseline you cannot claim a clean throughput delta, but you can still surface idle-seat waste, current review-time and change-failure trends, and the perception gap. The honest recommendation becomes: instrument these four signals now, before scaling further, so the next renewal decision runs on evidence instead of feel.
What is the fastest number that pays for the exercise?
Idle seats. Comparing licensed seats against 30-day active seats in the admin console and your identity provider almost always surfaces dormant licenses you can cut or renegotiate at renewal. That hard-dollar saving frequently covers the cost of the measurement work on its own, before any throughput analysis.
Is this method specific to Claude Code?
No, it is tool-agnostic. Only the adoption data source changes between Claude Code, GitHub Copilot, Cursor, or Windsurf. Everything that determines ROI, throughput versus baseline, PR size, review time, change-failure rate, and spend versus value by team, comes from your own version control, CI/CD, and incident systems, and is measured identically regardless of which tool you bought.
Where do the ROI benchmark numbers come from, and can I trust them?
Anchor to independent research: METR's randomized trial on perceived versus measured speed, DORA's multi-year study showing stability can degrade as throughput rises, Carnegie Mellon University on code complexity, and McKinsey on workflow redesign as the top correlate of impact. Measurement-vendor figures like those from Faros or DX are useful but self-interested, so label them as vendor telemetry and never present them as independent. Treat all of it as directional, since the evidence is young and moves fast.