AI tools for developers: which ones earn a place in the stack

Most roundups of AI tools rank things nobody measured. This one starts from the other end, with the three numbers you can actually check: a shared benchmark score, a printed price, and the research on whether any of it makes developers faster. On that evidence the picture is tighter than the marketing suggests. Three coding agents finish within 3.4 points of each other on the same task suite, the subscriptions that run them go from free to $100 a month, and the best controlled study of whether they help still can’t separate the effect from zero.

So the question worth asking is narrower than “what’s best”. It’s what earns a place in a stack you already have, given that the second tool costs money the first one already spent. Everything below comes from pages we opened in August 2026, and the prices are the ones printed on the vendors’ own pricing pages that month.

Three agents, one task suite, and a 3.4 point spread

The cleanest same-task comparison available in public is Terminal-Bench, which its authors describe as a collection of harbor-native benchmarks built to quantify an agent’s terminal mastery. Version 2.0 carries 89 tasks across software engineering, machine learning, security, data science and system administration. Agents are scored on task resolution success rate, so a run either fixed the thing or it didn’t.

That matters because it puts the harness and the model together, which is how you actually use these things. The 2.1 leaderboard carried 17 submitted entries when we read it, and the top of it looked like this.

Agent and model	Accuracy	Gap to leader
Claude Code with Fable 5	83.8% ± 1.2	leader
Codex with GPT-5.5	83.1% ± 1.1	0.7 points
Terminus 2 with Fable 5	80.4% ± 1.2	3.4 points

Two readings fall out of that table, and both cut against tool loyalty. The first is that the leader and the runner-up sit inside each other’s error bars, so the ranking between Claude Code and Codex isn’t a real result. The second is the loser, and it’s the interesting one: Terminus 2 runs the same Fable 5 model as the leader and lands 3.4 points behind it. Same model, different harness, measurably worse. The scaffolding around the model is doing work, which is the argument our piece on how coding agents actually work makes at length.

The usual caveat applies with force here. Vendors submit their own runs, benchmark tasks leak into training data over time, and 89 tasks is a small sample to hang a purchasing decision on. We’ve written before about why AI benchmarks keep lying to you, and none of that stops being true because the numbers are close.

What the subscription costs before a single token

Capability is the part everyone compares. Price is the part that decides what survives a renewal, so here it is from the three vendors’ own pages.

Product	Free tier	Entry paid tier	Top individual tier	Team or business
GitHub Copilot	$0, 2,000 completions a month	Pro, $10/mo, $15 monthly credits	Max, $100/mo, $200 monthly credits	Business $19/user/mo, Enterprise $39/user/mo
Cursor	Hobby, limited agent requests	Pro, $20/mo	Ultra, 20x Pro agent limits, price not printed	Teams, $40/user/mo
Claude	$0, 50% of weekly limits	Pro, $20/mo monthly or $17/mo annual	Max 20x, from $100/mo	Team seat $25/mo monthly or $20 annual

The shape of that table is more useful than any single row. Entry pricing has converged on $10 to $20 a month, which means the first tool is close to a rounding error against a developer salary. The spread opens at the top, where Copilot prints $39 a month for Pro+ with $70 of monthly credits and $100 for Max with $200 of credits, and where Claude starts its Max tiers at $100 a month.

Which brings the honest gap in this comparison. Cursor’s pricing page prints a figure for Hobby, Pro, Teams and Enterprise, but it doesn’t print one for Pro+ or Ultra, describing them only as 3x and 20x the Pro agent limits. GitHub’s page gives credit allowances for the individual plans and refers to pooled credits for Business and Enterprise without a number. We’re not going to estimate either, because a price you had to guess isn’t a price.

Entry pricing has converged on $10 to $20 a month. The real bill is the tokens underneath it.

Rundowns AI

The token bill underneath the subscription

Every credit-metered plan is a token bill wearing a flat-rate costume, so the API sheet tells you what exhausting your credits actually buys. Anthropic’s published rates put Claude Opus 5 at $5 per million input tokens and $25 per million output, Sonnet 5 at $2 and $10, and Haiku 4.5 at $1 and $5. On OpenAI’s sheet, gpt-5.6-sol is $5 in and $30 out, gpt-5.6-terra is $2 and $12, and gpt-5.6-luna is $0.20 and $1.20.

Two things in those numbers change how you budget. Output costs five to six times input on the flagship models, and agents generate a lot of output because they write files, run commands and read the results back. The counterweight is caching: Anthropic prices a cache hit at 0.1x the base input rate, and OpenAI lists cached input for gpt-5.6-sol at $0.50 against $5.00 standard.

That’s a tenfold discount on exactly the tokens an agent re-sends most, because a coding agent reads the same repository context on every turn. Anthropic’s batch API takes a further 50% off input and output for work that doesn’t need an answer now. The upshot is that a tool’s effective cost depends more on whether its harness caches well than on the sticker rate, and no vendor publishes that. Our model pricing comparison and the worked example of running an AI feature go through the arithmetic properly.

The productivity evidence is thinner than the pitch

Here’s where a comparison piece has to be careful, because the strongest claim made for these tools is the one with the weakest public evidence behind it. The most rigorous attempt is METR’s randomised controlled trial, and its results have moved.

The first run, published on 10 July 2025, took 16 experienced developers, recruited from large open-source repositories, through 246 issues. Developers took 19% longer to complete issues when AI tools were permitted. They had predicted a 24% speedup beforehand, and after finishing they still estimated AI had sped them up by 20%. Most were using Cursor Pro with Claude 3.5 or 3.7 Sonnet, frontier models at the time.

METR now flags that page with a warning that the results are out of date. The February 2026 update is bigger and much less quotable: 57 developers, 143 repositories and more than 800 tasks, made up of 10 returning participants and 47 new recruits. The reported estimate is a speedup of -18% for the returning group, with a confidence interval between -38% and +9%, and -4% among the new recruits, between -15% and +9%.

Both of those intervals cross zero, which means the study cannot distinguish the effect from nothing at all. METR is blunt about why, reporting that 30% to 50% of developers avoided submitting tasks because they didn’t want to work without AI. That selection effect biases the sample toward people the tools help least.

Due to the severity of these selection effects, we are working on changes to the design of our study.

METR, developer productivity experiment update, February 2026

That’s the state of the best evidence, and it’s worth sitting with. The organisation that produced the most-cited number against AI coding tools has said its current design can’t measure the effect reliably. Anyone selling you a percentage uplift is working from something weaker than that.

Developers use these tools without trusting them

The survey data is where the picture gets clearer, because adoption and confidence have come apart. Stack Overflow’s 2025 developer survey found 84% of respondents using or planning to use AI tools, up from 76% the year before, with 51% of professional developers using them daily.

Trust went the other way. More developers actively distrust the accuracy of AI tools, at 46%, than trust it, at 33%, and only 3% report high trust. The top frustration, cited by 66%, is “AI solutions that are almost right, but not quite”, and 45.2% said debugging AI-generated code is more time-consuming than writing it themselves.

Google’s 2025 DORA report, drawn from nearly 5,000 technology professionals, lands in the same place from a different angle. It found 90% using AI at work and more than 80% believing it increased their productivity, while 30% reported little or no trust in the code it generates. People are shipping code they don’t believe, which is a very specific kind of tool adoption.

Almost-right is the expensive failure mode, because it survives a glance and dies in review or in production. That’s the cost these tools shift rather than remove, and it lands on whoever reads the diff.

Where the saved time reappears as cost

DORA measured that shift directly. Unlike the previous year, the 2025 report observed a positive relationship between AI adoption and both software delivery throughput and product performance. The stability finding didn’t follow.

AI accelerates software development, but that acceleration can expose weaknesses downstream. Without robust control systems, like strong automated testing, mature version control practices, and fast feedback loops, an increase in change volume leads to instability.

2025 State of AI-assisted Software Development, DORA

The report’s own summary of the pattern is that AI doesn’t fix a team, it amplifies what’s already there. Strong teams get better, and struggling teams get their existing problems intensified. Which reframes the buying question completely: the constraint on value isn’t the tool’s benchmark score, it’s whether your review, test and release path can absorb more change per week.

That’s the same argument we made about agent reliability, and it explains why the tool that earns a place is often the boring one. A test runner an agent can call, or a review bot on the pull request, does more for throughput than a better autocomplete, because it widens the bottleneck instead of feeding it.

What earns a place, and what we couldn’t check

Pulling the fetched evidence together gives three tests a tool has to pass, and none of them is “is it the best”.

The first is whether it beats what you already run on a task you can check yourself. The Terminal-Bench spread says a second agent bought for a 0.7 point benchmark edge is buying noise, while the 3.4 point gap between two harnesses on the same model says the harness is worth switching for. The second is whether the price survives real usage, and on credit-metered plans that means asking how well the tool caches, since a cache hit runs at a tenth of the input rate. The third is whether it adds load to a review path that’s already the constraint, which is the failure DORA measured.

Four things we could not verify, and they matter. Cursor doesn’t print Pro+ or Ultra prices. GitHub doesn’t print credit allowances for Business or Enterprise. No vendor publishes cache hit rates for its own harness, so the effective token cost of each tool stays private. And METR, the one group running a controlled trial, has said its current design can’t measure the productivity effect reliably.

What would change this piece is a benchmark run by someone who isn’t submitting their own agent, a redesigned METR trial that closes the selection gap, or a vendor publishing cache hit rates alongside its price. Until then the printed numbers are the ones above, and the gap between the top two is smaller than the gap between a good review process and a bad one.