Together AI: DeepSeek-first cascade beats GPT-5.6 Sol by 10 points

Together AI published two DeepSWE comparisons this week, and both reached the same conclusion: put the cheap model first. Running DeepSeek V4 Pro 0813 ahead of GPT-5.6 Sol, and escalating only when the test suite rejects the answer, solved 83.0% of DeepSWE tasks at $3.35 apiece. Sol working alone solved 72.7% at $8.37.

That’s ten points more accurate for under half the price, and the cheap first stage is why. Pro costs $0.24 a rollout, roughly 35 times less than Sol, so the flagship’s price only lands on tasks the cheap model couldn’t clear. Those survivors also get a second independent attempt, which is how the cascade beat a perfect one-shot oracle router at 80.8%.

DeepSeek’s V4 Pro 0813 is the cheap side of that trade, a mixture-of-experts model that went generally available on 12 August 2026 with a 1M token context window, according to its OpenRouter listing. Together AI ran it at max effort against each rival on all 113 DeepSWE tasks, four trials each, for 904 rollouts in total.

The benchmark carries more weight here than most. DeepSWE is 113 original tasks written from scratch across 91 active open-source repositories and five languages, graded by hand-written verifiers instead of the tests that shipped with a merged fix. The paper reports an independent LLM judge disagreeing with those verifiers 1.4% of the time, against 32.4% for SWE-Bench Pro’s inherited tests. Its tasks, verifiers and evaluation trajectories are published on GitHub under Apache-2.0.

Single-shot, Sol looks clearly better at 72.7% pass@1 to Pro’s 62.8%. Give each model four attempts and the order flips, with Pro’s 88.5% pass@4 clearing Sol’s 85.8%. The cheap model reaches more of the board, it just needs more tries, and its tries are nearly free.

Model	pass@1	pass@4	Cost per rollout
DeepSeek V4 Pro 0813 (max)	62.8%	88.5%	$0.24
GPT-5.6 Sol (max)	72.7%	85.8%	$8.37
Claude Fable 5 (max)	69.7%	84.1%	$21.63

The Claude Fable 5 matchup runs the same shape at a wider spread. Fable takes the first attempt at 69.7%, loses pass@4 at 84.1%, and costs $21.63 a rollout, which is 90 times Pro’s price. Pro first with Fable on failure reached 82.7% at $8.28 a task. Order isn’t optional either, because Fable-first costs $21.71 for the same accuracy.

Failure profiles pull them further apart. Sol breaks tests that already passed in 20% of its failures, against 11% for both Pro and Fable, so Sol’s diffs need a regression gate before anyone accepts them. Fable’s misses are the costlier kind to debug, at 18% big misses against Pro’s 10%.

So Pro does not add much new reach on top of Sol; what it adds is the ability to clear the shared tasks at minimal cost.

Zain Hasan and Shobhit Dixit, Together AI

Two caveats sit on top of the numbers. Together AI sells hosted inference for these models, so it isn’t a neutral referee, and every figure comes from its own run rather than the published board. The public DeepSWE leaderboard is close but not identical, listing gpt-5.6-sol at 73% and claude-fable-5 at 70% pass@1, with claude-opus-5 ahead of both at 74%.

The error bars matter too. Together AI puts Sol at 72.7% plus or minus 2.2% and Pro at 62.8% plus or minus 3.1%, so the single-shot gap is real but not vast. That’s the familiar difficulty in reading model pricing against benchmark scores, where near-identical numbers can arrive at five times the bill.

What’s worth watching is whether the cascade survives outside a benchmark with a clean verifier. The method rests on a test suite good enough to reject a bad answer, which is exactly what most teams don’t have. Coding agents that can’t tell a pass from a fail have nothing to escalate on, and $3.35 a task quietly assumes they can.