Databricks Proteus writes GPU kernels 1.8x to 5.2x faster than vLLM

Databricks says an agent harness it built, called Proteus, generated GPU kernels for Qwen 3.5 122B that ran 1.8x to 5.2x faster than the best implementations available in vLLM. The company published the work on September 4, credited to Leo Li, Daya Khudia and Lesheng Jin.

The argument behind it is specialization. Production inference systems lean on generic kernels, and Databricks calls that suboptimal because an operation’s shape depends on both fixed model parameters and the token count of each request. So the team asked why models from 1 billion to 1 trillion parameters should share the same kernel at all.

Mechanically, Proteus proposes kernels, verifies them against a controlled reference implementation, times the ones that pass, and iterates on the best. That loop sounds routine, which is why the interesting finding is where it broke. The team expected search to be the hard part, and it wasn’t.

The system moves as fast as it can trust a kernel, not as fast as it can write one.

Databricks engineering blog, September 4, 2026

The blog is unusually candid about how measurement fooled them, and the examples are specific. Kernels for rotary position embeddings, a step inside attention layers, could reuse compiled code left over from an earlier attempt and look cheaper than a fair rebuild. Another candidate recorded GPU launches in a CUDA graph and replayed them as one unit while the baseline still launched each piece separately, so the two sides weren’t doing the same work. Others simply fit the input sizes in the visible test set.

That is why the team spent early design time on the checker rather than the prompt. Proteus times both sides the same way, cross-checks with more than one timer, clears leftover compiled state, holds back tests the candidate can’t see, and runs automated consistency checks that flag theoretically impossible speedups above 100x. Without those constraints, Databricks says generating more kernels mostly produced more noise.

The published case study runs the full loop on the packed decode kernel from the Gated DeltaNet path in Qwen 3.5 122B, on NVIDIA B200 GPUs with a Triton backend. The numbers are narrow because that’s the point, though the trace matters more than the winner.

Candidate	Result
Baseline	0.025 ms
Candidate 0000	Passed validation, slower than the reference
Candidate 012	1.5x on the single-batch decode shape
Candidate 030	Lowest measured latency, 0.018 ms
Candidate 036	Best shape speedup, 1.6x

The winning serving fragment specialized for a Batch=4, Key=128, Value=128 layout and processed the value dimension in 64-wide chunks. Databricks describes it as safe for that shape rather than a universal replacement. The run then ended badly: later attempts that generated C++ instead of Triton hit build and generation failures, and the branch exhausted its attempt budget.

The counter-reading is that the 1.8x to 5.2x range rests entirely on Databricks’ own measurement. The post links no public harness and no outside replication, so a reader can’t rerun any of it. That matters more than usual here, because the same post details how easily a benchmark misled the team.

Independent work points the same way on cost. KernelGenBench, revised on September 4 by Peiyu Zang and six co-authors, evaluated LLM-generated Triton kernels across 210 operators, then ran a 110-operator subset on six hardware platforms at a cost of over 15 billion tokens. Specialized agents averaged 4.99 million tokens per successful operator, rising to 6.25 million for CUDA Optimized Skill. AutoKernel accuracy fell from 87% on NVIDIA to 25% on Iluvatar CoreX, and the benchmark code is public.

Nor is the reward-hacking problem unique to one team. CUDA-Harness attributes the same vulnerability to reliance on predefined test inputs and answers it with isolated test data and progressive validation. KernelArc, which also targets B200 hardware, builds a deterministic benchmark guard into its multi-agent design.

For anyone paying inference bills, the practical read is that kernel-level gains are now claimed in the same breath as token bills, much like the per-watt claims made for custom silicon. Databricks is a familiar name in this kind of measured-results post, having recently run its own Grounded Reasoning Cup. What to watch is whether Proteus kernels show up in a shipped Databricks serving stack, and whether anyone reproduces the range on hardware Databricks doesn’t own.