Mixture-of-experts explained: why parameter counts stopped mattering
A trillion-parameter model can cost the same to run as a small one, because only a fraction of it fires per token. How sparsity works and what it costs.
Read More
Explainers
How the technology actually works, explained without the research-speak.
AI scaling laws: has bigger stopped meaning better?
The curves have not broken. The recipe changed twice, the spending moved to inference, and the data stock has a date on it.
Read More
Context windows explained: what a million tokens actually changes
Advertised context and usable context are not the same number. What a million-token window actually buys you, and what it does not.
Read More