The case against putting AI in everything

Almost every product you use added an AI feature this year. Most of them made the product slightly worse, and the companies shipping them had good reasons that had nothing to do with you.

There’s a test for whether an AI feature should exist, and most of the ones you’ve seen recently fail it.

The checkbox problem

Features get built because a competitor shipped one, because a board asked, or because the roadmap needed something to point at. None of those reasons involve a user problem.

The pressure is genuine, not imagined. Google restructured its AI leadership and lost four senior researchers in a week, knocking 4% off Alphabet. Boards read outcomes like that and conclude they need a visible AI story.

What follows is a feature designed to be announced rather than used, and the difference shows up immediately in how it behaves.

There’s a financing dimension too. Enormous sums are committed to AI infrastructure, including up to $500 billion arranged through Nvidia, and capital at that scale creates pressure to demonstrate usage everywhere.

That pressure runs downhill. It reaches product teams as a requirement to ship something visible this quarter, whether or not the underlying problem is one models solve well.

Three ways AI makes a product worse

Cost	What the user experiences
Latency	An instant action now takes seconds
Reliability	A deterministic result becomes a probabilistic one
Attention	Suggestions interrupt work that was going fine
Trust	Output must be checked, so the work is done twice

Latency is the most underestimated. Reasoning models spend compute at request time, which is exactly why they’re better at hard problems and exactly why they’re wrong inside a text box, as our piece on what thinking tokens buy sets out.

Replacing a function that always works with one that usually works is a downgrade, however impressive the usual case looks.

Reliability is the one nobody prices

A feature that works 95% of the time is a great demo and a poor tool, because the user now has to check every result to catch the 5%.

That checking cost often exceeds the time saved. If a summary needs reading against the original to trust it, you’ve read the original anyway.

The problem is structural rather than a tuning issue. Models are trained on human preference, and preference rewards fluent, confident, well-formed answers whether or not they’re correct, as the InstructGPT work demonstrated when a 1.3B tuned model was preferred over 175B GPT-3.

So the output arrives sounding right regardless of whether it is, which is precisely the property that makes checking necessary and tiring.

Error compounding makes it worse in anything multi-step. Our piece on why agents stall in production works through the arithmetic, and the short version is that per-step reliability collapses fast across a chain.

The market is already correcting

You don’t have to take this as opinion, because the retreat has started.

Microsoft merged its Copilot apps and killed several features outright: AI-generated podcasts, Group Chats, Copilot Labs and the animated assistant Mico.

Look at what died. Podcasts, group chat and a mascot were bets on AI as something social and playful, and none survived contact with how people actually use these tools. TechCrunch read the merger as a concession that the prior strategy was too complicated.

Gizmodo was blunter, calling the result easier to ignore. That’s harsh and it names the risk precisely.

Where it genuinely helps

This isn’t an argument against AI features. It’s an argument against reflexive ones, and the successful cases share a shape.

They work where the task was previously impossible rather than merely slow. Searching your own documents by meaning is a genuinely new capability, which is why retrieval systems stuck while chat wrappers did not.

They also work where the output is verifiable, which is why coding assistants survived. A test suite tells you immediately whether the suggestion was right, so the checking cost is close to zero.

And they work where the user opted in. A feature invoked deliberately is judged on its results. A feature that interrupts is judged on its worst moment.

Verifiability keeps recurring as the dividing line. DeepSeek’s R1 work got its strongest results on mathematics and code precisely because those can be checked mechanically, and the same property is what makes an AI feature trustworthy in a product.

The counter-argument

A fair objection says this is just conservatism with better manners.

Every interface shift looked like clutter at first, and users adapt faster than critics expect. Shipping imperfect features is how companies learn what works, and waiting for reliability means never shipping.

Costs are also falling fast enough to change the calculus. Capability now runs from $6 to $30 per million output tokens, and Epoch AI reports pre-training compute efficiency improving roughly 3x per year. Features uneconomic last year are viable now.

Both points land. Neither addresses the checking cost, which is a property of probabilistic output rather than of price.

The test to run before shipping

Four questions, and a feature should clear all of them.

Does it do something the product could not do before, or does it do the same thing less predictably? Is the output cheap for the user to verify? Does it wait to be invoked? And if it were removed tomorrow, would anyone notice?

That last question is the useful one, and Microsoft has just run the experiment publicly. It removed several AI features, and the reporting suggests the answer was mostly no.

Worth applying the same test to your own tools. Count the AI features you actively invoke versus the ones you dismiss, and the ratio usually settles the argument faster than any product review.

What would change this view is measurement rather than argument. Published retention data on AI features, showing sustained use rather than launch-week curiosity, would settle it.

Part of the problem is who these products imagine they are for, which our piece on what the industry gets wrong about its own users takes up.

The labour question runs on the same evidence problem. Our analysis of what the jobs data actually shows finds an effect at entry level and almost nothing in the aggregate.

Almost nobody publishes that, which is itself informative. Companies report adoption when the numbers flatter them and go quiet when they do not, so the absence of retention data is closer to an answer than a gap.