Five fixes that would explode consumer LLM adoption: caching, speed, model routing, ads, and agentic tasks

Summary

Five core product improvements would unlock consumer LLM adoption beyond the current base. The gaps are straightforward and mostly solvable with existing infrastructure.

Caching and instant response

Users ask identical questions repeatedly—when was OpenAI founded, the history of the Roman Empire, company backgrounds—yet each query triggers full inference. Caching these results and serving them instantly would mirror what Google does with knowledge panels. Raw speed is the second fix. ChatGPT's 5.2 Instant takes 38 seconds to deliver a response about LLM history. Cerebris inference running on Codec's desktop drops that to under two seconds with GPT 5.3 Spark Low. Speed matters in consumer because users abandon slow apps, but more importantly because instant response removes the friction that prevents follow-up questions. When there's no perceived waiting cost, users stay engaged and ask more. Google's response to Cerebris with TPUs remains unclear, as does whether Nvidia's Grok can match the speed.

Model routing and nomenclature

Consumer chat apps should bury model names so deep in the UI that users never see them. ChatGPT already supports natural language routing—saying "think really hard about this" switches from instant to thinking mode—but this should expand invisibly. The model router should detect when a question has been asked thousands of times and serve it from a database without hitting a GPU. It should also detect when a query requires search, code execution, or deep research, and route there automatically. Deep research is currently stuck behind a plus button and requires explicit selection, creating friction. Intelligent routing removes that choice friction while ensuring the right inference path for the right question.

Advertising as a business engine

Google's most successful products—Search, Maps, Chrome, Android—all feed the ads flywheel. Products with ad monetization get resources, engineering effort, and compound improvement. Perplexity abandoned its ads experiment in February 2026, citing user trust concerns. The timing is suspicious. The company started testing ads in 2014 and saw minimal advertiser uptake. Taz Patel, who led the ads effort, left the company, and Perplexity admitted less than half a percent of interested brands were let into the program. Sean Frank at Ridge Wallet says he's "first in line" to advertise on ChatGPT because the conversion is strong. The lack of demand for Perplexity ads suggests the product didn't work, not that ads are inherently trust-destructive. Without the ads flywheel, consumer LLM apps lose a critical funding and motivation mechanism.

Agentic code execution beyond reasoning

Reasoning models can write some Python and execute it, hence the Mac Mini boom for local execution. Most consumer data lives in the cloud now—camera rolls, email, messages—accessible via APIs. Integrations with Gmail already exist across major LLM apps. Rather than replicating OpenClaw on dedicated hardware, the better move is letting users fire off agent-level code generation to solve specific problems: scraping a restaurant with no API, automating a form submission with headless Chromium. This goes beyond inline Python snippets but doesn't require the full local-compute footprint.

Polish and UI details

Apps fail in patchy cell service. Text-to-speech cuts out mid-deep research on some platforms but not others. Some apps won't read deep research aloud even though they'll read normal reports. These friction points drive churn. If you run an AB test and find the button should be blue instead of green, tell your boss it's taste. You'll keep your job longer.