AT&T has sharply reduced what it spends on internal AI by matching each task to the model that fits it. For some coding-related work, costs fell by as much as 56 percent while quality slipped only 2 percent. The company's internal AI platform now handles about 45 billion tokens a day, and roughly 40 percent of that traffic already goes to open models.

From "which model is smartest" to "which model is enough"

For most of the enterprise AI era, the question has been which model performs best. AT&T is asking close to the opposite one: what is the cheapest model that can reliably finish the job in front of it? Answering that question automatically is the entire purpose of a model router.

AT&T funnels employee requests through routing infrastructure such as LiteLLM, which judges how hard the request is before sending it anywhere. Light work, like summarizing documents or previously submitted code, goes to open models including Meta's Llama and Google's Gemma. Work where a mistake is expensive, such as complex code generation, goes to current models from Anthropic and OpenAI. The employee experience is the same either way, but the price per request underneath is not.

The result: on some coding tasks, costs dropped 56 percent while quality declined 2 percent. That gap between 56 and 2 is also a measure of how much money the old approach, sending everything to a frontier model, was leaving on the table.

45 billion tokens a day, 40 percent on open models

The scale is concrete. AT&T's internal AI platform processes roughly 45 billion tokens per day, and about 40 percent of employee AI requests are routed to open models. The company intends to push that share to somewhere between 60 and 70 percent.

Mark Austin, a vice president at AT&T, has said that for many jobs open models are just as good as or better than models Anthropic and OpenAI sold previously. The comparison point matters. The claim is not that open models have caught the current frontier, but that performance equivalent to last year's frontier is now available in open weights. For a large share of internal work, that turns out to be enough.

The goal is equally plain: keep spending on premium AI services flat even as total AI usage keeps climbing.

Cheaper models do not automatically hurt the frontier

On its face, this looks like bad news for anyone selling expensive models. Jim Covello, who leads equity research at Goldman Sachs, sees another possibility.

In Covello's reading, progress in small, inexpensive open models works in favor of hyperscalers such as Amazon, Microsoft and Google rather than against them. If open models make it easier for enterprises to run AI profitably, more AI applications become worth building. More applications mean more tokens processed, and more demand for the data centers running underneath them. He points specifically to the improved odds of profitably filling the computing capacity these companies keep adding.

The context is a shift in market sentiment. For a long stretch, announcing higher capital expenditure was rewarded by investors. Over roughly the past quarter, Covello says, the market has begun questioning that assumption much harder. The issue is no longer whether Big Tech can build the infrastructure, but whether it can fill what it has built with demand that pays for itself.

Covello also describes a "model optimization layer," which sends high-consequence requests to expensive models and routine requests to cheap ones, as a technical bottleneck for enterprise AI. Solving it, in his view, is one of the keys to unlocking AI's economic value inside companies.

Summary

AT&T's experience shows that task-based model routing survives contact with production rather than staying a spreadsheet exercise. Costs on some coding tasks fell 56 percent against a 2 percent quality decline. Of 45 billion tokens a day, 40 percent already flows to open models, with a target of 60 to 70 percent. If the buying criterion shifts from raw capability ranking to cost effectiveness, enterprise AI procurement may drift away from consolidating on a single vendor and toward running several models side by side.