Coinbase Cuts AI Spend by Half With Open-Weight Models, Smarter Routing

Coinbase has reduced its internal AI spending by nearly half after overhauling how employees access and use large language models, replacing usage limits with cheaper default models, automated routing, and aggressive caching. 

CEO Brian Armstrong, in a post on X, said the company is experimenting with making open-weight models such as GLM 5.2 and Kimi 2.7 the default through its internal LLM gateway, while still allowing engineers to choose any model they consider appropriate for a task.

Armstrong said the company has “cut our AI spend nearly in half, while our token usage continues to grow.” He added that the objective is “not to suppress usage” but to build infrastructure capable of supporting exponential growth in AI workloads while keeping costs under control.

The company found that 91% of employees never reached their AI usage caps, leading it to abandon tighter limits in favour of cheaper default models. 

Source: X, Brian Armstrong

Engineers remain free to select frontier models when needed, while code reviews deliberately use multiple models to cross-check one another’s outputs.

A second change is automated model routing. Coinbase’s internal AI harnesses preprocessed prompts and direct requests to the most cost-effective model based on the task, pricing, and cache availability. 

Armstrong said that planning workloads may justify a frontier model, whereas execution tasks often do not. “Ultimately, humans shouldn’t be choosing models—AI can automate this task,” he wrote.

Mark Landgrebe, a senior software engineer at the company, said on X that the company routes AI requests through an internally developed gateway that provides a single endpoint for dozens of models while applying cross-provider failover, prompt redaction, logging and cost controls before requests reach external AI providers. 

He added that the routing system also factors in cache state, keeping conversations on the same model while cached context remains valid and only reconsidering model selection after the cache's time-to-live expires.

The company also significantly increased cache utilisation. “For example, our cache hit rate went from 5% to 60% in LibreChat once properly implemented,” said Armstrong, based on a result enabling repeated requests to reuse cached responses rather than generating fresh tokens. 

Landgrebe said cache hits require identical prompt prefixes, prompting Coinbase to maintain long, stable prefixes across conversations so only newly generated tokens are processed at full inference cost.

Coinbase has also instructed employees to keep context windows lean by starting new sessions for different tasks, limiting file context, disconnecting unused tools and avoiding unnecessary token consumption.

Landgrebe also revealed that third-party AI coding tools are routed through Coinbase’s internal AI gateway. 

He said default models for those tools are set through a lightweight client-side launcher, while the company is working to extend gateway-based automatic model routing into more customisable coding harnesses.

Instead of enforcing hard spending limits, Coinbase now emphasises visibility into AI usage. Employees can use any model and consume as many tokens as required, but spending is transparent, and higher AI usage is expected to correspond with greater business impact.

This follows Coinbase’s broader push to become an AI-native company. In May, the company cut about 700 jobs, or roughly 14% of its workforce, as Armstrong said advances in AI were enabling smaller, more productive teams and a flatter organisational structure.

Enterprises are now increasingly scrutinising AI inference costs amid a broader backlash against “tokenmaxxing”, the practice of maximising AI token usage without clear productivity gains. 

Recently, Uber reportedly introduced monthly spending caps of $1,500 per AI coding tool after exhausting its annual AI coding budget within four months. The company said the limits apply to agentic coding tools such as Cursor and Claude Code, while executives have questioned whether higher token consumption translates into proportionally better products. 

Reports also suggest that other companies, including Amazon, Meta, and Walmart, have also begun steering employees toward cheaper AI models or introducing tighter governance around AI usage as inference costs climb.