Bigger AI context windows are not always the answer
The article argues that simply enlarging an AI model's context window is an expensive and often inefficient way to handle large datasets. It recommends letting reasoning systems decide which data to process and delegate calculations to external tools instead of reading every record. Although per-token prices are dropping, growing token consumption means smarter data selection is needed.
Context limits have risen sharply: Anthropic moved Claude from 9,000 to 100,000 tokens in May 2023 and to 200,000 by November; Google’s Gemini 1.5 Pro reached one million in February 2024 and two million by June. By March 2026, Anthropic offered a million-token window across Claude models, and at least thirteen hosted frontier models supported one million or more.
Costs remain a constraint. Ramp data cited in the article shows per-million-token prices paid by US businesses fell 41% from March to September 2026, while Goldman Sachs projects AI agents may use 24 times more tokens by 2030. An Opetek test calculating EUR/USD volatility cost about $0.80 per query with raw data in context versus $0.07 with external computation.
The shift toward selective context and external tools could affect organizations running data-heavy AI workloads, including financial analysts, scientists, and software teams, by lowering query costs and changing system design priorities. It may also influence smaller users if cheaper architectures broaden access, while vendors may compete less on raw context size and more on orchestration and tool use. The broader public could see benefits through more efficient AI services, though outcomes depend on implementation and pricing.