Tokens Too Cheap to Meter: Why Access Will Soon Limit AI, Not Compute

The price of using machine learning intelligence is falling by several orders of magnitude a year, and it shows no sign of stopping. The developer’s own analysis predicts that LLMs will be integrated into every part of computing as infrastructure, not just as a product, within the next year or two. They expect LLMs running locally at current frontier quality on commodity hardware within three to six years. And they believe quality and access will become the limiting factor on AI 1 use, not sheer number of tokens.

That last point is the crux. The analysis is arguing that the bottleneck is shifting from how much compute you can buy to how much skill you can afford.

The Cost of a Token

LLMs are priced per-token. A token is a fragment of a word; it takes about 1.5 tokens to represent a whole word. Every token a model reads costs money, and every token it outputs costs money. The model provider — Anthropic or OpenAI — charges a fixed amount for each.

But the analysis makes a key distinction. The cost per token of frontier models is not consistently going down. Smaller models may cost less per token, but they use more tokens overall for the same task because they have to think more or correct their first drafts. The cost to complete a given task is what matters, not the raw price of each fragment.

The chart below shows the “Pareto frontier” of cost per task at present. The Y-axis tracks the quality of the model, as measured by a suite of benchmarks. The X-axis tracks the cost to complete those benchmarks. The chart is on a logarithmic scale, so larger Y-axis and smaller X-axis numbers are better.

Towards the top-right sits Claude Fable-5.1, expensive and intelligent. Towards the middle-left sits GPT-5.6 Luna, cheap and less intelligent. Models below the dotted line are basically not worth considering.

Compare that to the 2025 chart. The Y-axis (intelligence) stayed about the same, with less of a fall-off towards the cheap end. The X-axis (cost) got two orders of magnitude cheaper.

Hardware and Software Together

The analysis rests on two separate trends moving in the same direction. First, the hardware: GPUs are getting exponentially more efficient with every generation. The graph below shows the power efficiency of the GPU itself, with the X-axis tracking time and the Y-axis tracking efficiency. A straight line on this logarithmic graph means efficiency doubles about once every two years, a rate the analysis says we haven’t seen since Moore’s Law in the 1960s.

Second, the software: inference engines are improving rapidly. vLLM, an open-source inference engine, saw a 40% increase in efficiency in just 15 months, jumping from version 0.5.4 in September 2024 to version 0.11.1 in December 2025. NVIDIA showed up to 50% efficiency improvements on its MLPerf stack from 2.0 to 2.1. Intel showed a 2.4x throughput increase between MLPerf 6.0 and 6.1.

The analysis notes that serving workloads are getting more efficient faster than offline inference, and that the hardware stays fixed while the software changes.

What This Means for AI

The analysis predicts that quality and access will become the limiting factor on AI 1 use, not sheer number of tokens. That is a remarkable claim, and it rests on the evidence laid out above: falling token prices, more efficient hardware, and faster inference engines.

The analysis’s position is that the bottleneck is shifting from compute capacity to human skill. The tools are getting cheaper and faster, but the ability to wield them effectively is not growing at the same pace.

The Order of Change

The analysis’s timeline breaks down into three stages:

1. The next year or two: LLMs integrated into every part of computing as infrastructure, not just as a product.
2. The next three to six years: LLMs running locally at current frontier quality on commodity hardware.
3. The shift to quality and access as the limiting factor on AI 1 use.

Each stage is presented separately, not as a chain where one follows the other. The analysis predicts that the hardware and software improvements will keep coming, and that the access gap may grow as a result.

The analysis’s evidence is specific and quantitative. The GPU efficiency graph, the vLLM release dates, the MLPerf numbers — all of it points in the same direction. The analysis’s claim is that the future of AI is not about who can afford the most compute, but about who can afford the best access.

That is a striking prediction to make. Whether it holds up depends on the next few years of releases and benchmarks.

Source material: “Tokens too cheap to meter,” jyn.dev.

AI economics · GPU efficiency · inference engines · LLM integration · OpenAI pricing