Input tokens and output tokens: why prices differ

Reading a page aloud is quick. Writing a page yourself, one word at a time, while checking each word against everything before it, takes much longer. AI models face a similar gap between reading and writing.

Most per-use AI prices list two rates: one for input tokens and one for output tokens. Output often carries the higher rate of the two. This piece explains why, and what it means when you estimate a bill.

Two kinds of token in every request

Input tokens are everything you send to the model. That includes your question, any instructions, pasted documents and, in a chat, earlier messages.

Output tokens are everything the model writes back. That is the answer itself, plus any working the model produces along the way.

If tokens are new to you, start with our piece on what a token is and why it sets your AI bill. The short version is that a token is a small piece of text, and prices are counted per token.

Reading happens all at once

When a model reads your input, it can process many tokens side by side. The chips that run AI, usually GPUs, are built for exactly this kind of parallel work.

That means a long prompt is handled in one large, efficient pass. The machine stays busy, and the cost of each input token is spread across a lot of useful work.

Writing happens one token at a time

Writing is different. The model produces one token, then uses it to decide the next, then the next. Each step depends on the one before, so the steps cannot all run together.

Every one of those steps needs much of the model's weights to be read from memory. Much of the time, the chip is waiting on memory rather than doing arithmetic. The machine is occupied for longer per token, even though each step does less.

That is a main reason output tends to cost more. Each output token ties up the hardware for more time than each input token does. We explain the reading and writing stages in Inference, explained.

What else can affect the gap

The size of the gap between input and output prices varies by provider and by model. A few things tend to shape it.

Model size. Larger models move more data per output step, which can widen the gap.

Batching. Providers group many users' requests together to keep chips busy. How well they do this affects what they charge.

Reused input. Some providers offer lower rates when the same input is sent repeatedly, because parts of the reading work can be stored and reused. Terms vary.

Long contexts. Very long inputs can need more memory per request. Some providers charge differently above a certain length.

What this means for your bill

Because the rates differ, two requests with the same total tokens can cost very different amounts. The split between input and output matters as much as the total.

Reading-heavy work. Summarising long documents or classifying text sends a lot in and gets a little back. Input rates drive the cost.

Writing-heavy work. Drafting long reports or generating code sends a little in and gets a lot back. Output rates drive the cost.

Chat. Long conversations resend history as input each turn. Input grows steadily, even if answers stay short.

Knowing which pattern your workload follows tells you which rate to watch most closely. It also suggests where to save: shorter answers help writing-heavy work, and trimmed context helps reading-heavy work.

Questions to ask a provider

Before comparing offers, it helps to ask:

1. What are the separate input and output rates, in the same unit?
2. Is there a discounted rate for repeated or cached input, and what are the conditions?
3. Do rates change above a certain input length?
4. Does the model produce hidden working tokens, and are they billed as output?
5. On what date were these rates set?

We are not quoting rates here, because they change often. For dated input and output prices across models, see Model Watch. For how to read a price list, see Cost per million tokens: how to read an AI price list.

A note on speed

The same mechanics shape waiting time. Reading a long prompt adds a delay before the first word appears. Writing a long answer adds time for every token that follows.

So a workload's mix of input and output affects both cost and how quickly users see results. Where the work runs matters too. We look at distance in How far your AI answer travels.

What we are building

Inference is mostly this pattern of reading and writing, repeated at scale. Running it well depends on matching each workload to suitable hardware in a suitable place.

We're building standardized compute close to power, connectivity and demand, connected through one programmable network for AI inference. Workloads are designed to be placed by policy, on latency, data residency, model and load, and the network is designed so every workload can generate an auditable record of commitment, placement and delivery.

Knowing the shape of your work is the start of placing it well.

If you are planning an inference workload and want to understand its shape, use Request capacity below.