Cost per million tokens: how to read an AI price list

An AI price list can look like it was written for accountants. Per million tokens. Per GPU-hour. Reserved. Committed. Batch. Cached. You only need four steps to read it: find the unit, turn your work into that unit, read the small print and check the date.

This is a practical guide for the person who has to compare two offers and explain the choice to someone else. No finance background needed. A spreadsheet helps.

Step 1: Find the unit

Most AI price lists charge in one of three ways.

Per use. You pay for what you send and receive, usually counted in tokens. A token is a chunk of text, roughly a short word or part of a longer one. Prices are often quoted per million tokens, and input (what you send) and output (what comes back) are usually priced separately. Output tends to cost more.

Per machine time. You rent capacity by the hour, whether you use it fully or not. This is common when you run your own model or need dedicated hardware.

Per plan. A monthly fee covers a set amount of use, or a set number of seats, with extra charges above the limit.

Write down which unit each offer uses before you compare anything. Comparing a per-token price with an hourly rate is like comparing a taxi fare with a car lease.

Step 2: Turn your work into that unit

Next, describe your workload in plain numbers. You need four estimates:

1. How many requests you expect per day or month.
2. How long a typical request is (what you send in).
3. How long a typical answer is (what comes back).
4. Whether traffic is steady or arrives in spikes.

Your engineers can pull the first three from a short test or from logs. The fourth matters for machine-time pricing: if traffic spikes for two hours a day, you pay for the quiet hours too.

Here is an illustrative example with made-up round numbers. Say you send 100,000 requests a month. Each request sends about 1,000 tokens and gets about 300 back. That is 100 million input tokens and 30 million output tokens a month. Multiply each by its price per million, add them, and you have a monthly cost you can compare.

Do the same calculation for every offer, using the same workload. Now the numbers line up.

Step 3: Read the small print

The headline price is rarely the whole price. Look for these lines.

Input and output split. If your answers are long, output price matters more than input price. A list that looks cheap on input may be expensive for your mix.

Context length tiers. Some services charge more once a request passes a certain length. Long documents can move you into a higher tier.

Discounts for cached or repeated input. If you send the same instructions with every request, some providers charge less for the repeated part. This can change the picture a lot for chat and agent workloads.

Batch pricing. Work that can wait hours, such as overnight processing, is sometimes cheaper. Check whether any of your work qualifies.

Minimums and commitments. Reserved or committed prices are lower per unit but you pay whether you use them or not. Note the term, the minimum and the exit terms.

Rate limits. A cheap plan that caps how many requests you can send per minute may not handle your peak. Ask what happens when you hit the limit: queued, rejected or charged extra.

Data transfer and storage. Moving data in and out, or storing logs, can be billed separately.

Where it runs. If the price list does not say where requests are processed, ask. Location affects speed and, for some data, whether you can use the service at all.

Step 4: Check the date

Prices for AI work change often. A comparison is only as good as the date on it.

That is why we date every price we publish. On Model Watch #1, four paid models listed $2.00 per million input tokens, with outputs from $6 to $10, as of 22 October. Next month that line may be different. Keep a date next to every number in your own spreadsheet too, and re-check before you sign.

A one-page comparison

When you present the choice, keep it to one table with these columns:

• Provider and model
• Pricing unit
• Estimated monthly cost for your workload
• Commitment and exit terms
• Rate limits at your peak
• Where requests are processed
• Date the price was read

If two offers are close on cost, the other columns usually decide it.

Questions to send a provider

• Which unit do you charge in, and how do you count it?
• Are input and output priced separately?
• Do repeated or cached inputs cost less?
• Is there a lower price for work that can wait?
• What are the rate limits on this plan, and what happens above them?
• Where will my requests be processed, and can I choose?
• What does a commitment cost, and how do I get out of one?

What we are building

Price lists are hard to read partly because the thing being sold is hard to see. We're building standardized compute close to power, connectivity and demand, connected through one programmable network for AI inference. Workloads are designed to be placed by policy, on latency, data residency, model and load, and designed so every workload can generate an auditable record of commitment, placement and delivery, so a buyer can check what was delivered against what was billed.

Related reading: AI inference cost: why one task gets different prices and Hidden costs of AI capacity that aren't on the quote.

If you are working through quotes for inference, use Request capacity below.