How to size your AI workload before you buy

Before switching energy supplier, most people look at last year's bills. They want to know roughly how much they use, and when. Without that, every tariff looks reasonable and none can be checked.

Buying AI capacity is similar. A quote only means something when you can hold it up against your own use. This piece sets out a simple way to estimate your AI workload, with a worked example using made-up round numbers.

Start with the requests

A request is one question sent to a model and one answer sent back. It might be a customer chat message, a document summary or a search query.

Start by counting them. If you already run an AI feature, your own logs will show this. If you are planning one, estimate from the activity it replaces, such as support tickets, documents processed or searches made.

Count by day, not by month, so busy and quiet periods show up. (Turning that count into the units a price list uses is covered in Cost per million tokens: how to read an AI price list.)

Estimate the size of each request

Requests are not all the same size. Most AI services measure size in tokens, which are small pieces of words. A short question is a few tokens. A long document can be many thousands.

Split each request into two parts:

Input. What you send the model, including instructions, the question and any documents attached.

Output. What the model writes back.

The split matters because input and output are often priced differently. Output tends to cost more. (How input and output are priced is covered in Cost per million tokens: how to read an AI price list.)

You do not need precision. A typical size for each, with a sense of the range, is enough to start.

Find the peak, not just the average

Averages hide the moments that matter. A service that is quiet overnight and busy at lunchtime needs capacity for lunchtime.

Look for your busiest hour on a normal day, and your busiest day in a normal month. Note any known spikes, such as a product launch, a month-end process or a seasonal rush.

Peaks matter most if you are reserving capacity. They matter less if you pay strictly per request, but they still affect how fast answers come back when everyone is busy.

An illustrative worked example

The numbers below are made up and rounded for illustration only. They do not describe any real customer, model or price.

Imagine a support assistant that handles 10,000 requests a day. Each request sends about 1,000 input tokens and gets back about 500 output tokens.

That is 10 million input tokens and 5 million output tokens a day. Over a 30-day month, it is about 300 million input tokens and 150 million output tokens.

Now the peak. Suppose the busiest hour carries one tenth of the day's traffic. That is 1,000 requests in a single hour, far more than an even spread would suggest.

Write these down as a small table of requests, tokens and peak hour. That table is your yardstick.

Test the quote against it

With your yardstick, each part of a quote becomes something you can check.

1. Multiply each price by the matching unit in your estimate. Do input and output tokens have separate prices?
2. If the quote is per hour or per reserved block, can that block handle your peak hour?
3. What happens when you go over? Are extra requests refused, queued or charged at a different rate?
4. Are there minimum commitments that exceed what you expect to use?
5. What is not included, such as data transfer, storage or support? (See Hidden costs of AI capacity that aren't on the quote.)
6. Is the price dated, so you can compare it fairly with others? (See LLM API pricing: why we publish dated prices.)

Try the same estimate against two or three pricing models. The option with the lowest bill at your average may not have the lowest bill at your peak. (We compare approaches in AI inference cost: why one task gets different prices.)

Add the details that change the answer

Two more facts can change which quote fits.

Where the work may run. If some requests carry data that must be processed in a particular place, size that share separately. It may need its own capacity. (See Data residency for AI inference, in one page.)

How your use will change. Note a rough growth expectation for the next year. A quote that fits today may not fit in six months.

Revisit the estimate after a few weeks of real use. Your first numbers will be wrong in places, and that is fine. The point is to have numbers at all.

What we are building

We think buyers should be able to test any quote, including ours, against a clear picture of their own work.

Starfield deploys standardized compute close to power, connectivity and demand, and we're building one programmable network to connect it for AI inference. Workloads are designed to be placed by policy, on latency, data residency, model and load, and every workload is designed to generate an auditable record of commitment, placement and delivery.

Records of what actually ran also make next year's estimate easier than this year's.

If you have sized your workload and want to test it against capacity, use Request capacity below.