Why the opening word of an AI answer is the slow one

Picture ordering at a busy restaurant. There is a wait before your plate arrives. Once the kitchen starts sending out dishes, they come quickly, one after another.

An AI answer behaves in a similar way. There is a pause after you press enter, and then the words stream out steadily. This piece walks through what happens in that first pause, and why it is usually the part you notice most.

The trip out

Your question first has to travel to a data centre, and the answer has to travel back. Each step adds a small delay. We follow that journey in How far your AI answer travels.

Waiting in line

When your request arrives, it may not be handled straight away. Most AI services share their machines among many users, so requests wait their turn.

At a quiet hour, the line is short. At a busy hour, it can be longer, and the pause before the first word grows.

This is one reason the same tool can feel quick in the morning and sluggish in the afternoon.

Reading the whole question first

Before a model can write anything, it has to take in everything you sent. That includes your question, any earlier messages in the chat, and any documents attached.

The model reads all of this in one pass to build up its understanding. A short question is quick to read. A long document or a long conversation takes more work before the first word can be produced.

This step can be a large part of why the first word is slow. The model cannot start answering until it has finished reading.

Why the rest comes faster

Once reading is done, the model writes its answer piece by piece. Each new piece builds on the last, so the words appear as a steady stream.

Each piece is a smaller job than the initial read. That is why, after the pause, the text tends to flow at an even pace.

Some services wait until the whole answer is ready before showing it. In that case, the entire answer feels like one long pause, even if the work is the same.

Other things that add to the pause

A few further steps can sit in front of the model.

Checks and filters. Many services check requests before and after the model runs. These steps are useful but take some time.

Fetching extra information. Some tools look things up first, such as searching a document store. The model waits for those results.

Loading the model. If a model has not been used recently, it may need to be loaded onto the hardware before it can run. This start-up step can make an occasional request noticeably slower.

Model size. Larger models generally do more work per answer. They can be more capable and also slower to start.

Questions to ask about the pause

If the first word matters for your product, these questions help.

• Where will my requests be processed, and how far is that from my users?
• How does the pause change at busy times compared with quiet ones?
• Do long documents or long chats slow the first word, and by how much in practice?
• Is the model kept ready, or loaded on demand?
• Which checks run before the model, and can any run in parallel?

Ask for answers tied to a date and to stated conditions. A one-off demonstration on a quiet system tells you little about a busy Monday. We explain why dates matter in Why we publish dated prices.

What we are building

The pause before the first word is shaped by distance, queues, the size of what you send, and the model. Some of these are choices about where work runs.

We're building standardized compute close to power, connectivity and demand, connected through one programmable network for AI inference. The aim is to place workloads by policy, on latency, data residency, model and load, with an auditable record of commitment, placement and delivery for each workload.

Starfield Fabric, our control plane, is in development. It is designed to take distance and current load into account when deciding where each request should run.

If the first word matters to your users, use Request capacity below.