Adding AI capacity one unit at a time

A café owner who expects to get busier has two choices. She can rent a large space now and fill it slowly. Or she can rent a smaller one and take the shop next door when the queues get long.

AI capacity raises the same question, with much more money attached. This piece explains why adding capacity in steps suits demand that is hard to predict, and what it asks of the site to make each step work.

Why AI demand is hard to forecast

AI use can grow quickly once a feature catches on. It can also shift in ways that are hard to plan for. A new model may need more computing per answer, or less. A product may take off in one country and not another.

There is also the split between training and inference. Training comes in large bursts. Inference grows with the number of people using a product, day after day. (We explain the difference in Training and answering are different jobs.) Each pattern puts different demands on capacity, and the mix can change.

The cost of guessing wrong

If you build too much, you pay for power, cooling and space that sit idle. Hardware bought early may also age before it is fully used.

If you build too little, work queues up or goes elsewhere. Adding capacity late can take a long time, especially if it means a new building.

Growing in steps does not remove the guess. It makes each guess smaller, and gives you a chance to correct it before the next one.

What a step looks like

With modular sites, a step is usually one or more standard units added to an existing location. (We cover the basics in What is a modular data center?) Each unit brings its own compute, and often its own power handling and cooling.

A step can also be a new small site somewhere else. That can make sense when demand appears in a new region, or when data needs to stay in a particular country. (See Data residency in one page.)

Because the units are the same, each step should behave like the last. Teams already know how to install, test and run it. (That is the idea behind Why standard units matter.)

What it asks of the site

Growing one unit at a time sounds simple. It only works if the site was planned for it from the start.

Power headroom. The grid connection, or local supply, has to cover the units planned later, not just the first ones. If the connection is sized only for today, the next step may wait a long time. (See Spare power is not the same as usable power.)

Space and layout. There needs to be physical room for more units, with access for delivery and maintenance.

Cooling capacity. Each unit adds heat. The site needs a way to remove it, whether that is built into the units or shared. (We cover this in Why AI chips need so much cooling.)

Network capacity. More compute means more traffic. Fibre to the site should have room to grow, and ideally more than one route.

Permits. Some approvals may cover the full planned site. Others may need to be renewed or extended for each phase. Rules differ by place, and this article is not legal advice.

What it asks of operations

More units, and possibly more sites, mean more things to watch. Operations have to scale without adding the same amount of people each time.

That puts weight on remote monitoring, a consistent design across units, and clear procedures for adding and retiring them. It also helps if workloads can move between units and sites without being rebuilt. (We look at that in A control tower for AI work: what it would do.)

Questions to ask before you grow in steps

Whether you host capacity or buy it, these questions show whether a stepwise plan is real.

1. How much power is agreed for later phases, and on what terms?
2. Is there physical space reserved for future units?
3. How long does it take from ordering a unit to running work on it?
4. Can new units use newer hardware without redesigning the site?
5. How are workloads moved when a unit is added, serviced or retired?
6. Does each step need a new permit, or is it covered already?

What we are building

We are building for growth in steps, because demand is hard to predict.

We're building standardized compute close to power, connectivity and demand, joined by one programmable network for AI inference. The network is designed to place workloads by policy, on latency, data residency, model and load, and every workload is designed to generate an auditable record of commitment, placement and delivery.

Smaller, repeatable steps make each decision easier to get right.

If you have power and space and want to grow AI capacity over time, use Request capacity below.