Edge data center vs hyperscale: many small or a few big?

For years, the answer to "where should AI run?" was simple: in a very large data center, owned by a very large company. Bigger meant a lower cost per unit, easier management and easier buying.

That answer still holds for some work. For a growing share of it, it's being questioned. Here's a balanced look at both approaches, and why we think a well-run network will use each where it fits.

What big sites are good at

Large campuses exist for good reasons.

Scale for training. Training a frontier model means thousands of chips working on one job at once, connected by very fast links. That kind of work needs to sit in one place. Splitting it across distant sites would slow it down badly.

Efficient operations. One large site can share cooling, security, staff and spare parts across a lot of equipment. Many costs grow slower than the size of the building.

Simple buying. For a customer, one region with one provider is easy to understand. One contract, one set of tools, one bill.

Deep engineering. Big operators have spent years building reliable power, cooling and networks. Their sites are built to a high standard.

None of that is going away. Large campuses will keep doing the work they suit.

Where big sites struggle

The same strengths create some limits.

Waiting for power. A very large site needs a very large grid connection. In many places, new connections take years to arrive. Plans are ready long before the electricity is.

Distance from users. A few huge sites can't be close to everyone. For work that has to respond quickly, such as robotics, video analysis or industrial control, the distance to a far-away region shows up as waiting time.

One place, one set of rules. If data has to stay in a particular country, and the nearest large site is in another, the choice gets awkward.

Paying for more than you need. Large sites are built and priced for big, demanding jobs. Everyday inference that runs there can end up carrying costs it doesn't need.

Concentration. When much of the world's AI work depends on a handful of locations and owners, problems at any one of them reach a lot of people.

What small sites are good at

Smaller sites flip some of these trade-offs.

Faster to power. A smaller site needs a smaller connection. There's spare capacity on many local grids, at industrial sites and next to existing energy assets, that's too small for a giant campus but enough for a modest one.

Closer to demand. Many small sites can sit near the cities, factories, hospitals and networks that use AI every day.

Easier to match local rules. If a country, region or customer needs work to stay within certain borders, there's a better chance of having capacity there.

Spread-out risk. One small site going offline affects a small share of the work.

Where small sites struggle

They have real limits too.

Not for big training runs. Small sites aren't the right place to train very large models.

Harder to manage one by one. Each site needs power, cooling, connectivity, security and upkeep. Running hundreds of them as separate projects would be slow and expensive.

Harder to buy. No customer wants a separate contract with every small site. On its own, a small site is hard to sell.

That last point is the key one. Small sites only make sense at scale if something turns many of them into one service.

The network answer

The useful question is which work goes where, and whether there's a system that can make that choice well, request by request.

A sensible split looks something like this:

• Large campuses for training and for work that needs huge amounts of tightly connected compute.
• Smaller, distributed sites for inference: the everyday work of running models for people and machines, close to where they are.
• One network above them that places each request by its needs: how fast it must respond, where its data may go, which model it needs and where there's room.

For that to work, a few things have to be true. The sites need to be built from standard, repeatable units, so work behaves the same wherever it lands (more on that in Modular data center: why standard units matter). The network needs to understand each request's rules. And buyers need a record of where their work ran.

Questions for anyone planning AI capacity

1. Which of your workloads are training, and which are inference?
2. Which inference workloads need to respond quickly, or stay in a certain country?
3. How long would you wait for new capacity in your preferred region?
4. If you used several smaller sites, how would you buy and manage them as one?
5. Who would you call if one of those sites had a problem, and how would you know your work had moved?

The answers usually point to a mix.

Where Starfield fits

We're building Starfield Fabric as the layer that connects standardized compute at many sites into one programmable network for AI inference. It is designed to place workloads by policy, on latency, data residency, model and load.

We're not trying to replace large campuses. The aim is to make many smaller sites usable as one, so inference can run where it fits. Aggregation, not construction. The same idea applies to small places: see Edge data center: can a small town host AI capacity?

If you have power, sites or inference demand, talk to us: Partner with Starfield below.