AI infrastructure glossary: cloud, edge and in between

An AI proposal can read like a different language. Three vendors use five words for the same thing, and one word can mean three things depending on who says it.

This glossary is for the person who has to read the proposal and sign off on it. Each term gets a plain definition and, where it helps, the question it should prompt.

Where the work runs

Cloud. Computing you rent from a provider and reach over the internet. You don't own or see the machines. In AI, "cloud" usually means a few very large data center regions run by a handful of companies.

Region. A provider's name for a group of data centers in one geographic area. Ask which country a region sits in, and whether your requests ever leave it.

Data center. A building designed to house computers, with power, cooling, security and network connections. They range from a single room to campuses the size of several football fields.

Hyperscale. A very large class of data center and the companies that build them. Hyperscale sites are built for massive, shared demand. They take years to plan and connect to power.

Edge. Computing placed close to where data is created or used: in a city, a factory, a store or a cell tower, instead of in a distant region. "Edge" is a direction, not a fixed distance. Ask a vendor exactly where their edge is.

On-premises. Machines you own and run in your own building.

Colocation. Renting space, power and cooling in someone else's data center for machines you own or lease.

Modular data center. A data center built from standard, repeatable units that can be delivered and connected on site. The appeal is speed and repeatability.

Distributed infrastructure. Capacity spread across many sites instead of concentrated in a few, run so that work can move between them.

What the work is

Model. The trained AI system that does the work: answering questions, writing code, reading images. Some are open, meaning anyone can download and run them. Others are closed and only available through the company that built them.

Training. Teaching a model by running huge amounts of data through it. Done rarely, at large scale.

Inference. Using a trained model to answer a request. This is what most businesses buy, and it runs continuously.

Token. The small unit of text a model reads and writes, roughly a word or part of a word. Many AI services price by the token, often per million tokens, with separate prices for what goes in and what comes out.

Workload. A general word for a piece of computing work, such as a set of inference requests from one application.

Accelerator. A chip designed to run AI calculations quickly. GPUs are a well-known kind; there are others. A provider that supports several kinds gives you more options.

How fast and how reliable

Latency. The time between sending a request and getting the first part of the answer back. Distance, network quality and how busy the machines are all add to it. Ask how latency is measured, from where, and for which model.

Throughput. How much work a system can do in a given time, such as tokens per second. High throughput and low latency are different goals and sometimes pull against each other.

Uptime. The share of time a service is available. A promise about uptime is usually written into an SLA.

SLA (service level agreement). The contract terms that say what a provider guarantees and what happens if it falls short. Read the remedies, not only the headline figure.

Failover. What happens when one site or machine fails and work moves to another. Ask where it moves to, and whether that changes where your data is processed.

Where data goes

Data residency. A requirement that data is stored, and often processed, inside a named country or region.

Legal jurisdiction. The idea that data is subject to the laws of the country where it sits. Vendors use related words loosely, so ask what they mean in the contract.

Processing location. Where the computing on your data actually happens. It can differ from where the data is stored. For audits, this is often the question that matters.

Placement. The decision about which site runs a given request. Good placement follows rules you set, such as latency, residency or cost.

Policy. A rule a buyer sets for placement. A strict policy should never be relaxed. A softer one can allow more options when capacity is tight, within the limits you set.

How you pay

On-demand. Pay for what you use, when you use it. Flexible, usually a higher unit price.

Reserved capacity. Commit to a set amount of capacity for a period, usually at a lower unit price. You pay whether or not you use it.

Spot. Unused capacity sold cheaply, which can be taken back at short notice. Good for work that can wait; risky for work that can't.

Utilization. How much of the capacity you pay for actually does work. Idle capacity is a cost.

Egress. Charges for moving data out of a provider's network.

Proving what happened

Metering. Measuring how much work was done, for whom and where.

Delivery record. A record of what a provider committed to, where the work ran and what was delivered. It turns a promise into something a finance or compliance team can check.

Audit trail. The full history of those records over time.

What we're building

We built this list because we use these words every day. We are building Starfield Fabric, which is designed to place workloads by policy, on latency, data residency, model and load, and designed so every workload can generate an auditable record of commitment, placement and delivery.

If a proposal you're reading uses a term that isn't here, send it to us. Or talk to us about your own workloads through Starfield Fabric. To start with the basics, read what AI inference is. To see how requests could be placed, read a control tower for AI work.