Why AI chips need so much cooling
Run a laptop on your knees for an hour and you will feel it. Nothing is burning. The laptop is simply turning the electricity it uses into warmth, because that is where electricity ends up.
An AI data center does the same thing, many thousands of times over. This piece explains why AI chips produce so much heat, why that heat has to be removed, and why cooling now shapes where and how sites are built.
Where the power goes
A chip uses electricity to switch tiny circuits on and off. Each switch does useful work, such as a step in a calculation. But almost none of the energy leaves the chip as anything other than heat.
That is a basic rule of physics, not a design flaw. The electricity a computer draws is, in the end, the heat it gives off. So a site's power bill and its heat problem are really the same number seen from two sides. (We explain why AI draws so much power in Why AI uses so much electricity, in plain words.)
Why AI chips run hotter
AI work is mostly huge amounts of simple maths done in parallel. GPUs and similar chips are built to do exactly that, with many processing units working at once.
They also tend to run flat out for long periods. A training job can keep chips busy for weeks. Inference keeps them busy whenever people are asking questions. (See Inference, explained for people who don't build AI.)
On top of that, AI servers pack many chips close together, and racks hold many servers. The more power in a small space, the more heat in that same space. This is often called rack density, and for AI it has been rising.
What heat does to a chip
Chips have a safe operating temperature. Above it, they protect themselves by slowing down, which means less work for the same electricity. Run too hot for too long and parts wear out faster or fail.
So cooling is not a comfort feature. It is what lets the hardware deliver the work it was bought for, for as long as it was expected to last.
Where the heat has to go
Removing heat from a chip is only the first step. The heat then has to travel out of the server, out of the rack, out of the room, and finally out of the building.
At each step, something carries it. That might be air pushed by fans, or a liquid pumped through pipes. Eventually it reaches the outside, usually through cooling towers, dry coolers or chillers. Some sites put part of it to use instead. (We look at that in What can be done with a data center's waste heat.)
Every step uses some energy of its own. Fans, pumps and chillers all draw power, which is part of why the total electricity a site needs is more than its computers alone use.
Where the heat goes
- Power inThe chip uses electricity to switch tiny circuits on and off.
- Heat outAlmost all of that energy leaves the chip as heat.
- Out of the serverAir pushed by fans, or liquid in pipes, carries it away.Server, rack, room
- Out of the buildingCooling towers, dry coolers or chillers release it outside.
Why cooling shapes the whole site
Because heat tracks power so closely, cooling affects almost every big decision.
Hardware choice. Denser AI machines may need a cooling method the site was not built for. (See Air cooling or liquid cooling for AI?)
Location. Climate matters. Cooler air and access to water, or the ability to avoid using water, change how much energy cooling takes.
Water. Some cooling systems use water, often by letting it evaporate. Others are designed to use little or none. Local water supply and rules can decide which is acceptable.
Layout. Pipes, air paths and heat-rejection equipment need space, and they need to be planned before the racks arrive.
Cost. Cooling equipment and the energy it uses are part of what a unit of AI work really costs. (See Hidden costs of AI capacity that aren't on the quote.)
Questions to ask about cooling
If you are buying AI capacity or hosting it, these questions are worth asking.
- What rack density is the site designed to cool?
- Is cooling by air, by liquid, or a mix?
- How much of the site's power goes to cooling rather than computing?
- Does cooling use water, and how is local supply protected?
- What happens to the chips if part of the cooling fails?
- Is any of the heat reused?
What we are building
We're building cooling into the design of the unit, not adding it afterwards.
We're building standardized compute close to power, connectivity and demand, joined by one programmable network for AI inference. The network is designed to place workloads by policy, on latency, data residency, model and load, and every workload is designed to generate an auditable record of commitment, placement and delivery.
If you need AI capacity and want to know how it is cooled, use Request capacity below.
Related
Why AI uses so much electricity, in plain words · Inference, explained for people who don't build AI · Hidden costs of AI capacity that aren't on the quote · Starfield Fabric