Introduction

Power levels for compute racks in data centers have increased over the last several years, driven by the more advanced central processing unit (CPU) and graphics processing unit (GPU) architectures that large language models and other AI workloads require. As a result, the importance of thermal management has grown as well.

The main job of any thermal management system is simple in concept: absorb heat from areas where it isn't wanted and transfer it where it can be dispersed safely. As compute racks generate more heat, keeping the processors inside those racks operating within an acceptable temperature range requires different thermal management techniques to optimize performance. This white paper provides an overview of how thermal management systems have evolved as power levels have increased and looks at the thermal management systems that may be needed to handle the power levels of the not-so-distant future.

Related reading: "Data centers are generating more heat than ever. Cooling has to keep up."

One way to visualize the power levels of compute racks, and the amount of heat they generate, is to compare a rack to a traditional wood- or gas-burning fireplace found in homes. These fireplaces typically put out the equivalent of about 10kW of heat in roughly the same footprint as a compute rack, a load any typical air conditioning system can handle without issue. At a 50kW rack power level, though, that's 10 fireplaces packed into that footprint, and air cooling is no longer capable of regulating the room temperature with that amount of heat load. A 1.5MW rack, the equivalent of 150 fireplaces in that space, really puts the challenge of data center thermal management into perspective. Fortunately, liquid cooling offers a heat-absorption capability that's 3,000 times greater than air for the same volume.

Figure 1 illustrates how quickly the challenge of cooling has escalated, plotting rack power levels of the past, present and future against the architectures capable of handling them, and where each architecture reaches its limit. Rack power density in AI data centers has climbed from roughly 5kW to more than 100kW. Some AI clusters now approach 200kW per rack, with projections in the next few years to levels as high as 1.5MW. That's why cooling architectures are shifting from air cooling to liquid cooling to keep pace