Data centers are generating more heat than ever. Cooling has to keep up.

Data centers have powered the digital age for decades, and today, they’re enabling a variety of artificial intelligence (AI) applications that are transforming the world and our lives. But their power demands have only exploded recently, and rapidly.

“Twenty-five years ago, a typical data center rack may have consumed about 5kW,” said Patrick Zeng, general manager of data center thermal management at Texas Instruments. “Today’s racks can draw more than 100kW, and within a few years some could exceed a megawatt.”

All of that power must go somewhere. Nearly every watt a server draws is converted back into data center rooms as heat. Containing that heat is one of the toughest engineering challenges facing data center design today.

“When you run in the sun, your body gets hot,” Patrick said. “Your metabolism cools your body so that you don’t get a heat stroke. That’s exactly what thermal management does for a data center.”

When servers overheat, performance suffers and expenses rise. Take a large language model, for example. AI models process and generate language using tokens, the small chunks of text that serve as the basic units of computation.

“When a GPU overheats, it can’t produce tokens as quickly,” Patrick said. “So the end user experiences a slower response, and responses for data center operators are more expensive. ”

Traditional cooling infrastructure simply can’t keep pace. Engineers are now racing to rethink how data centers remove heat, whether it happens at the facility level or the server tray.

When air alone isn’t enough

For decades, air cooling was sufficient. Fans pushed cool air across hot components and carried heat away, keeping data centers operating reliably. But as AI pushed rack power beyond 20kW to 40kW, air alone can no longer remove heat quickly enough. Engineers are responding by combining air and liquid cooling.

Instead of cooling the air around a server, liquid cooling removes heat at its source. A cold plate sits directly against the chip inside a server tray, letting liquid carry heat away far more efficiently than air.

Now, engineers are applying that approach across entire data center facilities, from the coolant distribution units (CDUs) that circulate coolant between server racks and the external facility infrastructure, to the individual trays. But bringing liquid this close to electronics can become extremely costly, and this challenge requires an ecosystem of components that can move, monitor and control coolant.

Building blocks for liquid cooling

“Monitoring liquid coolant is critical to reliable operation of the entire thermal management system,” Patrick said. That’s why engineers keep constant watch over it, tracking temperature and humidity for any sign that something isn’t right.

The CDU houses the pumps and fans that form the mechanical heart of the loop, circulating coolant through the system. Motor drivers control these pumps and fans, with built-in protection against failure. Valves regulate how much coolant reaches each tray, opening or closing based on real-time demand. 

Sensors monitor flow, temperature, humidity and pressure, watching for the earliest signs of a leak or a failing component. Microcontrollers tie everything together, processing sensor data in real time and adjusting the system accordingly.

“With data centers, reliability is always the No. 1 consideration,” Patrick said. A pump that fails or a sensor that misreads can take an entire rack of AI computing components offline.

Meeting reliability standards requires precision at every step. TI’s portfolio spans the full coolant path, from the heating, ventilation and air-conditioning (HVAC) systems that cool an entire data center facility, to the CDU that feeds the racks, to the individual server tray.

“We’re not starting from scratch,” Patrick said. “We’re building on technologies in commercial heating, ventilation and air-conditioning (HVAC) systems that we’ve used for years in building automation and appliances. That means faster time to market, because the foundation is already there. And it means reliability, because this is proven technology.”

What’s next for thermal management

Even as rack power increases to the megawatt level and operators adopt new cooling technologies accordingly, one design challenge will persist for engineers: reliability.

The next frontier of reliability in data center cooling combines multiple sensors through edge AI and sensor fusion. Today’s systems often rely on a single sensor to catch a leak. Tomorrow’s systems will analyze a web of real-time signals – motor current, compressor speed, humidity, pressure and temperature – to recognize subtle patterns that no single sensor can detect. Correlating those signals is too complex for traditional rule-based software.

“Edge AI provides a unique advantage: fast response time and low latency,” Patrick said.

Whatever form thermal management takes, its purpose doesn’t change: giving engineers the tools to keep AI services responsive and data centers reliable, help operators control costs, and keep the infrastructure behind modern computing running at the pace AI now demands.