The Evolution of AI and Thermal Considerations in the Data Center

Executive summary

Artificial Intelligence (AI) and High-Performance Computing (HPC) workloads are driving dramatic increases in compute density across data centers, significantly elevating both thermal output and power demand. Traditional air-cooling methods are struggling to keep pace with cooling the data center infrastructure and components that drive and enable AI and HPC workloads.  As the power profile of infrastructure continues to dramatically increase the power and thermal bounds of the components, these components are rapidly reaching the limits of their effectiveness. 

Direct Liquid Cooling (DLC) is an effective methodology that enables cooling of the server and server components including CPU, GPUs, DPUs, NICs and solid-state drives (SSDs), and presents a transformative opportunity to enhance thermal efficiency and reduction of operational costs. The Solidigm fully liquid-cooled, hot-swappable SSD (Solidigm™ D7-PS1010 E1.S), enables the design, development, and deployment of next generation fanless and thermally optimized AI server architectures, addressing the dual challenges of thermal management and cost reduction in modern data centers.

AI workloads and the rising thermal challenge

As AI continues to evolve, server workloads are becoming increasingly compute and power intensive, leading to rack power densities exceeding 100 kW. These high-performance GPU clusters generate unprecedented heat flux, far beyond what traditional air-cooling systems can handle. This shift is pushing many data centers toward adopting new cooling paradigms. While conventional air-cooled data centers typically operate with a Power Usage Effectiveness (PUE) around 2.0,¹ next-generation liquid-cooled data centers are achieving PUEs as low as 1.2,¹ signaling a substantial improvement in energy efficiency. The byproduct and added advantage of lowering the PUE is the reduction of the data center operational cost which can be significant.

Cooling accounts for 35% to 45%¹ of the energy costs in a traditionally air-cooled data center. Liquid cooling offers a compelling alternative, reducing energy consumption by up to 10% and improving PUE by approximately 15%. These enhancements not only lower operational costs but also contribute to environmental sustainability. ABI Research projects the liquid cooling market to reach $3.7 billion by 2030, propelled by demand for greater compute density and energy efficiency gains ranging from 300% to 500%.²

Liquid cooling techniques & infrastructure

Data centers are increasingly adopting a range of liquid cooling techniques, including direct-to-chip cold plates, rear-door heat exchangers, and immersion cooling systems. Cold plate systems, in particular, can support rack densities up to 120 kW, as seen in deployments featuring the NVIDIA GB300 platforms. While liquid cooling retrofits are feasible, the most efficient deployments are often seen in greenfield data center builds, where infrastructure can be purpose-designed to accommodate DLC systems from the outset.

In high-density AI servers, which drive the leading edge of the power and performance profile, consideration needs to be given to all server components that add to the server thermal load. Traditional air-based cooling methods which utilize forced air that is pushed by fans into and across the server components are proving inadequate for these high-performance components. Liquid cooling offers a superior thermal management solution, with liquids possessing thermal conductivity advantages of 1000x to 3000x³ over air, enabling better heat dissipation and improved overall system performance and longevity.

Next generation liquid-cooled SSD solution

Traditional approaches to liquid cooling SSDs include placing cold plates on both sides of a SSD. Cooling fluid is pumped through the cold plate which cools the SSD and components that reside within and on both sides of the SSD.

In a liquid-cooled SSD implementation, the cold plate and SSD typically reside in an enclosure allowing for device structural integrity, serviceability, and optimal thermal transfer rate. To maximize the thermal exchange between the cold plate and SSD, a Thermal Interface Material (TIM) is used. The TIM interfaces the SSD to the cold plate, achieving a high surface contact area between the cold plate and the SSD enabling the highest thermal efficiency.

The Solidigm D7-PS1010 9.5mm E1.S SSD introduces a breakthrough in storage thermal management by utilizing a single cold plate to cool both sides of the SSD. This design increases the drive storage density and capacity of the server due to the reduction in the number of cold plates required to cool a single SDD. In addition, the Solidigm D7-PS1010 9.5mm E1.S SSD and cold plate enclosure, designed in collaboration with NVIDIA is capable of dissipating greater than 25W with a single-sided cold plate. It also enables full hot-swap capability, which is a significant challenge with a dual cold plate solution, and is future-ready to support and enable the next generation of fanless server configurations.

Available in form factors of 9.5mm for air-cooled and liquid-cooled deployments, and 15mm for air-cooled-only deployments, the drive offers flexibility for diverse infrastructure needs. The Solidigm D7-PS1010 9.5mm E1.S SSD enables the creation of compact, fully liquid-cooled 1U servers, eliminating the need for internal fans and allowing for higher density and thermal efficiency.

By extending liquid cooling to SSDs, data centers can reduce thermal throttling and improve system reliability. The fully liquid-cooled, hot-swappable design of Solidigm SSDs supports enterprise-grade serviceability. It also facilitates the development of ultra-compact, fanless 1U servers, which enhance rack-level density, simplify airflow design, and reduce capital and operational expenditures. These servers can also lower total energy consumption and help organizations meet Environmental, Social, and Governance (ESG) goals.

By incorporating liquid cooling of SSDs, data centers can minimize the footprint dedicated to cooling systems, allowing for a more compact layout. This space efficiency enables either higher compute density within the same facility or smaller overall data center size without sacrificing performance. The reduction in cooling-related space, which can account for a substantial portion of non-rack areas in air-cooled setups, leads to an overall space saving of approximately 23% to 27%.⁴

Summary

The path forward for AI infrastructure lies in adopting subsystem-wide liquid cooling, allowing all server components to operate at their targeted operational efficiency and functionality, delivering the highest performance. Incorporating fully liquid-cooled SSDs like the Solidigm D7-PS1010 from the outset of new data center builds would yield the most benefit. For existing deployments, retrofitting will require infrastructure updates, including the implementation of cooling distribution units. Evaluating improvements in PUE and total cost of ownership (TCO) from reduced HVAC and server maintenance will help validate investment. Future designs must harmonize cooling across CPUs, GPUs, DPUs, NICs, and storage to fully realize operational and sustainability benefits.

AI-driven workloads are reshaping data center architectures and pushing traditional cooling to its limits. Liquid cooling, extended to storage devices, provides an opportunity to reduce operational costs through enhancing thermal efficiency. Solidigm fully liquid-cooled SSDs provides a path to thermally optimized, next generation fanless, high-density server designs that enhance operational efficiency, while cutting energy use and costs. These innovations position data centers to meet the demands of tomorrow’s AI applications with greater agility and sustainability.