A single NVIDIA H100 GPU draws 700 watts under full load. Pack eight of them into a server node, and you are looking at 5.6 kilowatts just from GPUs --- before CPUs, networking, storage, and power conversion losses push the node past 10 kW. Scale that to a 100-node cluster, and you have a megawatt of heat to manage. Traditional air cooling was not designed for this density, and it shows.
The shift to liquid cooling in AI data centers is not a luxury or a future trend. It is a present-day necessity driven by thermal physics. NVIDIA's latest Blackwell Ultra (GB300) and B200 GPUs are designed exclusively for liquid-cooled deployments. The NVL72 rack, which houses 72 GPUs drawing over 100 kW total, simply cannot be air-cooled.
For organizations running GPU workloads on platforms like io.net, the cooling infrastructure is handled by data center partners. But understanding liquid cooling matters because it directly affects the hardware you can access, the pricing you pay, and the sustainability profile of your AI operations.
Why Air Cooling Has Hit Its Limit
The Thermal Density Problem
| Generation | GPU TDP | 8-GPU Node Power | Rack Power (4 nodes) | Air Coolable? |
|---|---|---|---|---|
| V100 (2017) | 300W | 2.4 kW | ~12 kW | Yes |
| A100 (2020) | 400W | 3.2 kW | ~16 kW | Yes (marginal) |
| H100 SXM (2023) | 700W | 5.6 kW | ~28 kW | Barely |
| B200 (2025) | 1,000W | 8.0 kW | ~40 kW | No |
| GB300 (2026) | 1,400W | 11.2 kW | ~56 kW | No |
Air cooling tops out at roughly 30-35 kW per rack in a well-designed data center. Anything above that requires liquid cooling. Since every current-generation and next-generation NVIDIA GPU exceeds this threshold in dense configurations, liquid cooling has become the baseline for serious AI infrastructure.
The Efficiency Gap
Air cooling's inefficiency compounds at scale:
- Fan power: Air-cooled data centers spend 30-40% of total energy on cooling
- Temperature limits: Higher ambient temperatures force GPU throttling, reducing performance by 5-15%
- Density constraints: Air cooling limits rack density, requiring more floor space per GPU
- PUE impact: Air-cooled AI facilities typically achieve PUE of 1.4-1.6. Liquid-cooled facilities achieve 1.05-1.15.
A PUE of 1.4 means 40% energy overhead for cooling. At 1.1, the overhead drops to 10%. For a 10 MW facility running 24/7, that is $2.6 million per year in energy savings (at $0.10/kWh).
Types of Liquid Cooling for GPU Clusters
Direct-to-Chip (Cold Plate) Cooling
The most common approach for GPU clusters. Coolant flows through metal cold plates mounted directly on GPU packages, absorbing heat at the source.
How it works: Chilled water or dielectric fluid circulates through cold plates on each GPU. Heat transfers from the GPU die through a thermal interface material to the cold plate, then to the fluid. The heated fluid flows to a cooling distribution unit (CDU) outside the rack, where it exchanges heat with a facility water loop.
Advantages: - Removes 80-90% of heat directly from the hottest components - Compatible with existing server form factors - Proven at scale (used by NVIDIA DGX, HPE Cray, Dell PowerEdge) - Lower fluid volumes than immersion cooling
Limitations: - Still requires some air cooling for non-GPU components (RAM, NVMe, VRMs) - Leak risk at connector points - Requires plumbing infrastructure per rack
Immersion Cooling (Single-Phase)
Servers are submerged in a non-conductive dielectric fluid bath. Heat transfers from all components to the fluid, which circulates to external heat exchangers.
Advantages: - Cools all components, not just GPUs - No fans needed (silent operation) - Enables extreme density (up to 100+ kW per rack) - Very low PUE (1.02-1.06)
Limitations: - Requires specialized server designs (no standard form factors) - Higher upfront cost for tanks and fluid - Maintenance requires draining fluid - Limited vendor ecosystem
Two-Phase Immersion Cooling
Similar to single-phase, but the dielectric fluid boils at the component surface, absorbing significantly more heat through phase change. Vapor rises, condenses on a heat exchanger, and drips back.
Advantages: - Highest cooling capacity per unit volume - Self-regulating (boiling point acts as temperature governor) - Exceptional PUE (1.01-1.03)
Limitations: - Most expensive option - Fluid management is complex - Limited to specific dielectric fluids (expensive, environmentally regulated)
Comparison Table
| Method | Max Rack Power | PUE | Relative Cost | Maturity |
|---|---|---|---|---|
| Air cooling | 30-35 kW | 1.4-1.6 | Low | Mature |
| Direct-to-chip | 60-80 kW | 1.10-1.20 | Medium | Mature |
| Single-phase immersion | 100-200 kW | 1.02-1.06 | High | Growing |
| Two-phase immersion | 200+ kW | 1.01-1.03 | Very high | Emerging |
Access Liquid-Cooled GPU Clusters on io.net
io.net's data center partners provide liquid-cooled H100 and upcoming GB300 clusters. Get the density and efficiency of cutting-edge cooling without building your own facility.
Energy and Cost Impact
Power Usage Effectiveness (PUE) Analysis
PUE measures total facility power divided by IT equipment power. Lower is better.
| Cooling Method | PUE | Annual Energy Cost (10 MW IT load, $0.10/kWh) | Cooling Overhead Cost |
|---|---|---|---|
| Air cooling (traditional) | 1.50 | $13.14M | $4.38M |
| Air cooling (optimized) | 1.35 | $11.83M | $3.07M |
| Direct-to-chip | 1.12 | $9.81M | $1.05M |
| Immersion (single-phase) | 1.05 | $9.20M | $0.44M |
Switching from traditional air cooling to direct-to-chip liquid cooling saves approximately $3.3 million annually per 10 MW of IT load. That savings flows into lower GPU rental prices on platforms like io.net.
Total Cost of Ownership Impact
Liquid cooling has higher upfront costs but dramatically lower operating costs:
| Cost Category | Air Cooled | Direct-to-Chip | Immersion |
|---|---|---|---|
| Cooling infrastructure (per rack) | $5,000-$10,000 | $15,000-$30,000 | $30,000-$60,000 |
| Annual energy savings vs. air | -- | $3,000-$5,000/rack | $4,000-$6,000/rack |
| Payback period | -- | 3-5 years | 5-8 years |
| Floor space efficiency | 1.0x | 1.5-2.0x | 2.0-3.0x |
For high-utilization AI workloads running 24/7, the payback period on liquid cooling is closer to 2-3 years.
GPU Performance Under Different Cooling Methods
Cooling directly affects GPU performance through thermal throttling avoidance:
| Scenario | GPU Temperature | Clock Speed | Performance vs. Spec |
|---|---|---|---|
| Air cooled, 25C ambient | 75-85C | Base-Boost | 90-100% |
| Air cooled, 35C ambient | 85-90C | Throttled | 80-90% |
| Direct-to-chip, 25C coolant | 55-65C | Max Boost sustained | 100% |
| Immersion, 35C fluid | 50-60C | Max Boost sustained | 100% |
In hot climates or dense deployments, liquid cooling provides a measurable performance advantage --- your GPUs sustain higher clock speeds for longer.
Sustainability and Carbon Impact
Carbon Footprint Comparison
For a 1,000-GPU H100 cluster running 24/7 for one year:
| Cooling Method | Total Energy (MWh/yr) | CO2 (metric tons, US avg grid) | Reduction vs. Air |
|---|---|---|---|
| Air cooling (PUE 1.5) | 9,198 | 3,771 | Baseline |
| Direct-to-chip (PUE 1.12) | 6,868 | 2,816 | 25% |
| Immersion (PUE 1.05) | 6,439 | 2,640 | 30% |
For organizations with ESG commitments or carbon reduction targets, liquid-cooled GPU infrastructure provides a meaningful reduction in Scope 2 emissions.
Water Usage
A common concern with liquid cooling: does it use more water? The answer depends on the cooling method.
- Direct-to-chip with dry coolers: Zero water consumption (closed loop, air-cooled CDU)
- Direct-to-chip with cooling towers: Uses water for evaporative cooling of the facility loop
- Immersion with dry coolers: Zero water consumption
Many modern liquid cooling deployments use dry coolers exclusively, making them less water-intensive than air-cooled facilities that rely on evaporative cooling towers.
Planning a Liquid-Cooled GPU Deployment
For io.net Users
When renting GPU capacity through io.net, the cooling infrastructure is managed by data center partners. What matters to you:
- Specify SXM form factor: H100 SXM and B200 SXM GPUs in io.net's network are typically in liquid-cooled facilities, ensuring maximum performance.
- Check GPU temperature: If monitoring shows sustained temperatures above 80C, request a migration to a liquid-cooled cluster.
- Prefer NVLink connectivity: Liquid-cooled facilities tend to offer denser, better-interconnected clusters.
For Data Center Operators Joining io.net
If you operate a data center and want to contribute GPU capacity to io.net's network:
| Investment | Cost Range | Capacity Added |
|---|---|---|
| Add cold plates to existing racks | $15K-$30K per rack | 40-60 kW per rack |
| Rear-door heat exchangers | $8K-$15K per rack | 20-35 kW per rack |
| Full CDU deployment | $50K-$100K per row | 200-400 kW per row |
| Immersion cooling pods | $100K-$200K per pod | 100-300 kW per pod |
Retrofit vs. New Build
| Approach | Timeline | Cost per kW | Best For |
|---|---|---|---|
| Retrofit with cold plates | 2-4 months | $200-$400/kW | Existing facilities with water |
| Retrofit with rear-door HX | 1-3 months | $150-$300/kW | Quick wins, moderate density |
| New build, direct-to-chip | 12-18 months | $300-$500/kW | Purpose-built AI facility |
| New build, immersion | 18-24 months | $500-$800/kW | Maximum density, best PUE |
Future Trends in GPU Cooling
2026-2028 Outlook
- Liquid cooling becomes mandatory: GB300 and Vera Rubin architectures require it. Air-cooled facilities cannot host next-gen GPUs.
- Waste heat recovery: Data centers will sell GPU waste heat for district heating, offsetting energy costs by 10-20%.
- Direct-to-chip standardization: OCP (Open Compute Project) standards for liquid cooling connectors will reduce proprietary lock-in.
- Edge liquid cooling: Smaller, pre-fabricated liquid-cooled pods for edge AI deployments.
Impact on GPU Cloud Pricing
As liquid cooling becomes standard, the cost premium for liquid-cooled GPUs disappears. Instead, air-cooled facilities face a penalty:
- Cannot host latest GPU generations
- Higher operating costs passed to customers
- Lower density means higher floor space costs
Platforms like io.net that aggregate capacity from modern, liquid-cooled facilities will maintain a pricing advantage over legacy infrastructure.
Frequently Asked Questions
Do I need to worry about cooling if I rent GPUs on io.net?
No. Cooling is handled by the data center partner. However, you benefit from liquid cooling through higher GPU performance (no thermal throttling) and lower prices (reduced operating costs passed through to you).
Is liquid cooling safe? What about leaks?
Modern direct-to-chip cooling systems use leak detection sensors, drip trays, and automatic shutoff valves. The risk of damaging equipment from leaks is very low in well-maintained facilities. Immersion cooling uses non-conductive fluid, so a leak does not short-circuit components.
How much does liquid cooling save on electricity?
Typically 25-35% reduction in total facility energy consumption compared to air cooling. For a 10 MW facility, that translates to $2.5-$4 million per year in energy savings.
Can existing data centers be retrofitted?
Yes. Adding cold plates to existing GPU servers is the most common retrofit. It requires installing plumbing (supply and return lines), a CDU per row, and connecting to the facility cooling plant. Timeline: 2-4 months per phase.
Does liquid cooling improve GPU performance?
Yes. Liquid-cooled GPUs maintain lower junction temperatures (55-65C vs. 75-85C air-cooled), allowing sustained boost clocks. Real-world performance improvement: 5-15% depending on workload and ambient conditions.
What is PUE and why does it matter?
Power Usage Effectiveness measures total facility power divided by IT power. PUE of 1.0 means zero cooling overhead (theoretical ideal). Air-cooled facilities: 1.4-1.6. Liquid-cooled: 1.05-1.15. Lower PUE = lower operating costs = lower GPU rental prices.
Are NVIDIA's newest GPUs liquid-cooling only?
The NVL72 rack (B200 and GB300) requires liquid cooling. SXM form factor GPUs are designed for liquid cooling but can run in well-designed air-cooled facilities with reduced density. PCIe GPUs remain air-coolable.
How does liquid cooling affect sustainability reporting?
Liquid cooling reduces Scope 2 emissions by 25-35% compared to equivalent air-cooled infrastructure. Many organizations include GPU compute in their carbon accounting, making cooling method a relevant ESG factor.
Conclusion
Liquid cooling is no longer optional for cutting-edge AI infrastructure. The thermal demands of modern GPUs --- 700W for H100, 1,000W for B200, 1,400W for GB300 --- have pushed beyond what air cooling can handle safely and efficiently.
For teams renting GPU capacity through io.net, this translates into tangible benefits: better performance from thermally optimized GPUs, lower prices from energy-efficient facilities, and access to next-generation hardware that physically requires liquid cooling.
The shift is happening now. Facilities investing in liquid cooling infrastructure are the ones that will host the next generation of AI compute. io.net's network includes these facilities, giving you access to the best-cooled, highest-performing GPUs at competitive prices.
Access liquid-cooled GPU clusters through io.net. Explore available hardware and start your next AI workload today.