On-chip two-phase cooling of datacenters: Cooling system and energy recovery evaluation

Cooling of datacenters is estimated to have an annual electricity cost of 1.4 billion dollars in the United States and 3.6 billion dollars worldwide. Currently, refrigerated air is the most widely used means of cooling datacenter’s servers, which typically represents 40-45% of the total energy consumed in a datacenter. Based on the above issues, thermal designers of datacenters and server manufacturers now seem to agree that there is an immediate need to improve the server cooling process. The goal of the present study is to propose and simulate the performance of a novel hybrid two-phase cooling cycle with micro-evaporator elements (multi-microchannel evaporators) for direct cooling of the chips and auxiliary electronics on blade server boards (savings in energy consumption of over 60% are expected). Different working fluids were considered, namely water, HFC134a and a new, more environmentally friendly, refrigerant HFO1234ze. The results so far demonstrated that the pumping power consumption is on the order of 5 times higher for the water-cooled cycle. Additionally, a case study considering the hybrid cooling cycle applied on a datacenter and exploring the application of energy recovered in the condenser on a feedwater heater of a coal power plant was also investigated (modern datacenters require the dissipation of 5-15 MW of heat). Aspects such as minimization of energy consumption and CO2 footprint and maximization of energy recovery (exergetic efficiency) and power plant efficiency are investigated.

Experimental Evaluation and Control of Two-Phase Multi-Microchannel Evaporator Cooling Systems for High Efficiency Data Center

Duan Wu

Thermal designers of data centers and server manufacturers are showing a greater concern regarding the cooling of the new generation data centers, which consume considerably more electricity and dissipate much more waste heat, a situation that is creating a re- thinking about the most effective cooling systems for the future beyond conventional air cooling of the chips/servers. Potentially, a significantly better solution is to make use of on- chip two-phase cooling, which, besides improving the cooling performance at the chip level while also consuming less energy to drive the cooling process, also adds the capability to reuse the waste heat in a convenient manner, since higher evaporating and condensing temperatures of the two-phase cooling system (from 60-95 °C) are possible with such a new “green” cooling technology. In the present thesis, three such two-phase cooling cycles using micro-evaporation technology were experimentally evaluated with specific attention being paid to (i) controllability of the two-phase cooling system, (ii) energy consumption and (iii) overall exergetic efficiency, with the emphasis on (i). The controllers were evaluated by tracking and disturbance rejection tests, which were shown to be efficient and effective. The average temperatures of the chips were maintained below the upper limit of 85 °C of computer CPU’s for all tests evaluated in steady state and transient conditions. In general, simple SISO and SIMO strategies were sufficient to attain the requirements of control. Regarding energy and exergy analyses, the experimental results showed that all these systems can be thermodynamically improved since only about 6% of the exergy supplied is in fact recovered in the condenser in the present setup. Additionally, a series of tests covering a wide range of operating conditions under steady state regime were done. The main idea was to generate a “map of performance” of the different cooling systems in terms of energy consumption, energy recovery at the condenser and heat exchanger performance. A total of 120 tests were done which considered all combinations of the variables involved. Finally, empirical and semi-empirical correlations for overall thermal conductance and performance of all components and piping of all these systems were developed based on the experimental results obtained, which can be used for simulations and validations of potential codes developed to design and/or evaluate performance of cooling systems. An overall energy balance analysis for each system using the correlations developed showed that 99.17% of the experimental data were bounded within ± 10%.

EPFL2012

Dynamic flow control and performance comparison of different concepts of two-phase on-chip cooling cycles

Jackson Braz Marcinichen, Bruno Michel, John Richard Thome, Duan Wu

A hybrid on-chip two-phase cooling cycle specifically designed to cool server boards with chips of high performance computers was experimentally evaluated considering steady-state and transient operation of two parallel pseudo-chips and auxiliary electronics mimicking a real server board. Control strategies were developed and evaluated by reference tracking and disturbance rejection tests considering several setpoints of controlled variables. The hybrid cycle, operating with a common refrigerant R134a as the working fluid, was energetically and exergetically compared with two other cooling cycles experimentally evaluated in a previous study, one driven by an oil-free gear pump and another by an oil-free mini-compressor. The results showed that, for a specific steady state condition and heat load, respectively 28.9%, 51.9% and 62.5% of the energy out of the pump, compressor and hybrid cycles were associated with heat losses. The differences observed between the three cycles were justified firstly due to the concept of the cycles, i.e. cycles with the compressor showed as expected lower thermal performance than that with pump since its appeal is for energy recovery (benefitting from a higher condensing temperature) and secondly due to the irreversibilities observed in drivers, condenser and piping (thermal insulation). In summary, the three cycles proved to be efficient, simple and reliable concepts to cool server boards (CPUs, DIMMs etc.), showing high thermal performance and potential for heat recovery when compared with traditional air-cooling systems in current use in data centers. It can also be said that the pump cycle showed the best results in terms of energy and exergy, with the cooling and heat recovery performances reaching a maximum of about 5 and 1.8 times higher than the other cycles (worth noting that the focus in the present study was two-phase flow control and proof-of-concept of different cooling loops, meaning that no "optimal" system design was attempted and the differences above can be reduced). (C) 2013 Elsevier Ltd. All rights reserved.

Elsevier2014

Efficiency Improvements of a Thermal Power Plant by Making Use of the Waste Heat of Large Datacenters using Two-Phase On-Chip Cooling

Nicolas Lamaison, Jackson Braz Marcinichen, John Richard Thome

Cooling of datacenters is estimated to have an annual electricity cost of 1.4 billion dollars in the United States and 3.6 billion dollars worldwide. Currently, refrigerated air is the most widely used means of cooling datacenter’s servers. Modern datacenters require a heat dissipation rate in the order of 5 to 15 MW and current air cooling technology represents around 45% of the total energy consumed. Based on the above issues, thermal designers of datacenters and server manufacturers now seem to agree that there is an immediate need to improve the server cooling process. On-chip cooling research is being developed in this context to propose a new, more efficient cooling technology. This also allows the recoveryof the heat rejected by the servers in a proper way, making it possible to reuse elsewhere, in another application. The present investigation develops a case study considering two different cooling systems applied on a datacenter and exploring the application of energy recovered in the condenser on a feedwater heater of a coal power plant. The effects of the evaporating and condensing temperatures on the cooling cycle performance and the potential to recover energy, and consequently the effect on the power plant efficiency, are evaluated. The analyses consider the main objective function to be the minimization of energy consumption, the corresponding CO2 footprint and operating costs. From the datacenter’s point of view, when compared with traditional air-cooling systems, energy consumption, without considering energy recovery, can be reduced by as much as 45% when using a liquid pumping cycle and 35% when using a vapour compression cycle. From the power plant point of view, the results showed that, when the pressure of the feedwater heater is optimized, an increase of up to 6.5% of the overall power plant efficiency can be obtained when using a vapour compression cycle to cool the datacenter. Considering the vapour compression cycle and a datacenter of 100000 blades, overall savings (considering the power plant and the datacenter as a whole system) of 2170 tons of CO2 and $0.34 million per MW of electricity production were obtained. Additional investigation was developed considering the effects of partial operation of the datacenter and/or the power utility on the parameters mentioned beforehand. For such an investigation the start-up was the ideal match between a datacenter of 100000 blades and a power plant, both with 100% of operational uptime. It has been shown that some cases could lead to impossible thermodynamic operations, meaning that special attention must be given when the design of such integrated utilities (datacenter and power plant) is made.

2011

Experimental Evaluation and Control of Two-Phase Multi-Microchannel Evaporator Cooling Systems for High Efficiency Data Center

Duan Wu

EPFL2012

Dynamic flow control and performance comparison of different concepts of two-phase on-chip cooling cycles

Jackson Braz Marcinichen, Bruno Michel, John Richard Thome, Duan Wu

Elsevier2014

Efficiency Improvements of a Thermal Power Plant by Making Use of the Waste Heat of Large Datacenters using Two-Phase On-Chip Cooling

Nicolas Lamaison, Jackson Braz Marcinichen, John Richard Thome

Cooling of datacenters is estimated to have an annual electricity cost of 1.4 billion dollars in the United States and 3.6 billion dollars worldwide. Currently, refrigerated air is the most widely used means of cooling datacenter’s servers. Modern datacenters require a heat dissipation rate in the order of 5 to 15 MW and current air cooling technology represents around 45% of the total energy consumed. Based on the above issues, thermal designers of datacenters and server manufacturers now seem to agree that there is an immediate need to improve the server cooling process. On-chip cooling research is being developed in this context to propose a new, more efficient cooling technology. This also allows the recoveryof the heat rejected by the servers in a proper way, making it possible to reuse elsewhere, in another application. The present investigation develops a case study considering two different cooling systems applied on a datacenter and exploring the application of energy recovered in the condenser on a feedwater heater of a coal power plant. The effects of the evaporating and condensing temperatures on the cooling cycle performance and the potential to recover energy, and consequently the effect on the power plant efficiency, are evaluated. The analyses consider the main objective function to be the minimization of energy consumption, the corresponding CO2 footprint and operating costs. From the datacenter’s point of view, when compared with traditional air-cooling systems, energy consumption, without considering energy recovery, can be reduced by as much as 45% when using a liquid pumping cycle and 35% when using a vapour compression cycle. From the power plant point of view, the results showed that, when the pressure of the feedwater heater is optimized, an increase of up to 6.5% of the overall power plant efficiency can be obtained when using a vapour compression cycle to cool the datacenter. Considering the vapour compression cycle and a datacenter of 100000 blades, overall savings (considering the power plant and the datacenter as a whole system) of 2170 tons of CO2 and $0.34 million per MW of electricity production were obtained. Additional investigation was developed considering the effects of partial operation of the datacenter and/or the power utility on the parameters mentioned beforehand. For such an investigation the start-up was the ideal match between a datacenter of 100000 blades and a power plant, both with 100% of operational uptime. It has been shown that some cases could lead to impossible thermodynamic operations, meaning that special attention must be given when the design of such integrated utilities (datacenter and power plant) is made.

2011