0
Research Papers: Heat Transfer in Manufacturing

Comprehensive Experimental and Computational Analysis of a Fully Contained Hybrid Server Cabinet

[+] Author and Article Information
Kourosh Nemati

Mechanical Engineering Department,
State University of New York,
Binghamton, NY 13902
e-mail: knemati@binghamton.edu

Husam A. Alissa, Bruce T. Murray, Bahgat G. Sammakia

Mechanical Engineering Department,
State University of New York,
Binghamton, NY 13902

Russell Tipton

Emerson Network Power,
Columbus, OH 43085

Mark J. Seymour

Future Facilities,
London SE1 7HX, UK

1Corresponding author.

Contributed by the Heat Transfer Division of ASME for publication in the JOURNAL OF HEAT TRANSFER. Manuscript received May 17, 2016; final manuscript received February 28, 2017; published online April 11, 2017. Assoc. Editor: Milind A. Jog.

J. Heat Transfer 139(8), 082101 (Apr 11, 2017) (12 pages) Paper No: HT-16-1281; doi: 10.1115/1.4036100 History: Received May 17, 2016; Revised February 28, 2017

The rapid growth in the number of data centers combined with the high-density heat dissipation of computer and telecommunications equipment has made energy efficient thermal management of data centers a key research area. Localized hybrid air–water cooling is one approach to more effectively control the cooling when there is wide variation in the amount of dissipation in neighboring racks while the traditional air cooling approach requires overprovisioning. In a closed, hybrid air–water cooled server cabinet, the generated heat is removed by a self-contained system that does not interact with the room level air cooling system. Here, a hybrid-cooled enclosed cabinet and all its internal components were characterized experimentally in steady-state mode (e.g., experimentally determined heat-exchanger effectiveness and IT characterization). Also, a comprehensive numerical model of the cabinet was developed and validated using the experimental data. The computational model employs full numerical modeling of the cabinet geometry and compact models to represent the servers and the air/water heat exchanger. The compact models were developed based on experimental flow and thermal characterization of the internal components. The cabinet level model has been used to simulate a number of operating scenarios relevant to data center applications such as the effect of air leakage within the cabinet. The effect of the air side and the water side failure of the cooling system on the IT performance were investigated experimentally. A comparison was made of the amount of time required to exceed the operating temperature limit for the two scenarios.

Copyright © 2017 by ASME
Your Session has timed out. Please sign back in to continue.

References

Wei, J. , 2011, “ Hybrid Cooling Technology for Large-Scale Computing Systems: From Back to the Future,” ASME Paper No. IPACK2011-52045.
Schmidt, R. , Chu, R. C. , Ellsworth, M. , Iyengar, M. , Porter, D. , Kamath, V. , and Lehman, B. , 2005, “ Maintaining Datacom Rack Inlet Air Temperatures With Water Cooled Heat Exchanger,” ASME Paper No. IPACK2005-73468.
Iyengar, M. , David, M. , Parida, P. , Kamath, V. , Kochuparambil, B. , Graybill, D. , Schultz, M. , Gaynes, M. , Simons, R. , Schmidt, R. , and Chainer, T. , 2012, “ Server Liquid Cooling With Chiller-Less Data Center Design to Enable Significant Energy Savings,” 28th Annual IEEE Semiconductor Thermal Measurement and Management Symposium (SEMI-THERM), San Jose, CA, Mar. 18–22, pp. 212–223.
David, M. P. , Iyengar, M. K. , Parida, P. , Simons, R. E. , Schultz, M. , Gaynes, M. , Schmidt, R. , and Chainer, T. , 2012, “ Impact of Operating Conditions on a Chiller-Less Data Center Test Facility With Liquid Cooled Servers,” 13th IEEE Intersociety Conference on Thermal and Thermomechanical Phenomena in Electronic Systems (ITherm), San Diego, CA, May. 30–Jun. 1, pp. 562–573.
Rambo, J. , and Joshi, Y. , 2005, “ Thermal Performance Metrics for Arranging Forced Air Cooled Servers in a Data Processing Cabinet,” ASME J. Electron. Packag., 127(4), pp. 452–459. [CrossRef]
Nemati, K. , Murray, B. T. , and Sammakia, B. , 2014, “ Experimental Characterization and Modeling of a Water-Cooled Server Cabinet,” IEEE Intersociety Conference on Thermal and Thermomechanical Phenomena in Electronic Systems (ITherm), Orlando, FL, May. 27–30, pp. 723–728.
Nemati, K. , Gao, T. , Murray, B. T. , and Sammakia, B. , 2014, “ Experimental Characterization of the Rear Door Fans and Heat Exchanger of a Fully-Enclosed, Hybrid-Cooled Server Cabinet,” 30th Annual IEEE Semiconductor Thermal Measurement and Management Symposium (SEMI-THERM), San Jose, CA, Mar. 9–13, pp. 1435–1443.
Nemati, K. , Alissa, H. , Murray, B. T. , Sammakia, B. , and Seymour, M. , 2015, “ Experimentally Validated Numerical Model of a Fully-Enclosed Hybrid Cooled Server Cabinet,” ASME Paper No. IPACK2015-48244.
Douchet, F. , Nortershauser, D. , Le Masson, S. , and Glouannec, P. , 2015, “ Experimental and Numerical Study of Water-Cooled Datacom Equipment,” Appl. Therm. Eng., 84, pp. 350–359. [CrossRef]
Schmidt, R. , Iyengar, M. , Porter, D. , Weber, G. , Graybill, D. , and Steffes, J. , 2010, “ Open Side Car Heat Exchanger That Removes Entire Server Heat Load Without any Added Fan Power,” 12th IEEE Intersociety Conference on Thermal and Thermomechanical Phenomena in Electronic Systems (ITherm), Las Vegas, NV, Jun. 2–5, pp. 1–6.
Dietze, C., 2006, “ CoolTherm Manual, Server Cabinet With Integrated Liquid Cooling, 12–25/35 KW,” Knurr, Lommatzsch, Germany, accessed Mar. 13, 2017, www.knurrusa.com
Joshi, Y. , and Kumar, P. , 2012, Energy Efficient Thermal Management of Data Centers, Springer Science & Business Media, New York.
Cane, G. , 2012, “ A Road Map to a Fully Deployed Data Centre,” White Paper, CBRE, Global Corporate Services, Wilmington, DE.
Samadiani, E. , Joshi, Y. , Hamann, H. , Iyengar, M. K. , Kamalsy, S. , and Lacey, J. , 2012, “ Reduced Order Thermal Modeling of Data Centers Via Distributed Sensor Data,” ASME J. Heat Transfer, 134(4), p. 041401. [CrossRef]
Bana, M. , Docca, A. , and Davis, S. , 2014, “ An ACE Performance Assessment Case Study From Compromised to Optimized One Data Center: $10 Million Saved,” White Paper, Future Facilities, San Jose, CA.
Alissa, H. , Nemati, K. , Sammakia, B. , Schneebli, K. , Roger, S. , and Seymour, M. , 2016, “ Chip to Facility Ramifications of Containment Solution on IT Airflow and Uptime,” IEEE Trans. Compon. Packag. Manuf. Technol., 6(1), pp. 67–78. [CrossRef]
Wang, C.-C. , Tsai, Y.-M. , and Lu, D.-C. , 1998, “ Comprehensive Study of Convex-Louver and Wavy Fin-and-Tube Heat Exchangers,” J. Thermophys. Heat Transfer, 12(3), pp. 423–430. [CrossRef]
Arghode, V. K. , Sundaralingam, V. , Joshi, Y. , and Phelps, W. , 2013, “ Thermal Characteristics of Open and Contained Data Center Cold Aisle,” ASME J. Heat Transfer, 135(6), p. 061901. [CrossRef]
Degree Control, 2011, “Airflow and Airflow Temperature Sensors for the ATM2400 Measurement System, The AccuSense User's Guide,” Degree Control, Milford, NH.
Alissa, H. , Nemati, K. , Sammakia, B. , Seymour, M. , Schneebli, K. , and Schmidt, R. , 2015, “ Experimental and Numerical Characterization of a Raised Floor Data Center Using Rapid Operational Flow Curves Model,” ASME Paper No. IPACK2015-48234.
Future Facilities, 2009, “ 6Sigma Room User's Guide,” Future Facilities, San Jose, CA.
Geng, H. , 2014, Data Center Handbook, Wiley, London.
Pardey, Z. M. , and VanGilder, J. W. , 2014, “ Further Exploration of a Compact Transient Server Model,” IEEE Intersociety Conference on Thermal and Thermomechanical Phenomena in Electronic Systems (ITherm), Orlando, FL, May 27–30, pp. 1322–1329.
Liebert, 2010, “Coolant Distribution Unit, 100 kW Capacity, Installation Manual,” Emerson Liebert, Columbus, OH.

Figures

Grahic Jump Location
Fig. 1

Illustration of the cabinet air flow circulation: (a) top view and (b) side view

Grahic Jump Location
Fig. 2

Experimental setup of enclosed cabinet (with open doors): (a) front side and (b) rear side

Grahic Jump Location
Fig. 3

Grid of 17 velocity/temperature sensors at upstream of the heat exchanger box: (a) grid and (b) sensor

Grahic Jump Location
Fig. 4

V-shaped heat exchanger top view

Grahic Jump Location
Fig. 5

Experimentally measured effectiveness

Grahic Jump Location
Fig. 6

Server thermal mass characterization experimental setup

Grahic Jump Location
Fig. 7

Measured impedance, fan, and flow curves for 2 RU server

Grahic Jump Location
Fig. 8

Comparison between theoretical and effective flow curves

Grahic Jump Location
Fig. 9

Rear door fan curves for three fan speeds

Grahic Jump Location
Fig. 10

Rear door image and modeled flow results: (a) front view and (b) rear view

Grahic Jump Location
Fig. 11

Calibration of heat exchanger models: (a) thin heat exchanger model and (b) V-shaped heat exchanger model

Grahic Jump Location
Fig. 12

Computational model of the cabinet: (a) cabinet front side and (b) an example of the air flow pattern in the cabinet

Grahic Jump Location
Fig. 13

The air velocity distribution at the inlet of the heat exchanger (a) and (b) for the higher fan speed case and (c) and (d) for the lower fan speed case: (a) experimental result, (b) numerical results, (c) experimental result, and (d) numerical results

Grahic Jump Location
Fig. 14

Cabinet leakage: (a) leakage between heat exchanger box and rear door, (b) between servers leakage, and (c) cable cutout leakage on the top of the cabinet

Grahic Jump Location
Fig. 15

Air stream temperature at the front side of cabinet: (a) with leakage and (b) without leakage

Grahic Jump Location
Fig. 16

Average inlet temperature of the IT equipment for different conditions

Grahic Jump Location
Fig. 17

Transient temperature variation for 1 RU server used for semi-empirical thermal mass assessment

Grahic Jump Location
Fig. 18

Heat exchanger supply temperatures (17 lines on top), return air, and supply water temperatures-water pump failure scenario

Grahic Jump Location
Fig. 19

Effectiveness, water mass flow rate, and server's inlet temperature-water pump failure scenario

Grahic Jump Location
Fig. 20

Inlet/Outlet air and water temperatures-blower failure scenario

Grahic Jump Location
Fig. 21

Effectiveness, water mass flow rate, and server's inlet temperature-blower failure scenario

Tables

Errata

Discussions

Some tools below are only available to our subscribers or users with an online account.

Related Content

Customize your page view by dragging and repositioning the boxes below.

Related Journal Articles
Related eBook Content
Topic Collections

Sorry! You do not have access to this content. For assistance or to subscribe, please contact us:

  • TELEPHONE: 1-800-843-2763 (Toll-free in the USA)
  • EMAIL: asmedigitalcollection@asme.org
Sign In