characterization of a high-density data center.
The density data center focuses on azone with heat dissipation greater than 8 KW and up to 26 KW per frame.
To gain insight into operational health, we conducted a data center survey.
The purpose of the survey is to measure and collect data on power consumption, airflow and temperature throughout the data center.
These results are used to calculate fluid dynamics (CFD)
Model for detailed analysis.
The thematic studies used in this study are similar to those of the National Center for Environmental forecasting (NCEP)
Case study of data center (Schmidt2004).
Plan density installation based on published best practices, consultant advice, guidance from industry associations and organizations, and advice from equipment manufacturers.
Due to the choice in the market, each data center is different from the air center of the computer roomconditioning (CRAC)
Selection and placement of raised units
Floor height and perforated panel position.
Although publications such as ASHRAE\'s hot guidelines for data processing environments (ASHRAE 2004)
Provides a general set of guidelines for device placement and layout configuration, there is no cooking method to plan high
Airflow in ahleigh
Density data centers are complex and require engineering analysis of the environment.
However, the ultimate goal is to provide information technology (IT)
Devices operating in the data center have the right temperature and humidity at the air inlet.
Power Trends and cooling applications for Datacom devices (ASHRAE2005)
Provide Power trends at the device level and tell how to use them when making infrastructure decisions.
Many leading data equipment manufacturers offer devices that track trends in thermal load density.
With the increased processing and storage requirements, the growth of applications, and the reduction of floor space due to technical compaction, air cooling becomes more and more challenging.
Questions about when liquid cooling is needed and what methods should be used (
Area, frame, rack-
Install drawer or chiplevel)
Not easy to answer.
Existing conditions in the data center must be studied and combined with IT procurement and planning strategies to try and predict the future.
High-existing thermal load, airflow and temperature conditions
This paper introduces the density data center and analysis of the research in November 2005.
Introduction to the data center is the San Diego Supercomputer (SDSC)
University of California, San Diego.
The area of the data center based on the drawings is 88x140 (26. 8 x 42. 7 m).
However, the area of CRAC equipment, distribution panel and IT equipment is 11, 490 [ft. sup. 2](1067 [m. sup. 2]).
Figure 1 shows an overall plan view of the area considering further investigation.
The ITequipment installed in the overview area is 7039-
Server model 651, IBM server pSeries 651 and 690-7045 IBM
SW4 IBM eServerpSeries high performance switch (HPS)Model SW4.
Data center board-to-
16 feet high board (4. 9 m)
The decomposition is as follows: 2 ft (0. 6m)
10 ft (concrete ground floor of elevated floor)3. 0 m)
Ceiling and 4 feet (1. 2 m)
Above the ceiling.
As shown in Figure 2, remove the selected ceiling panel in the hot channel to facilitate it equipment to discharge air to the CRAC unit through the extension of the return pipe.
The CRAC unit also installed steering blades. [
Figure 1 slightly][
The entire data center uses a pseudo-arrangement of the hot Channel/cold channel, with different channel spacing (
Distance from onecold aisle center to next cold aisle Center)
, Raised floor panels from 7 to 10.
2x2 ft for all panels (0. 6 x 0. 6 m).
According to the diameter of each hole and the pattern of the hole, the perforated raised floor panel is estimated to have an opening of 22%.
There are a few blocks from under the insulated frozen floor-
Water supply and return pipes in cable bundles and cable trays (
See Figure 3 and Figure 4).
Use the AlnorBalometer capture cover to measure the airflow of the measuring tool through the perforated panel.
Because the capture cover hinders the airflow of the perforated panel, the reading must be adjusted by a correction factor.
However, the Alnor capture cover has a built-in
In the reverse pressure compensation function, the flow impedance of the capture cover is explained by flaps.
This function is used for perforated panels for each measurement.
The measurement accuracy based on the manufacturer\'s specification sheet is [+ or -]
3% of the airflow reading.
Cable cutting is measured with an Extech weathervane with a speedometer.
The measurement accuracy based on the manufacturer\'s data sheet is [+ or -]
2% of speed reading
Temperature measurement of air inlet of IT equipment is carried out with Omega model hh Type 23 digital thermometerT thermocouple.
The measurement accuracy based on the manufacturer\'s product specifications is [+ or -]0.
1% of the temperature reading.
All test equipment referenced is in accordance with ISO/IEC standard 17025 (ISO/IEC 2005).
All voltage and current measurements are provided by the staff of the atSDSC.
Using aFluke T5-measure the AMP data of the circuit provided by the SDSC
600 electrical tester and fluke 33 fitter table.
The instrument was not calibrated, but input power was checked for several frames.
Alaptop with custom communication software is connected to the interface on the frame with a percentage difference of less than 5%. [
Figure 3 slightly][
Figure 4 slightly]
Measurement methods and results power measurements with the help of SDSCpersonnel collect the heat load in the data center on site, including IT equipment, cracunit and lighting.
IT devices in the data center are provided by 208 v ac, either three-phase or line-to-
Line and 120 v ac line-to-neutral.
TheCRAC unit provides 480 v ac line-to-line.
Table 1 shows the interruption of power consumption.
There seems to be a big difference in the thermal output range of the CRAC unit and further investigation is required.
The heat output of the IT device is calculated by multiplying the number of amps by the associated power supply connection.
The result is voltage. amps (VA)
, But the power factor is 0.
95 is provided by sdsc for the determination of Watt, as most IT devices have compliance with the harmonic emission standard 61000-3-2 (ISO/IEC2005).
However, it is important to note that all IBM p690 and p655three-
Stage server, which is a large load in the data center, does not belong to 61000-3-
2, including active mitigation, the power factor is very close.
Table 2 shows high-Density server.
The numbers in brackets are IBM\'s hardware.
The maximum error estimate of the total heat load is [+ or -]5%.
Through the communication interface on the large-capacity power system, several p655s are measured directly on the rack.
The measurement results are advantageous compared to the data from the power panel, although the power panel is always above and below 5%.
This result is most likely due to the measurement error and distribution loss of the non-calibrated instrument.
Total 12 Liebert deluxe systems/3 freeze systems for airflow measurement and modeling
Faucet installed on elevated floor.
Table 3 provides the manufacturer specifications for each CRAC unit.
Calculation of reasonable cooling according to return air temperature of 73 [degrees]F (23[degrees]C)
Dry ball, relative humidity of return wind 37%, water supply temperature 41 [degrees]F (5[degrees]C)
Adjust the water temperature to the chiller degrees]F (12[degrees]C).
Collect and average return wind data from the Liebert monitoring system, as the temperature and humidity sensors are located in different locations as shown in Table 3.
Due to the extension of the return pipe, the method used to measure the airflow of each CRAC unit in the NCEP data center cannot be used, in our study, there is not enough time to cross the rectangular pipe speed through an equal area or a log-to-cut (Richardson 2001).
However, the results of computational fluid dynamics (CFD)
Simulation tool for air flow and temperature distribution in raised
The floor environment indicates that the actual airflow is within the manufacturer\'s measurement tolerance
The specified airflow.
CFD results will be discussed at the end of this section.
There are a total of 233 perforated panels in the data center.
The total open area is 200 [ft. sup. 2](18. 6 [m. sup. 2])
Openenabled panel based on 22%.
Each perforated panel is measured through the Alnorcapture cover with back pressure compensation and the airflow is recorded.
The total volume airflow of the 233 perforated panel is 99,947 cfm (2830 [m. sup. 3]/min).
The data center also has 306 cutouts for power and communication cable routing.
Measure and adjust the open area of each opening according to the estimated cable blockage.
The total open area is 92 [ft. sup. 2](8. 5 [m. sup. 2])
Due to time constraints, only 17 openings are measured using a wind ball with a weathervane.
Data collected on CRAC units, perforated plates and cuts during the study were used to establish CFD models and compare the results with the measured results.
In addition, cable trays and cable bundles, as well as under the important floor of the freezing equipment, are blocked
Water supply and return lines are added to the model.
In addition, the data center also checked the surrounding opening in the bumpfloor cavity.
No one was mentioned, but no exhaustive review was conducted.
Finally, it is estimated that the leakage area of air distribution between perforated plates is 0.
2 based on the average width of the gap between floor panels.
The typical percentage value of the leak area can be from 0. 1% to 0.
2%, but can be as high as 0. 35% (Radmehr et al. 2005).
The model runs several times with different distributed leak values to arrive at an acceptable comparison between the measured value and the predicted value.
However, since the data center has been around for a while and has a high static pressure area, the leak may be larger than assumed.
Water meter 094 inch (w. g. )(0. 023 kPa).
All in all, there is only one.
The difference in total cfm between the measured and simulated airflow was 7%.
Figure 5 shows the measurement results and modeling results on the aperfored panelby-panel basis.
The average static pressure of the entire elevated floor is 0. 048 in. w. g. (0. 012 kPa).
The initial operation of the model shows some wide range of offsets because the damper is used on some perforated plates.
Since the appropriate airflow rate can be obtained, and the pressure drop usually varies with the square of the airflow rate, the model can be improved.
Enter additional airflow resistance in a polynomial expression for a specific perforated panel and re-run the model.
Air flow of AA row perforated plate, closest to CRAC unitsCCU8-
12, showing the maximum difference between the measured value and the model value, but the perforated plate in the adjacent row shows a good correlation.
The comparison of the cut shows a good correlation between the measured opening and the modeled opening less than 25 [in. . sup. 2](161 [cm. sup. 2]).
The percentage difference is less than 15%.
The larger incision comparison showed that the predicted volume airflow was significantly higher than the measured airflow.
The reason for the difference is likely to be related to the measurement technology.
Regardless of the size of the incision, three measurements are made to obtain the data in the shortest possible time.
Because the area of the weather vane is 5 [in. . sup. 2](32 [cm. sup. 2])
100 [said] it is difficult to obtain a representative linear airflow with three large openingsin. . sup. 2](645 [cm. sup. 2])
Before the integration of this area[
Figure 5 Slightly]
The temperature measurement temperature is recorded at the frame air inlet in the overview area shown in Figure 1.
Capture a temperature reading at a height of 68 for each frame. 9 in. (1750 mm)
And according to ASHRAE on 1963 (2004)
Hot Guide for data processing environment 2 in. (50 mm)
Before the cover.
The air temperature at the outlet of each perforated plate is measured and recorded by the cover.
Due to the extension of the return air pipe, the return air of the CRAC unit cannot be easily measured.
Instead, record the CRACsensor data. HIGH-
Density analysis of total air density
Adjustment capacity 138 W /[ft. sup. 2](1485 W/[m. sup. 2])
The current heat load density is 132 W /[ft. sup. 2](1421 W/[m. sup. 2]).
Although this information is not particularly useful in the overall operational health assessment, it does not show airflow distribution issues, hot spots, etc. , gross W/[ft. sup. 2](W/[m. sup. 2])
It is a common method for real estate operators to calculate the cost of data centers.
Table 4 gives a view of the total airflow in the data center with estimated accuracy.
Table 4 shows that about half of the airflow in the data center comes from the cuts and leaks.
Although there may be some benefits to the cooling of IT equipment, previous research (
Schmidt and Cruz 2002)
Display the cut air is heated by IT exhaust before returning to the IT air intake.
The frame power, airflow rate and inlet temperature of the area outlined in figure 1 are similar to the NCEP data center profile.
The district is divided into three parts for further study.
Figure 6-an environmental feature map is drawn
10, this indicates that the intake temperature is within or below the recommended drying temperature of ASHRAE level 1
68 [temperature range of bulb]degrees]F-77[degrees]F(20[degrees]C-25[degrees]C)
Regardless of rack power consumption and airflow rate through IT devices.
Table 5 shows the correlation between the average perforated panel supply airflow, cut-out airflow, frame airflow, and frame temperature rise for connections and segments.
The average perforated panel supply airflow comes from the panel directly in front of the frame, although the cold channel may be 4 feet (101. 6 mm)wide.
For comparison with the average frame airflow, the average perforated panel airflow is adjusted to frame width 1. 25 panels.
The frame airflow is as follows: 2960 cfm (84 [m. sup. 3]/min)
P655, 1100 cfm (31[m. sup. 3]/min)
, P690, 800 cfm (23 [m. sup. 3]/min)
For these frameworks.
The frame temperature rise is calculated based on the difference between the average temperature of the air leaving a given row of perforated panels and the total return air temperature of the theCRAC sensor near the row.
Table 5 shows that the airflow rate of the perforated panel at the front of the frame is much smaller than that of the frame airflow rate.
Although the air flow is unbalanced, as shown in Figure 6, the intake temperature is within or below Level 110.
If combined with perforated panels and cuts adjacent to the frame to provide airflow, the calculated temperature rise based on the average frame power is shown in Table 6.
The calculated temperature rise in Table 6 is higher than the actual temperature rise in each part of table 5.
Therefore, the conclusion is the same as the NCEP Data Center ---
When local freezing, the frozen air in the data center is transferred from the high density area to the low density area
The density box is much lower than it might be considered sufficient.
The air flow compares the actual air flow and the calculated air flow from the data center. The air flow comparison measures the heat load and temperature difference.
Measurement and modeling based on perforated panel measurement, limited cable cut-out--is185,700 cfm (5258 [m. sup. 3]/min)
The estimated accuracy from various simulation results is 10.
The actual airflow range is 130-204,270 cfm (4733-5784 [m. sup. 3]/min).
The average temperature difference between the temperature measurement of the perforated panel and the CRACsensors is 59 [degrees]F (15[degrees]C)
The estimated error is 10%.
The total thermal load accuracy is estimated to be [+ or -]5%.
The calculated airflow is 177,664 cfm (5031 [m. sup. 3]/min)
Within the scope of 437-207,205 cfm (4345-5867 [m. sup. 3]/min).
There is a good overlap between the actual and calculated airflow range;
Therefore, the comparison of the data is verified.
Summary This paper describes in detail a high
Data Center. On-
Measure and collect on-site measurements of heat load, airflow and temperature to study the data center.
These parameters are used to build CFD models and run simulations to provide detailed parameters such as cut-outs and leaking airflow that cannot be achieved during the study due to time or physical limitations.
Based on the comparison between the perforated panel measurement and the model results, the model was validated because the difference in total airflow percentage was only 2%.
The air flow comparison also confirms that the actual air flow is consistent with the calculated air flow. The high-
The density area of the data center is studied using IT equipment.
The key IT equipment health indicator is the inlet temperature, even if the maximum value of the airflow rate of the perforated panel and the incision is 2-
30 lower than the frame airflow rate.
Although local conditions do not seem sufficient to meet and maintain high-
Traffic throughout the data center can handle the thermal load of the entire data center.
However, with more high
Density equipment is installed, and even with sufficient cooling capacity, there is a risk of a local rise in the inlet temperature.
The conclusions of SDSCand NCEP data centers are similar.
The author wants to thank the doctor.
Dr. Roger Schmidt.
Harman, Dane Miller, and Harlal qitel in Hendry have helped to collect and interpret the data.
Without their contribution, qualitative and thesis is impossible.
The authors also thank the staff of the SDSC, especially Mike Datte and Jeff Filliez, for their full cooperation in allowing IBM to research the data center and publish the results.
ASHRAE reference. 2004.
Hot Guide for data processing environment.
Atlanta: American Society of Heating, Refrigeration and Air
Condition Engineer CompanyASHRAE. 2005.
Power Trends and cooling applications for Datacom devices.
Atlanta: American Society of Heating, Refrigeration and Air
Air conditioning engineer Limited[
Figure 6 slightly][
Figure 7 Slightly][
Figure 9 omitted[
Figure 10 slightly]IEC. 2005.
Emc (EMC), Part 3-2:Limits--
Emission limit of harmonic current (
Input current of equipment]left arrow]16 A per phase).
GENEVA, Switzerland: International Committee of electricians. ISO/IEC. 2005.
General requirements for testing and calibration laboratory capabilities.
GENEVA, Switzerland: International Organization for standardizing. Radmehr, A. , R. Schmidt, K. Karki, and S. Patankar. 2005.
Distributed leak stream in bump
Floor data center.
IPACK 2005 Progress in Electronic Packaging of Proceedings 2005
73273, asme/Pacific Rim Technical Conference and Exhibition on the integration and packaging of micro-electro-mechanical systems, NEMS and electrical subsystems, page401-08. Richardson, G. 2001.
Accurately traverse in a rectangular pipe.
Related air balance board TAB Magazine, issue 2001 of summer, p. 20-27. Schmidt, R. 2004.
Methods for thermal properties of data centers.
ASHRAETransactions 110 (2):635-42. Schmidt, R. , and E. Cruz. 2002. Raised-
Floor computer data center: the effect of exit heat island and coolaisle on the inlet temperature of the rack.
Thermal Mechanical phenomena in electronic systems, minutes of the international community conference, P. 2002580-94. Joseph F.
Priesko, Joseph F.
Prisco is a senior engineer at IBM, Rochester, MN.