Data Center System Design
Reference: Publications by Ir. Pua Ching Tian/ ASHRAE / Trane
Written and compiled by :
Ir. Tan Yoke Lee
MIEM, Mi-FIRE E
[email protected] Data Center System Design
There are many cooling design schemes and systems available for data center.
The actual selection would be dependent on the followings:
1. Heat output of server racks. Racks can consume from 5kW per rack up to
about 50kW to 100kW depending on the CPU and GPU installed; with AI racks
consuming the most power, and thus generating the most heat. Air cooling
may be sufficient for low heat rejection but when AI racks are installed, air
cooling may not longer be sufficient. The chips may require liquid cooling
while the remaining 30% of heat load should be rejected by air cooling.
2. The design supply air temperature depending on the class of server
equipment in accordance to ASHRAE from class A1 to A4 and also H! and H2.
The recommended temperature is about 18-22deg C supply air temperature.
3. Air cooling may use air flow management such as hot and cold aisle. Examples
of equipment are roof top DX units, CRAC units, CRAH units, fan wall units,
rear door cooling, etc.
4. While water cooling will have the facility water system, technology cooling
system and cooling distribution unit. An approach of about 4deg C is expected
between TCS and FWS at CDU.
Data Center System Design
There are many cooling design schemes and systems available for data center.
The actual selection would be dependent on the followings:
6. Servers using liquid cooling can be classified as class W17, W27, W32, W40, W45 and W+. The numeral defines the
facility water supply temperature. The type of cooling equipment will be dependent on the Wclass.
Data Center System Design
There are many cooling design schemes and systems available for data center.
The actual selection would be dependent on the followings:
7. Thus, for higher class W32 and above, evaporative cooling can be used and normally closed loop cooling tower is
preferable to maintain the water quality running through the CDS.
8. For lower class, chiller shall be used to produce chill water of temperature about 17deg C and above. When ambient
temperature is lower at certain months, water side economizer with cooling tower can be used instead of chiller.
9. Air cooling equipment may reject heat via DX unit, air cooled chiller or water cooled chiller. Economizer mode can be
used during cool months when air temperature is less than 18deg C. However, when using economizer mode, the
amount of outdoor air brought in need to adjust to heat removal requirement of equipment and should be balanced by
exhaust air to create only slightly positive pressurisation.
Data Center System Design
Not using chiller and combining
liquid cooling and air cooling.
• Liquid cooling is achieved by
cooling tower where supply cold
water temperature to CDS is
32deg C and return is about
37deg C.
• Air cooling is used DX type fan
wall unit and DX crac unit. If fan
wall unit is using chilled water, a
chiller compressor is required.
• Note that the air flow
management could be
improved using hot and cold
aisle. The air returning to the DX
unit or fan wall unit shall be
from hot aisle and can be up to
37deg C or more depending on
the class of server equipment.
• The supply air temperature
from CRAC DX or Fan wall unit
can be about 22deg C.
Data Center System Design
Not using chiller and combining
liquid cooling and air cooling.
• Note that primary cold water
circuit is in looped to improve
redundancy.
• CDU is a heat exchanger
between primary and
technology cooling water
system. There is a secondary
VSD pump inside CDU which
supplies water at temperature
of 36 deg C to data center rack.
Data Center System Design
The supply chilled water to rear door
cooling shall be 18deg C and return
Combing rear door cooling with liquid cooling such as direct to chip. temperature shall be the temperature
class of the direct to chip cooling
equipment say W27 or W32.
18 deg C
We can pipe the colder chilled water
to the colder equipment such as rear
door cooling and use the leaving water
from rear door cooling as water
32 deg C
entering the direct to chip cooling
which requires warmer water.
The return water temperature to
37 deg C chiller can be between 32 to 37deg C.
The configuration shows is a primary
secondary system.
32deg C
Images credit to trane
Data Center System Design
• Fan Wall unit layout Fan wall unit layout.
• The server rack is blue in color.
• The red layout is the hot aisle while the
remaining of the area in data hall is cold aisle.
• Fan wall unit provides cold air about 18-22 deg
C to the cold aisle and can be DX or chilled
water type.
• The cold air is then sucked in by the server fan
through the front of server and is discharged to
the back of server.
• The hot air discharged through the back of
server is contained in hot aisle and returned to
the back of fan wall unit.
• The fan section is facing the front of the hall
while the filter section is located at the back.
Access to the filter and maybe fan can be made
from the perimeter space without having to
access the server hall. This will make the
service easier with higher security.
Data Center System Design
• Fan Wall unit layout Fan wall unit layout.
• The server rack is blue in color.
• The red layout is the hot aisle while the
remaining of the area in data hall is cold aisle.
• Fan wall unit provides cold air about 18-22 deg
C to the cold aisle.
• The cold air is then sucked in by the server fan
through the front of server and is discharged to
the back of server.
• The hot air discharged through the back of
server is contained in hot aisle and returned to
the back of fan wall unit.
• The fan section is facing the front of the hall
while the filter section is located at the back.
Access to the filter and maybe fan can be made
from the perimeter space without having to
access the server hall. This will make the
service easier with higher security.
Section view
Data Center System Design
• Fan Wall unit layout Fan wall unit layout.
• The server rack is blue in color.
• The red layout is the hot aisle while the
remaining of the area in data hall is cold
aisle.
• Fan wall unit provides cold air about 18-
22 deg C to the cold aisle.
• The cold air is then sucked in by the
server fan through the front of server and
is discharged to the back of server.
• The hot air discharged through the back
of server is contained in hot aisle and
returned to the back of fan wall unit.
• The fan section is facing the front of the
hall while the filter section is located at
the back. Access to the filter and maybe
fan can be made from the perimeter
space without having to access the server
hall. This will make the service easier
Section view with higher security.
Data Center System Design
• Fan Wall unit layout
Data Center System Design
• Using CRAH unit or fan wall unit, chiller compressor and buffer tank system.
Bypass line
Secondary loop
35 deg C CWR
Bypass line
Data Center System Design
• Utilising Air Cooling Only Using CRAH unit or fan wall unit
• Using chiller compressor and buffer tank system.
• There is possibility of using both Chilled water CRAH and DX CRAH unit to ensure that the failure of chilled water
system does not render the data hall without any cooling.
• Buffer tank can be used as standby when chiller cooling is not available during rotation and start up.
• The condenser loop uses open loop cooling tower to feed chiller condenser via condenser pump. Closed looped
cooling tower is not required as the condenser water is not going into CDU. Condenser water supply to chiller is 30deg
C and return can be 35deg C.
• Arrangement is one chiller to one chilled water pump, one condenser water pump and one cooling tower going into
one DDC panel for chiller management.
• The chiller shall be N+1 or 2N depending on whether designing for tier III or tier IV.
• Chilled water is leaving chiller at 18deg C and returning from data hall at 27deg C. Thus chiller can operate at greater
efficiency since the lift is only slight more than 35-18= 17deg C as compared to conventional chill water system.
• Chill water from chiller enters the primary loop at 18deg C and is fed to secondary loop via vsd secondary pump. The
loop system also works like decoupler pipe for primary/ secondary.
• Buffer tank is connected to secondary loop. During charging mode only, the bypass line is opened such that the water
exiting the buffer tank goes to the return line. Buffer tank acts as thermal energy storage for standby purpose when
chiller is not operating properly. Size buffer tank to 4 litres per kW cooling capacity.
• When not charging the bypass line is closed.
• Note that the water cooled chillers can also be replaced by air cooled chiller.
Data Center System Design
Adding water side economizer
When deploying chiller and
cooling tower in mild temperature,
the plausible operating hours of
water side economizer should be
considered.
When favourable, adding
economizer to transfer heat from
chill water return to condenser
water directly should be
considered.
Water side economizer is piped
parallel to chiller and during cool
weather, the chiller is switched off.
Only cooling tower is operating.
Image credit to MEP Academy
Data Center System Design
Roof top units can be DX
or chilled water type.
Where the ambient
temperature has high wet
bulb depression, we can
use that to cool the return
air from hot aisle which
can be as hot as 37deg C
via indirect evaporative
cooling. However, water
supply shall be available.
And downstream from the
evaporative cooling, we
can install DX or chilled
water coil to remove the
remaining of heat to
supply chill water at
temperature of 18deg C.
Image credit to MEP academy
Data Center System Design
In conclusion, the system to be implemented can be quite innovative for energy savings considering the followings:-
1. Air cooling or liquid cooling depending on server racks requirement.
2. The class of air cooled and water cooled servers. When using air cooling, air flow management should be
implemented.
3. Elevating the supply chilled water and cooling water temperature will decrease the compressor lift or enable using
water side and air side economizer.
4. Redundancy shall be considered for all equipment and control with N+! For tier III and 2N for tier IV.
5. Buffer tank shall be provided for chilled water system during the switching and rotations of chillers.
6. Evaporative cooling can be used where the wet bulb depression is high and the facility water temperature requirement
is very high.
