01 Data Centre Fundamentals (1605 - Recert)
01 Data Centre Fundamentals (1605 - Recert)
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
There are many descriptions used - the terminology has not been agreed upon by the
________________________________
industry, and tends to be company-specific. For the purpose of this course, we will
________________________________
refer to these rooms as ‘Data Centres’.
When defining a Data Centre, is it down to size, what it is used for or what critical
________________________________
services it supports? Think about how you would answer this question in regard to
your own working environment.
________________________________
When you view it in this context, you can see there is no such thing as a generic Data
Centre and no single rulebook for managing one.
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
The term ‘Data Centre’ is used by many people to mean many things.
________________________________
► There is no fixed size or complexity for a Data Centre, it can range from a single
________________________________
equipment room up to a large complex with several computer rooms, each one
containing many racks of equipment.
________________________________
► TIA 942 defines it as shown above, but it is still open to interpretation.
________________________________
► TIA 942 sets out good principles for Data Centres that apply across the range. It
is not definitive for every instance, and there are often variations that can be
________________________________
made.
________________________________
► TIA 942 can be used as the basis for a Data Centre design, providing a minimum
standard that should be expected.
________________________________
► The EU Code of Conduct sets out good practices for energy efficient design and
operation in Data Centres of all shapes and sizes.
________________________________
The key thing is that it is fit for purpose and meets the business strategy.
________________________________
________________________________
________________________________
________________________________
It can be more useful to think in terms of function, because all modern Data Centres
________________________________
have similar requirements, both structurally and in the provisioning of services.
In________________________________
order to successfully provide the planned services, it is important that continual
functioning of the Data Centre IT infrastructure is achieved, with the operational
________________________________
support of the power and cooling infrastructures.
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
Data Centre operation involves high levels of engineering for a technologically complex
________________________________
computer facility. It is therefore important to understand that the management
________________________________
process differs significantly in its approach when compared to more traditional
commercial premises.
________________________________
Any organisation that tries to design and operate a Data Centre in the same way as
they would an office is doomed to fail to meet its expectations.
________________________________
The three main categories of data centres are shown above. The management
process and role of the Data Centre Technician may vary significantly depending on the
________________________________
type of Data Centre, but the principles applied are essentially the same.
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
Enterprise Data Centres tend to house all or the majority of the company’s data
processing resources.
________________________________
Many organisations, particularly larger ones, have chosen to provide and operate their
own Data Centres stating a number of reasons, including:
►________________________________
Security and control of the data, commercial sensitivities, etc.
►________________________________
Reliability and availability of services
► Ability to respond to changing requirements
________________________________
► Response time to a call for the data
►________________________________
Better integration into corporate processes
Some organisations fail to gain the full potential of the Data Centre because their
________________________________
culture prevents them from doing so.
________________________________
Operators can fall into comfort zones, or self contained and independent teams,
interacting minimally with other departments. There is no holistic direction, often
________________________________
resulting in slow processes and responses.
________________________________
________________________________
________________________________
________________________________
Managed hosting facilities are becoming more attractive to some businesses because
they can rent IT infrastructure instead of investing in it themselves. Servers, networks,
________________________________
applications, bandwidth and other equipment are managed by employees of the facility.
Some businesses, especially Small to Medium Enterprises (SMEs), view the need for
________________________________
data storage, servers and high capacity network connectivity as not something they
either want to, or are able to, manage themselves.
________________________________
Businesses who need constant network connectivity and redundant power may not
________________________________
have the capital to invest in the infrastructure. Many SMEs do not want to invest in
Data Centre infrastructure and staffing, but they still need a reliable network for
________________________________
internal applications and email.
A SME can rent a few servers on a monthly basis, adding servers and storage
________________________________
continually as they grow. They can also increase temporarily during busy periods of the
year. SMEs say they like this approach as they can concentrate on their core business.
________________________________
There are also reported cost advantages with managed hosting.
________________________________
Frost & Sullivan report that US companies who have subscribed to hosted IT
infrastructure services have seen dramatic savings, up to 75%, compared to owning and
________________________________
operating their own facility.
________________________________
________________________________
The Colocation (Co-lo, also called an Internet Data Centre / IDC) operator provides
________________________________
the environment and the IT, and telecommunications equipment is provided by the
customer.
________________________________
Service levels vary but typically, the customer can have customisable units with
separate shipping and receiving areas, dedicated entrance doors, branded lobbies and
________________________________
personnel. Customers can often choose their own space size, infrastructure and
operational options, for example:
________________________________
► Raised floors
►________________________________
Chilled water or air-side cooling
________________________________
► Container-based or rack-based compute area
► Power density
►________________________________
Energy efficiency
________________________________
The type of customers that use hosting facilities has evolved in recent years. A
growing number of large enterprises and even some financial institutions run their
________________________________
Data Centres at outsourced facilities.
One large US company said "We could justify building our own Data Centre but aside
________________________________
from the investment, we would have a number of service level agreements to adhere
to and the additional expertise required would be a significant cost. Our domain is not
________________________________
operating a Data Centre."
________________________________
Let’s remind ourselves of what Data Centres are trying to achieve. The business
________________________________
strategy is the fundamental driver which dictates everything below it in the stack, and
________________________________
therefore all of the subsequent strategies and processes in the supporting IT structure.
This top-down approach is how the stack positions things strategically. Operationally,
________________________________
though, we tend to think of it as a bottom-up approach where:
►We start off by sourcing energy from a supply.
________________________________
►This energy is then delivered through the M&E plant to the IT equipment
________________________________
►The IT equipment runs the application software
►The software supports the business activities that the Data Centre is there for.
________________________________
Except sometimes we forget the last bullet point.
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
There is an increasing number of options appearing for businesses to deliver their IT
strategy. The choice of Enterprise, Managed Hosting and Co-Lo solutions is being
________________________________
extended by service providers, whose offsite alternatives include complete solutions
hosted in "the cloud".
________________________________
The choices made by an organisation have major impacts on the operational processes
required. At one end, everything is delivered in-house; at the other end, it is all
________________________________
managed through Service Level Agreements (SLAs).
________________________________
A key question here is whether decisions are based on ensuring strategic IT
requirements or are merely cost based. Either way, it must meet the business needs.
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
Leveraging third-party computing capability over the network is a good way to cut
________________________________
costs, increase scale, and improve agility. The concept of cloud computing involves a
Data Centre somewhere in the world, or even multiple Data Centres scattered around
________________________________
the world. This is a paradigm shift from the historical client-server architecture, where
the network users owned, maintained, and operated their own network infrastructure,
________________________________
server rooms, data servers and applications.
________________________________
Typical cloud computing providers deliver common business applications online that
are accessed from web browsers, while the software and data are stored on the
________________________________
servers or SAN devices. These applications are broadly divided into the following
categories:
________________________________
Software as a Service (SaaS)
________________________________
Utility (Service) Computing
Web Services
________________________________
Platform as a Service (PaaS)
________________________________
Managed Service Providers (MSP)
Service Commerce
________________________________
Internet Integration
________________________________
These Data Centres are hosting the servers and applications the clients use to operate
their businesses.
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
Heating,Ventilation and Air Conditioning (HVAC) systems
AC and distribution, central heating and cooling, water cooling delivery elements,
________________________________
pressure management systems, and variable frequency drives
________________________________
Building management systems
Controls interfacing with HVAC, electrical, fire alarm and security systems
________________________________
Primary and standby electrical systems
________________________________
Power distribution systems, lighting controls, automatic transfer switches,
uninterruptible power supply systems, and generators
________________________________
Voice and data distribution systems
Cabling, telephony systems, and networking equipment
________________________________
Fire detection and suppression systems
________________________________
Fire detection equipment and alarms, notification systems, wet or dry sprinkler
systems, gaseous fire suppression systems, and the interface between detection
________________________________
and suppression components
________________________________
Plumbing systems
Hot and cold water, sanitary waste, and storm drainage systems
________________________________
Speciality systems
________________________________
Elevators and escalators
Building elements
Building envelope, exterior curtain walls, and roofing structure
Over the last fifteen years, the industry has quantified the importance of devoting considerable
________________________________
attention to a Facility Operations Plan.
The design and structure of the department that operates the electrical, cooling, and fire
________________________________
detection / suppression systems in the Data Centre is the first step. For example, a minimum of
two trained individuals per shift on a continuous shift schedule are required.
Annual objectives for this group should include collective goals for consistent facilities systems
________________________________
uptime and successful/safe completion of all assigned preventive maintenance (PM) tasks and
customer requests.
________________________________
The next objective should be defining site-specific procedures and training programs with
detailed schedules customized to the unique systems configuration at each site.
________________________________
Development of emergency response and system transfer procedures is required. Procedures
should be “tested” individually with Facilities staff members for clarity before they are finalized.
________________________________
A clear, concise and consistent procedure format should be employed - one which includes a
means to “check off” each step as it is completed.
________________________________
Successful Data Centre Facilities teams adhere to a rigid schedule for planned maintenance
tasks. Your Facilities group should follow a work schedule generated by an automated program,
which has been customized by your group for your specific site.
________________________________
Monthly site-specific training sessions should be developed by your Facilities team with a focus
on which emergencies (typically system failures), they wish to be most prepared for. This
________________________________
program will provide an annual chance for your Facilities team to simulate the desired response
when an emergency occurs.
________________________________
Every Data Centre needs to define its policies, procedures, and operational processes relating
to change control, standard operating procedures, emergency operating procedures, security
________________________________
and access control, crisis management, etc.
Customer and supplier contract details Operational Level Agreements (OLAs) and SLAs
________________________________
Fundamentally eliminate the “silos”. Recognize the mutual support and dependence of IT and
Facilities on each other. Create a Multi-disciplinary team (IT and Facilities) with combined goals
to drive collaboration and planning (sink or swim together).
________________________________
A Data Centre facility will operate successfully if the Facilities team is provided
management support, appropriate resources and site-specific systems experience.
________________________________
Effectively deploying a Facility Operations strategy, and the additional control
processes, will provide for a much higher reliability potential over the life of the facility.
________________________________
With these practices in place, you may realistically achieve multiple years of continuous
________________________________
facilities systems availability – significant savings when compared to the average
operating experience in the critical Data Centre industry.
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
Standards identify various spaces within a model Data Centre, however this is not
________________________________
100% applicable to all situations. The structure of a building has a major impact on the
________________________________
layout of the Data Centre, not all are greenfield sites.
The model is useful to define spaces, identify relationships, and also to set guidelines on
________________________________
which spaces are normal workplaces, and which are equipment areas.
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
. ________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
No other service should cross the computer room space
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
The Data Centre designer considered the needs of the business and put together a
________________________________
design philosophy to deliver a facility to support the IT platforms required to support
________________________________
the IT strategy, which underpins the business strategy.
The Data Centre Technician must ensure its performance is kept in-line with the
________________________________
design parameters, as well as the changing technologies throughout the lifecycle of the
Data Centre.
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
The business strategy when the design was commissioned is probably not the same as
________________________________
the business strategy throughout the lifetime of the Data Centre.
________________________________
The Data Centre must be able to evolve to meet these business changes.
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
Who does the Data Centre Technician come into contact with?
________________________________
There are many stakeholders involved, with different needs, requirements and opinions.
________________________________
The Data Centre Technician may come into contact with some or all of these groups,
and needs to understand their place in the overall scheme of the Data Centre.
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
Standards
Recommendations, a guide to a minimum requirement. Use is voluntary and
________________________________
available to the public.
________________________________
Established by consensus of all parties concerned
Based on consolidated results of science, technology and experience
________________________________
Approved and published by recognized standardization body
________________________________
Regulations
Legislation
________________________________
Use is mandatory and available to the public
________________________________
Developed by an authority under public observation
Provide technical specifications either directly or by reference, (e.g. to Standards)
________________________________
Adopted by an authority
________________________________
Code (USA)
A systematic statement of laws or regulations
________________________________
A system of principles or rules given statutory force
________________________________
In Europe a Code relates to Best Practices
________________________________
________________________________
All the above standards will have relationships with numerous other standards that
cover specific aspects, and will be cross-referenced.
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
This is an example of the Standards to show the terminology can be different between
Standards in different parts of the world. It is important the Data Centre Technician is
________________________________
familiar with the ones he/she needs for his/her country.
The Data Centre cabling infrastructure recommendations from various Standards
________________________________
follows the traditional hierarchical star topology, with the usual distance limitations
placed on the different subsystems.
________________________________
It is useful to look at the Standards and read the descriptions of the different areas.
________________________________
Standards can be overlaid and fortunately, they fall into three broad categories as
shown:
►________________________________
Components, assembly, application and performance
________________________________
► Implementation
► Validation
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
Since other industries may be involved in the Data Centre design, the Data Centre Technician should also be
knowledgeable about the related Standards, Local Regulations and National Regulations appropriate to the
________________________________
region.
Worldwide
________________________________
► IEC 60364 defines common rules for electrical installations, IEC 61936 sets out rules for installations >1kV.
90% of the electrical installations in the world are built to IEC 60364 & IEC 61936, or to the US National
Electrical Code. The general aims of electrical Standards are the same, and there has been a considerable
________________________________
amount of effort over many years to bring the different standards closer together.
► American Society of Heating, Refrigerating and Air-Conditioning Engineers (ASHRAE) Standards are also
________________________________
commonly quoted in Europe as a ‘Standard’ for Data Centres.
► Building Codes, Planning Rules, Utility Codes and Telco Codes can also apply to Data Centres, e.g. ITU,
________________________________
Telcordia GRE, etc.
Europe
________________________________
The European Standard EN 50110 “Operation of Electrical Installations” consists of two parts:
► EN 50110-1 contains minimum requirements valid for all CENELEC countries and some additional
________________________________
informative annexes dealing with safe working.
► EN 50110-2 is a set of normative annexes (one per country) which specify either the present safety
________________________________
requirements, or give the national supplements to these minimum requirements.
► BS EN 7671 is the country variant for the UK which ensures compliance with the IEC and EN Standards.
________________________________
The following Building Standards are now being aligned to the Data Centre design criteria: BS 8536: 2010
Facility Management Briefing: Code of Practice.
________________________________
► BREEAM - Building Research Establishment Environmental Assessment Method: Energy efficiency
measures based on environmental impact considerations.
________________________________
► ETSI 300 019 covers the environmental engineering requirements for telecommunications equipment.
► CDM (Construction Design & Management) Regulations 2007
USA
________________________________
► International, National Fire Protection Association (NFPA) Codes and Local Codes can all apply.
________________________________
► The NEC applies in all 50 States, but many cities are adopting (or have already adopted) the IEC Standards,
although currently this is sometimes limited to domestic dwellings. Note that IEC 60364 is a framework,
________________________________
whereas the NEC is a very detailed Standard. US adoption of IEC 60364 will not negate the need for a
detailed Standard like the NEC, rather the NEC will form part of the US framework in much the same way
that BS 7671 does in the UK.
________________________________
► NEC 70E specifically addresses electrical hazards in the workplace and, irrespective of the ongoing debates,
is a good guideline to comply with the requirements of OSHA.
________________________________
► Telcordia General Requirements (GR-xx-CORE) are telecommunications Standards, which broadly
harmonise with ETSI Standards, and are applicable to telecommunications buildings and equipment. GR-63-
________________________________
CORE sets out standards for physical protection of telecoms equipment, but also covers noise and thermal
performance.
________________________________
► The International Trade Union Confederation (ITUC) represents 170 million workers in 157 countries and
territories, and has 312 national affiliates.
________________________________
► The Alliance for Telecommunications Industry Solutions (ATIS) is a Standards development organization
committed to the rapid development of global, market-driven Standards for the information, entertainment
and communications industry. It is a member and major US contributor to the International
________________________________
Telecommunication Union (ITU).
SUMMARY
________________________________
This can be a very complex area but wherever he or she is located, the Data Centre Technician needs to keep
abreast of the latest discussions regarding Codes and Standards, as they can often be wrongly (and expensively)
________________________________
applied.
However, if it is not possible to memorise all of the Standards in detail, the Data Centre Technician should at
________________________________
least REMEMBER 2 THINGS!
1. Local Codes and Standards are the key.
2. When in doubt, CHECK.
________________________________
The Data Centre Technician should remember that a Standard is a guide to a minimum requirement,
not a one size fits all solution.
________________________________
Spaces
TIA-942: 5-key functional areas:
________________________________
(1) Entrance Room (ER)
(2) Main Distribution Area (MDA)
________________________________
(3) Horizontal Distribution Area (HDA)
(4) Zone Distribution Area (ZDA), opt.
________________________________
(5) Equipment Distribution Area (EDA)
________________________________
Ideally separate rooms but not practical for normal organisations; Can be consolidated with defined
areas
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
The interior design of the Data Centre can enhance or impair its performance. This is
not just a cosmetic exercise, and it needs to be maintained.
________________________________
The surfaces should be sealed to avoid dust, but they also reflect light and can make a
big difference to the lighting levels in the room.
________________________________
Floors should also be light in colour and anti-static. Anti-static coating wears off and
________________________________
must be maintained.
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
Walls have a range of requirements that must be adhered to.
minimum requirements:
These are the
________________________________
In a Data Centre, it is important the computer room walls are completely sealed to
avoid leakage, both inwards and outwards. This is for a number of reasons but has
________________________________
been a historical point of weakness.
________________________________
One area that has often overlooked is the correct sealing of firestop penetrations.
The Data Centre Technician should note this can be (and often is) due to poor
________________________________
maintenance procedures, and can have a big impact on the fire containment plan.
Some Data Centres (usually historical ones) have glass in one or more of the
________________________________
computer room walls. This is firstly a security concern but it can also be a source of
________________________________
external heat gain in the room.
In hotter climates, this can place a considerable additional burden on the cooling
________________________________
systems.
________________________________
________________________________
________________________________
________________________________
________________________________
Static, causing electrostatic discharge (ESD), is a known failure mechanism for
electronic equipment, although there is not much data to quantify its effect in Data
________________________________
Centres.The dryer the air, the greater the risk of ESD.
In Europe, the standard for raised floors is EN 12825
________________________________
In the USA, the standard for raised floors is TIA 569B
________________________________
ASHRAE has published many influential documents, such as Thermal Guidelines for
Data Processing Environments, which calls for humidity levels to be kept between
________________________________
40 and 60%: “Low humidity is known to cause problems with static build up…….”
Failure rates will be improved by controlling static problems, which can be
________________________________
summarised as:
________________________________
Use a floor with an antistatic finish (and ask any supplier to define exactly
what that means)
________________________________
Connect every sixth leg of a raised floor system to an earth point
________________________________
Supply your Data Centre with a good quality telecommunications-grade
earth system
________________________________
Control humidity to prevent it from falling below 35% RH
________________________________
________________________________
________________________________
Note:
This is actually very basic and does not account for what may be above or below
________________________________
the Data Centre floor.
Specific standards exist: TIA 569B, EN 12825, MOB PF2 (Method of Building
________________________________
Performance Specification, ‘Platform Floors [Raised Access Floors]’)
________________________________
Most Data Centres exceed these figures.
Data equipment can be very heavy, and rack weights are increasing, with some racks
________________________________
being designed to support up to 1.5 tonnes (1.65 tons). Therefore, another obvious
requirement of the building for the floor to be strong enough.
________________________________
TIA 942: and other Standards recommend floor loading capacities in the different
________________________________
data centre spaces and, in the computer room, the floor should be capable of
withstanding a distributed load of 12 kPa (250 lb/sq.ft).
________________________________
The reality is that most floors are >50kPa (1,000 lb/sq.ft). The Data Centre
Technician must understand and control the load capacity of the floor.
________________________________
Penetrations for cabinet mounting and anchor supports can have an impact on the
floor strength, so this must be considered carefully.
________________________________
Floor loading has a high risk impact and should be discussed fully with the structural
________________________________
engineer.
________________________________
________________________________
In a multi-storey building, the floor slab is actually the ceiling slab of the storey below.
Anything hanging in the storey below the Data Centre will impact the floor loading in
________________________________
the Data Centre. Conversely anything on the floor above will impact the hanging
capacity of the Data Centre ceiling.
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
Another obvious requirement is that it must be possible to get the equipment in or
out, of the Data Centre. This can be a problem where Data Centre access is via other
________________________________
parts of the building.
As the Data Centre is a secure environment, it should go without saying the doors
________________________________
should be lockable with a suitable grade of lock.
________________________________
If the doors form part of the emergency exit route, then they must comply with local
codes.
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
Fire doors have additional requirements. In the event of fire, the doors must restrict
its spread for a pre-determined time. This will be taken into account by the design of
________________________________
the Data Centre, but must also recognise any local regulations and codes of practice.
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
It is important that the Data Centre is suitably lit both when it is in unmanned (for
CCTV cameras, for example) and when operators or maintenance staff are working in
________________________________
the Data Centre. Too often, insufficient attention is paid to the requirements of those
who need to work in the Data Centre, with the assumption being that the IT and
________________________________
support equipment will manage itself, once installed.
The designer and the electrician need to collaborate to ensure that the luminaire
________________________________
positions provide sufficient “wash” lighting, but also sufficient spot or detail lighting to
enable work to be undertaken on the racks without discomfort or difficulty. Proper
________________________________
lighting is also a Health and Safety legislation requirement.
A________________________________
Data Centre manager should consider lighting levels when racks are being relocated,
or when undertaking retrofit work, and should, if necessary, include in the project costs
the________________________________
re-provision or re-positioning of some luminaires.
Another point to consider is that lighting uses power and generates heat. A typical,
________________________________
older lighting system can add up to 3-4% to the electrical load, and generate an extra
amount of heat which the cooling system will need to remove. The energy difference
________________________________
between efficient an inefficient lighting systems and layouts can be used by a Data
Centre manager in managing his or her targets within the organisation’s Energy
________________________________
Efficiency policy.
________________________________
There are several measures of lighting, but, roughly: one foot-candle is equal to one
lumen per square foot, or approximately 10.764 lux
________________________________
________________________________
NFPA 75 is a good example of specific guidance for fire protection in a Data Centre
within the USA. The key is to follow your local fire regulations.
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
During the 1980s, Data Centre operators did not have the resources to help them
________________________________
determine the levels of reliability for their facility. The Site Uptime Network® was
formed in 1993 by The Uptime Institute with a clear purpose. To directly address and
________________________________
resolve issues affecting continuous site Mechanical and Electrical (M&E) infrastructure
availability.
________________________________
Since then, various organisations have developed ways of classifying Data Centres.
The Uptime Institute has a long-standing classification based on four Tiers
________________________________
TIA 942 has their Ratings system
Syska Hennessy have an extended (and it is claimed), more inclusive classification
________________________________
based on ten criticality levels
There are other classifications in use as well, which try to balance criticality vs. cost
________________________________
Whichever classification you choose, when you look at physical systems and
________________________________
infrastructure, there are four common levels of component provision to consider in
practice:
________________________________
► N: Only the number of components needed is provided, there is no resilience and
any component failure will cause downtime.
________________________________
► N+1: Where the number of components provided is one more than the number
(N) of components needed.
►________________________________
2N: Comprises two complete systems, each containing N components, and are run
in parallel to hot swap between each other.
►________________________________
2(N+1): Comprises two N+1 systems in parallel. One side can be taken down for
maintenance, and the other side will be protected from exposure to the risk of
________________________________
downtime by its own redundant module.
When considering the configuration of systems to achieve the desired reliability levels,
it is a good idea to balance redundancy with simplicity.
________________________________
Data Centre operators have developed various power distribution architectures to
ensure power is reliably delivered to their servers (power supplies).
________________________________
Data Centre Tiers originally were defined by the Uptime Institute and have been
________________________________
generally accepted within the industry for a number of years.
The power distribution architectures vary based on the desired “Tier” structure.
________________________________
The reliability of the Tiers and the additional power distribution architectures
establish the overall availability of the Data Centre.
________________________________
Concurrently maintainable means one part of the system can be serviced, replaced, or
________________________________
fail without affecting the normal operation of the remaining part. Many Data Centres
are built to be fault-tolerant, in some systems, but are not certified.
________________________________
Tier Rating Details
________________________________
Tier I: Basic Data Centre that has non-redundant capacity components and a single,
non-redundant distribution path to serve the compute devices.
________________________________
Tier II: This rating of Data Centre has redundant capacity components with a single,
non-redundant distribution path to serve the computers.
________________________________
Tier III: This is a concurrently-maintainable facility, has redundant capacity
components and multiple independent distribution paths serving the computers,
________________________________
although generally only one distribution path is operational at any time.
Tier IV: Fault-tolerant facility with multiple, independent, physically isolated systems
________________________________
that each have redundant capacity components and multiple, independent, diverse,
active distribution paths to the compute devices.
________________________________
Determining the actual Uptime Institute Tier rating of a site for design topology is a
________________________________
logical process, although it is often done incorrectly. The tier topology rating for the
whole site is limited by the Tier rating of the weakest subsystem that impacts on
________________________________
operations. For example:
Site has a robust Tier IV Uninterruptible Power Supply (UPS) configuration
________________________________
Site also has a Tier II configured chilled water system
________________________________
Resulting site rating is Tier II
The use of the word ‘Tier’ in this context is controlled by the Uptime Institute,
________________________________
therefore certification of a facility’s Tier level can only be performed by Uptime
Institute.
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
It is generally considered that both user and application requirements are the main drivers behind
________________________________
the reliability of the Data Centre.
________________________________
Terminology and Definitions
Network-Critical Physical Infrastructure (NCPI) is the foundation upon which
________________________________
Information Technology (IT) and telecommunication networks reside. It is the “backbone” of
the business, as its elements provide the power, cooling, physical housing, security, fire
________________________________
protection, and cabling which allow Information Technology to function.
Reliability: The ability of a system or component to perform its required functions under
________________________________
stated conditions for a specified period of time.
Availability: The degree to which a system or component is operational and accessible when it
________________________________
is required for use. Reliability factors into availability, as does recovery time after a failure
occurs. In a Data Centre, having a reliable system design is the most critical variable.
________________________________
Five 9s of Availability derives from the world of networking, where if the network is
available 99.999% of the time (which translates to all but 5 minutes per year) it is
________________________________
considered highly reliable.
But it doesn’t translate so neatly to a Data Centre. Too often, it is used to refer to the
________________________________
amount of time the Data Centre is powered up. But loss of power is only one part of the
equation when it comes to Data Centre availability.
________________________________
Resiliency: A planned part of a facility's architecture and is usually associated with other
Disaster Recovery (DR) planning and Data Centre DR architectural considerations, such as data
protection. The adjective resilient means "having the ability to spring back”.
________________________________
Fault-tolerant: The attribute of a concurrently maintainable and operable system, or a facility
where redundancy is not lost during failure or maintenance activity.
________________________________
Redundancy: The provision of secondary components that either become instantly operational
or are continuously operational, so that in the event of a primary component failure, the result
will not be mission failure.
________________________________
Availability - Based on the general definition of the availability Tier and the reliability
of the associated equipment, the availability of the system can be calculated, as detailed
________________________________
above.
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
Many of the risk elements addressed in the Tier levels were identified through user
experiences. Often these experiences reflect lessons learned the ‘hard way’, principally
________________________________
through unplanned outages and system downtime.
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
Electrical
Utility Service
________________________________
Lightning Protection
Power Backbone
________________________________
UPS Systems
UPS Batteries
________________________________
Engine Generator
Load Bank
________________________________
Critical Power Distribution
Grounding
________________________________
Mechanical
Raised Floor Cooling
________________________________
UPS Cooling
Mechanical Plant
________________________________
Support Systems
Contamination
________________________________
Fire Detection and Protection
Physical Security
________________________________
Alarms and Monitoring
________________________________
________________________________
________________________________
Conservative over-provisioning of power capacity for availability reasons results in
significant capital investment for large enterprises because this capacity is designed for
________________________________
failure conditions that do not happen often. On the other hand, under-provisioning this
capacity runs the risk of affecting the performance of the Data Centre when failures
________________________________
do happen, through either service unavailability or degraded service performance.
Hence, there are interactions and trade-offs between power capacity utilization, power
________________________________
redundancy, and Data Centre performance that are often overlooked.
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
________________________________
This basic system is reasonably resilient, i.e., it has:
► A main electrical supply from the utility.
________________________________
► A distribution network to split the critical IT load from the non-critical load
________________________________
► A UPS of some description to ensure that the supply to the critical IT load is
maintained.
________________________________
► A standby system, usually a generator, to replace the utility supply in the case of a
________________________________
utility failure.
► Consider the single points of failure that would cause the Data Centre to
________________________________
become unavailable.
► Planned maintenance work will require most or all of the facility infrastructure
________________________________
systems to be shut down, impacting computer operations.
________________________________
The Uptime Institute Tier classification is widely used in the Data Centre community,
often incorrectly, and this configuration would fall into Tier 1.
________________________________
This Uptime Institute Tier classification is still used as a common reference by Data
Centre owners and operators, and can lead to some poor (and costly) decisions if
________________________________
used in isolation.
________________________________
________________________________
________________________________
This system is more resilient, i.e., it has:
► A main electrical supply from the utility
________________________________
► A distribution network to split the critical IT load from the non-critical load.
________________________________
► Duplicated UPS of some description to ensure the supply to the critical IT load
is maintained
________________________________
► Duplicated standby systems, usually a generator, to replace the utility power
________________________________
supply in the case of a utility failure
► Redundant capacity components can be removed from service on a planned basis
________________________________
without causing shut down to the computer devices.
The additional capital cost of the duplicated components is self-evident, but there is
________________________________
also an ongoing running cost penalty, as the systems are running at low utilisation by
design.
________________________________
This would be Uptime Institute Tier ll
________________________________
________________________________
________________________________
________________________________
________________________________
This system has redundant capacity components in the power path to the critical load,
i.e., it has:
________________________________
► Diverse electrical supplies from the utility
________________________________
► Duplicated distribution networks to split the critical IT load from the non-critical
load.
________________________________
► Redundant UPS of some description to ensure the supply to the critical IT load
is maintained
________________________________
► Redundant standby systems, usually a generator, to replace the utility power
________________________________
supply in the case of a utility failure
► Any of the capacity components and elements in the distribution paths can be
________________________________
removed from service on a planned basis without impacting computer devices
The additional capital cost of the duplicated systems is self-evident, but there is also a
________________________________
very significant ongoing running cost penalty, as the systems are running at very low
________________________________
utilisation by design.
This would be Uptime Institute Tier lll
________________________________
________________________________
________________________________
________________________________
This system is fully-duplicated in the power path to the critical IT load, i.e., it has:
► Diverse electrical supplies from the utility
________________________________
► Duplicated distribution networks to split the critical IT load from the non-critical
________________________________
load.
► Duplicated UPS of some description in each network to ensure the supply to the
________________________________
critical IT load is maintained
________________________________
► Duplicated standby systems, usually a generator, in each network to replace the
utility power supply in the case of a utility failure
________________________________
► All of the capacity components and elements in the distribution paths can be
removed from service on a planned basis without impacting computer devices
________________________________
► A single failure of any capacity system, component or distribution element will
not impact the operation of the computer equipment.
________________________________
► All components must be compartmentalised
________________________________
► There is a requirement for continuous cooling
________________________________
The additional capital cost of the duplicated systems is self-evident but there is also a
very significant ongoing running cost penalty as the systems are running at very low
________________________________
utilisation by design.
This would be Uptime Institute Tier 1V.
________________________________
________________________________
The Data Centre designer has implemented a system architecture that will balance
the key constraints and limitations of the Data Centre infrastructure.
________________________________
The Data Centre Technician is faced with ongoing dynamic decisions from his
management to balance the same four key constraints. The alignment of one
________________________________
constraint can have an adverse impact upon another of the constraints if it is not
correctly assessed and balanced across the Data Centre.
________________________________
A report by Gartner Group suggests that >80% of Data Centres never reach >60%
________________________________
of their projected IT design, due to hitting the limit in one or more of the 4
constraints.
________________________________
Additionally, it is necessary to be aware of airflow management. Airflow management
affects cooling capability.
________________________________
The goal of Airflow Management is to minimize recirculation of hot air and minimize
________________________________
by-pass of cold air in the Data Centre.
Successfully implemented, both measures result in energy savings and better thermal
________________________________
conditions, which can increase Data Centre capacity.
________________________________
________________________________
________________________________