Mission Critical
Training
Agenda

SAFETY

Leadership

Communications

“A Day In The Life…”

Vendor Expectations

Safety Hazards

Technical
Generator
Overview

UPS

Batteries

Fire system

Switchgear

Cleaning

HVAC

Site Awareness

Systems Integration
SAFETY
 SAFETY is TRINITY Group Construction's number ONE
priority
 No one outranks SAFETY
 NONE OF US ARE SMART ENOUGH TO MAKE NEW
MISTAKES
 If you see something that is UNSAFE, STOP and address
the situation
Managed Services
Leadership
Leadership
 Ownership
 Peer Leadership
 Characteristics
 Personality
 Integrity
Leadership
Ownership

Taking ownership means being fully accountable to the
customer for the entire scope of a given contract.
 On a personal level, ownership is about YOU taking the
initiative to ensure problems do not remain outstanding.
 YOU take full responsibility for problems at your facility and
pursue them until they have been remedied, rather than
deferring them to someone else.
 This is not to say delegation won’t be necessary. However, if
you delegate to someone or coordinate with someone, you are
still responsible for the outcome.
Ownership





We are responsible for any and all service-related disciplines
associated with the contract under which we operate.
Customers often refer to this as “having one neck to choke,” which
also happens to be the value we provide to them.
Our customers are typically data center managers who oversee
server installs/upgrades/repairs and software issues, and IT
managers who are responsible for network administration,
programming, tech support, etc.
Maintaining the equipment responsible for powering and cooling
their systems is generally not their area of expertise.
Having someone around who is cognizant of these functionalities
allows them to focus on their main duties.
Leadership
Ownership
 You will often be required you to work outside of a normal
routine and “think outside the box.”
 Taking ownership includes spending the time and mobilizing
the proper resources necessary to develop innovative
solutions.
 The over-riding question becomes:
If we aren’t able to keep our customer’s infrastructure
systems reliable, then what are they paying us for?
Leadership
Peer leadership




There will be times when you are required to oversee someone you consider your
peer or possibly an elder.
This person may be a fellow company
employee or a technician from another
company who you may assume is more
experienced or possesses
more knowledge than you.
Your qualification in Mission Critical Services establishes YOU as the supervisor
when such people are onsite with you.
You must, therefore, expect and demand that your relationship with them be
treated as such.
Leadership
Characteristics




Do you possess the proper characteristics to take on the role of a trusted and valued
Leader?
Customers have every right to expect their mission critical provider to measure up to a
certain caliber.
Customers often look to us to manage their facilities when they cannot – due to their
absence or their limitations in certain skills and knowledge – or to serve as a reliable
backup as a means of eliminating a single point of failure.
Desirable characteristics:

Punctual

Consistent

Open

Professional

Team player

Straightforward

Integrity

Confident

Effective

Reliable

Passionate

Lead by example
Leadership
Personality

Your willingness and ability to interact with customers and vendors
is essential to obtaining a successful outcome.
 Maintaining a positive, upbeat personality regardless of the
situation (passive or tense) can make or break a contract.
 The customer needs to know you’re on their side and in full control
of any situation that might present itself. Your demeanor and
general body language are key indicators.
 By demonstrating you are a responsible individual who can remain
“cool under fire,” the customer will come to rely on your outgoing
professionalism and dedication.
Leadership
Personality




If you come across as being untrustworthy or you tend to exhibit other negative
personality traits, the customer may be less communicative with you and develop
trust issues.
ALWAYS be as up front with the customer as
possible.
ALWAYS be proactive in communicating to
them any perceived problems, solutions or
concerns.
The single caveat is that you must always
remember:
You work for our company first and the customer second
Leadership
Integrity

Integrity is often described as “doing the right thing, even
when no one is watching.”
 Can you be counted on to always
do the right thing?




Do I really need to report that alarm?
Does this change in the procedure
REALLY need to have an approved
MOP revision?
Maybe what the customer doesn’t know won’t hurt them.
The points listed above all display signs of flawed integrity.
Leadership
Integrity






What if you compromised your integrity and the issue magnified?
How can you possibly regain your customer’s trust?
How can we possibly retain or regain the contract if the customer
decides he cannot depend on us?
Can we withstand the backlash resulting from a lost customer’s
negative report on us to his industry friends?
Even worse, can a lapse in integrity cost you your job?
We all must feel a personal obligation to ALWAYS perform our
assigned tasks to the very best of our abilities.
QUESTIONS
?
Communication






Customers
Building engineers
Vendors
NOC
Supervisor
Contracts Management Team
Communication
Customers





Your customer is your primary on-site contact with whom you
communicate directly. There may be more than one.
Customers range from experienced electrical engineers to IT
personnel with little knowledge of mission critical infrastructure.
Customer behaviors vary from intense micro-managers to
seemingly disinterested parties.
Know your customer! Know your customer’s expectations! Know
how to deal with your customer!
Make your customer feel comfortable in your presence and
confident in your abilities. Be an effective communicator!
Communication
Building engineers





Many customer sites consist of COLO rooms or leased floor space
in a commercial building.
A building engineer is the person responsible for the building
where the customer site exists.
Communication with building engineers is vital. Many customer
systems are tied into building systems.
The building engineer needs to know if certain systems need to
be de-activated or power transfers are required.
The building engineer should typically be contacted days or
weeks in advance of performing a maintenance service.
Communication
Vendors





Vendors are technicians from companies that have been
contracted to perform certain services on our behalf.
YOU are essentially THEIR customer, and they are required to
perform work in a manner that is satisfactory to YOU.
It is important to develop and maintain good working
relationships with your vendors.
Their knowledge of the equipment they are contracted to service
will likely exceed your knowledge of that equipment.
You will need to rely on vendors to properly describe issues so you
can communicate this information to the customer.
Communication
NOC/Supervisor






The National Operations Center (NOC) serves as your off-site resource while you are
performing customer services.
The NOC is responsible for scheduling your service dates and for scheduling any
required vendor services.
The NOC is also responsible for coordinating any required follow-up corrective
actions or escalations.
Contact with vendor offices or supervisory personnel should also be routed through
the NOC for documentation purposes and to keep the thorough flow of information
intact.
The NOC frees you from the distractions of dealing with a myriad of phone calls and
emails while performing services.
Your supervisor must be informed of any issues that are considered worthy of being
reported, recorded or escalated.
Communication
Project Management Team



The PM is more familiar with their site than NOC personnel, and has a
better working relationship with the customer.
The PM can be a good source
of information regarding site
history and recurring problems.
When communicating with the
PM, realize that most of their
customer interaction is done
from the office. You may need
to help them understand what is happening in the field and how
they might best approach situations with the customer.
Communication
Project Management Team

The Project Management Team
responsible for customer
involving:
is
communications
 Contractual questions
 Pricing out-of-scope work
 Determining which vendors must be
onsite to support a scheduled service

Contact the assigned Project Manager (PM) if you have any questions or
concerns, or if you require service assistance. Remember to keep the NOC
involved in the conversation.
QUESTIONS
?
“A Day in the Life…”
“A Day in the Life…”
Prior to the scheduled service date




Review the associated service ticket-request for the assignment.
 This is the best time to formulate questions about the assignment. Is the service
request clear about your assignment, or is something vague? “I’ll figure it out
when I get there” is NOT is a very good means of setting yourself up for success.
Obtain an approved MOP/SMOP for the assignment.
 This may involve you creating the MOP (once you
have been qualified to do so) and having it
reviewed for approval.
Acknowledge the assignment.
Call each assigned vendor to discuss/verify:
 The scope of the maintenance (quarterly, monthly,
semiannual, annual, corrective, troubleshooting,
equipment involved, how they plan to proceed, etc.).
 The date and time they are scheduled to be onsite.
 Any details that might require clarification (PPE requirements, etc.).
“A Day in the Life…”
Prior to the scheduled service date (continued)

Check with the customer to ensure site access is granted
for you and all of your assigned vendors within the
designated time frame.
 Verify you have a Field Service Report (FSR) ready for
recording the details of your maintenance visit.
 Review previous documentation for incomplete items or
abnormalities.
“A Day in the Life…”
Preliminaries on the date of service (continued)


Ensure the associated MOPs/SMOPs are approved and signed by the
customer and all vendors involved.
Call the NOC to confirm the start of each service ticket.
 Inform them of any alarms you expect to receive. This will prevent them from
responding to anticipated alarms when they should be acting on unanticipated
alarms.



Acknowledge this conversation (including the name of the person
contacted at NOC) on the MOP/SMOP.
Verify fire system disabling was performed.
Verify appropriate customer personnel were informed about the reduced
firefighting capacity during the upcoming services.
 Depending on the customer’s system configuration, this may require having
someone tour periodically with a fire extinguisher while the system is disabled.
“A Day in the Life…”
Preliminaries on the date of service (continued)

Ensure all on-site monitoring capabilities have either been turned off or
appropriate monitoring personnel have been informed about the types of
alarms that might be generated during the course of the upcoming
service.
 Many customers have their own NOCs or remote monitoring systems that
generate alarms.


Ensure all attending vendors have properly
checked in with the customer and are compliant
with any security and access requirements.
Make sure all attending vendors look and act
professionally and appear fit and ready for work.
 You are responsible for your vendors’ performance and behavior!
“A Day in the Life…”
Pre-service meeting with vendors


Ensure all personnel involved in performing a MOP/SMOP have read and
agreed to the Critical Facility Work Rules.
Conduct an overview of the service:
 Responsibilities and limitations. Who is required to perform which tasks?
Capability for performing those tasks. Are any specific qualifications involved?
 Failures that might occur and possible back-out procedures (which should be
detailed in the MOP).
 Safety precautions. Ensure vendors and anyone within the restricted boundaries
of the work area is wearing the proper PPE.
 Actions that should be taken in the event of an unexpected occurrence.
 Hazardous materials in the vicinity (attach SDS for each hazard).
 Energy sources associated with the equipment (electrical, mechanical, etc.).
Specifically discuss any required lockouts/tagouts. Verify the vendor has enough
locks/tags in their possession to accommodate these requirements.
“A Day in the Life…”
Pre-service meeting with vendors (continued)


Walk-through of the entire procedure.
Simulate operator actions.
 Have operators point to appropriate items.


Specify what operators should observe
following their actions.
Mandate that switches and breakers
cannot be opened or closed unless
clearly specified by a designated person
and witnessed by the MOP/SMOP
supervisor.
“A Day in the Life…”
Conducting maintenance







Ensure all involved personnel are aware of and abide by all relevant safety policies.
Ensure all involved personnel possess a solid understanding of how the equipment works and are
given ready access to appropriate documentation.
Ensure all steps are performed as specified in the approved MOP/SMOP. Any deviation from what
is documented in the latest MOP/SMOP requires review, approval and acceptance by the
customer and Quality Assurance!
Following completion, re-attach all enclosure panels and covers after ensuring tools and debris
have been removed from inside the equipment.
The equipment should be left in the same condition as when it was opened, or better!
Remove all tools, debris and loose parts from the surrounding work area. The entire work area
must be cleaned up and left in a neat and orderly condition.
You and the vendor must complete all of the appropriate service reports and obtain the necessary
signatures.
“A Day in the Life…”
Completing the service





Return all fire systems and monitoring systems to operational status.
Ensure all work-generated alarms have are
cleared.
Inform all previously notified groups (local
monitoring company, NOC, building
management, etc.) that all services
have been completed and all alarms
are again active and online.
Ensure facility fire systems are restored to their full
capacity. Communicate this information t the proper
parties.
Conduct a walk-through of the service area to ensure
there are no longer any issues or alarms. Report any abnormalities.
“A Day in the Life…”
Completing the service (continued)







Ensure all vendors properly check themselves out before departing the site.
Ensure all required paperwork has been completed and is acceptable.
Thoroughly review your vendor’s paperwork to ensure it meets your
standards.
Ensure your point of contact and other customer personnel (as requested)
are de-briefed on the service.
Discuss the results and possible future visits.
Inform customer personnel they will receive final documentation once all
vendor reports have been gathered and incorporated into the main service
report.
Properly check yourself out and depart from the site.
“A Day in the Life…”
Completing service documentation
 Once all service documentation has been gathered, download to the




appropriate document management system (DMS)
Our Document Management systems is XXXXXXXX?
Once all documents for a given service
ticket have been downloaded to the DMS,
return to your original service ticket to
note the status of the maintenance as
well as any issues or procedures that should be recorded.
Contact the Contact Manager by phone or email to discuss any issues
requiring follow-up.
Assist the CM as necessary to ensure these items get addressed.
QUESTIONS
?
Managed Services
Sub Contractor Expectations
Sub Contractor Expectations

Your relationship with your SUB’s can make or break your
relationship with your customers.
 Your customer’s expectation is that the services package they
purchased from us should be of a consistently high quality.
 It is incumbent upon us to satisfy each customer’s expectations,
regardless of which SUB we use, or which technician is assigned to
perform the services.
 The actions of your SUB’s reflect directly on YOU…
Sub Contractor Expectations
SUB compliance






Your SUB must understand that you are their customer. As the systems expert, you
should value your SUB’s input. However, whatever you ask of your SUB should be
indisputable.
You must control all SUB activities. You must effectively communicate your customer’s
regulations and intentions.
You should be present for all SUB-performed work. If you need to leave the area, your
SUB should be forewarned NOT to make any changes or modifications during your
absence.
You must ensure they observe all procedures and prevent them from ignoring or
creating steps or from taking risks of any kind.
Be aware that most SUB’s aren’t accustomed to this level of oversight and may balk at
the idea
The SUB must follow our safety procedures or theirs, whichever is the most stringent!
Sub Contractor Expectations
Liaison technique





Our staff is probably accustomed to directly interfacing with customers
about issues and providing them with general updates.
Your customer may also have a history of working with your SUB’s prior to,
or outside of, their relationship with us. Thus, conversations may occur
between them without your knowledge.
Because your customer has purchased the service(s) through us, it is
important that you position yourself as their point of contact.
You must ensure both parties understand that you own the contract, and
you have access to the resources required by the customer.
It is essential for you to remain cognizant of anything and everything
communicated between your customer and your SUB/vendor.
Sub Contractor Expectations
SUB accountability







Your customer should and will hold you accountable for being thorough and proactive
in your performance on their behalf.
You must hold your SUB’s to the same standards of performance.
Expect them to be punctual, be prepared and keep you informed.
They should be willing to answer questions and exhibit flexibility when the need
arises.
You should expect them to follow-up until all issues are resolved, or provide a contact
at their company who will follow-up.
Their reports should be clear, concise and complete.
They should respond well to feedback on their performance, their reporting and their
professionalism.
Sub Contractor Expectations
SUB evaluations




One of the best ways to keep customers happy is to ensure them we are working with
quality SUB’s.
To ensure high-caliber SUB’s, we require vital feedback from you in the form of critical
reviews of your SUB’s technicians.
From your initial contact with them until completion of their assigned services, You
should be observing the following:

Professionalism

Maturity

Cleanliness and hygiene

Timeliness of reports

Punctuality

Thoroughness of follow-up

Communication skills
SUB’s and their employees will also be reviewed by the PM team and our service
managers.
QUESTIONS
?
Safety Hazards
Safety Hazards
UPS, switchgear, PDU, STS, ATS, RPP








Arc flash/NFPA70E: There is always the potential for arc flash when working with energized
equipment. PPE must include flame-resistant clothing, safety glasses, hood or face shield with
balaclava, ear protection, insulated gloves and cotton under garments. Remove all metal jewelry
from every part of your body.
Electrical safety: Appropriate precautions must be taken to avoid electrical shock.
Housekeeping: Keep tools and materials outside the work area, and particularly away from
energized equipment, to prevent trips.
2-man work rule: When removing panel covers, one worker firmly holds the cover with two hands
while a second worker removes the screws. This minimizes the risk of slips that could result in
opening breakers or making contact with energized parts.
Muscle pulls and strains: Use proper lifting methods when moving equipment.
Cuts: Check panels, trays and other equipment for sharp edges and avoid them.
Switching: Switching operations should always be performed with the doors closed.
LOTO: Use specified LOTO practices and procedures to safeguard technicians from unexpected
energizing or startup of machinery and equipment.
Safety Hazards
Batteries








Chemicals: Proper precautions should be taken when watering wet cell batteries. Appropriate PPE
should include face shield with balaclava safety glasses, apron, rubber gloves and slip-resistant
shoes.
Spills: Have a spill kit accessible when working with electrolyte, when watering batteries or when
taking hydrometer readings.
Eyes: Have an eyewash station readily accessible when working with batteries.
Muscle pulls and strains: Batteries are heavy and require proper lifting techniques when removing
or replacing them.
Hydrogen gas: Batteries emit hydrogen gas, which is harmful to your health. The battery area
should be properly ventilated and include appropriate detection devices.
Electrical safety: Proper precautions should be taken to avoid electrical shock. Tools must be
properly insulated to prevent grounding.
Switching: Switching operations should always be performed with the doors closed.
LOTO: Use specified LOTO practices and procedures to safeguard technicians from unexpected
energizing or startup of machinery and equipment.
Safety Hazards
Fire system








Electrical safety: Proper precautions should be taken to avoid electrical shock. Some energized
parts are exposed when panel is opened.
Ladder safety: Ladders may be employed to reach detectors during device testing. Refer to the
appropriate Ladder Policy.
Safety glasses: Device testing requires technicians to periodically look up toward the ceiling or
under the floor. In both cases, the eyes are vulnerable to debris.
Slips, trips and falls: Floor openings should not be left unmarked or unmanned to prevent slips,
trips and falls.
Muscle pulls and strains: Use proper lifting methods when moving equipment. Apply the “2-man
work rule” when moving heavy equipment.
Cuts: Check panels and other equipment for sharp edges and avoid them.
Fire watch: Periodic checks should be made while the fire system is in bypass.
Extinguishers: Proper extinguishers should be readily available to address the different types of
fires (electrical, chemical, etc.) that may be encountered.
Safety Hazards
Generator






Electrical safety: Proper precautions should be taken to avoid electrical shock. The E-stop should be pressed to
prevent startup while working inside the unit.
Hearing protection: Appropriate hearing protection should be worn while the generator or load bank is running.
Burn hazard: Certain generator and load bank components can get extremely hot. After running, give equipment
adequate time to cool down before touching. Be cautious about touching materials in contact with hot equipment
or in line with the exhaust from the load bank.
Muscle pulls and strains: Use proper lifting methods when lifting batteries, buckets of oil, pumps, or any heavy
parts.
Spills: Have a spill kit accessible when performing maintenance activities involving fuel (fuel polishing or
refueling). Verify the integrity of any required spill containment, the proper marking of hazardous fluids and the
availability of MSDS information.
Flammable liquids: Strictly enforce! There MUST be no open flames, sparks or smoking permitted in the vicinity of
the generator or the load bank.
Safety Hazards
Generator (continued)





Rotating parts: Keep human body parts clear of rotating parts and machinery to avoid being
subjected to cutting or crushing. DO NOT wear loose-fitting clothing or jewelry of any kind. These
items can be pulled into the machinery.
Safety glasses/face shield: The enclosure may contain a high volume of debris and particles which
can become stirred around when the generator starts. A face shield may also be required when
handling fuel (i.e., fuel polishing, refueling or working with the pump – which includes dealing
with fuel under pressure).
Trip hazard: Load bank cables should be placed clear of footpaths and egress routes. They should
also be kept out of any area that might be exposed to water.
Weather: Weather should always be assessed prior to starting a load bank preventive
maintenance. Rain or snow or other forms of precipitation could cause dangerous working
conditions.
LOTO: Use specified LOTO practices and procedures to safeguard technicians from unexpected
energizing or startup of machinery and equipment.
Safety Hazards
HVAC








Electrical safety: Appropriate precautions must be taken to avoid electrical shock. Some energized parts will be
exposed while taking readings.
Muscle pulls and strains: Use proper lifting methods when moving equipment. Employ the “2 man rule” when
moving such heavy equipment as compressors.
Fall hazard: Ladders may be used to access units located above the ceiling. Take proper precautions when standing
on rooftops. Consider using a safety harness.
Eye protection: Air blowing from units stirs up dust and debris. Wear appropriate eye protection (safety glasses,
etc.). A face shield may need to be worn when handling chemicals such as glycol.
Rotating parts: Keep human body parts clear of rotating parts and machinery to avoid being subjected to cutting
or crushing. DO NOT wear loose-fitting clothing or jewelry of any kind. These items can be pulled into the
machinery.
Weather: Weather should always be assessed prior to working on rooftops. Be cautious of slips during or after rain
or snow. If lightning is a possibility, rooftop activities should be postponed.
Hazardous Materials: MSDS should be available for all chemicals (oil, coil cleaning solution, etc.). Ensure PPE
complies with MSDS and company policies.
LOTO: Use specified LOTO practices and procedures to safeguard technicians from unexpected energizing or
startup of machinery and equipment.
Safety Hazards
Cleaning



Slips, trips and falls: Wear proper footwear to prevent slips caused by a wet floor.
Floor openings should not be left unmarked or unmanned.
Safety glasses: Air blown under the floor can stir dust and debris. Be especially
cautious when removing tiles from the floor.
Chemicals: Proper PPE should be worn when handling cleaning chemicals.
IR scan


Arc flash/NFPA70E: There is always the potential for arc flash when working with
energized equipment. PPE must include flame-resistant clothing, safety glasses, hood
or face shield with Balaclava, ear protection, insulated gloves and cotton under
garments. Remove all metal jewelry from every part of your body.
Electrical safety: Appropriate precautions must be taken to avoid electrical shock. DO
NOT allow any tools or camera parts to enter the panel.
QUESTIONS
?
Technical Overview:
Generator
Generators
Generator Safety
Fuel polishing

Working with flammable liquids
 No open flames, sparks, or smoking in the vicinity.

Chemicals are hazardous to the eyes,
nose and throat
 Wear face shield when working with diesel fuel.
 Wear chemical gloves when working with fuel.

Liquids under pressure
 Wear face shield when turning on the pump, checking for leaks, or
cleaning filters.

Heavy equipment (fuel polishing pump, filters, piping)
 Bend at the waist.
 Employ a 2-man lift for anything weighing over 70 pounds.
Generator Safety
Load bank

Noise hazard
 Wear heaving protection when generator
and load bank are operating.
 Maintain distance from generator and
load bank when possible.




Load banks get very hot
 Burn hazard – Do not touch downstream side of load bank until load is off and load
bank has cooled down for at least 10 minutes.
Ensure load bank is not exhausting toward any flammable material (i.e., leaves,
bushes, grass).
Ensure load bank is not exhausting toward any vehicles, structures or other items that
may be heated to unsafe temperatures.
Ensure load bank fans are all operating properly prior to adding load.
Generator Safety
Load bank

When disconnecting building power, the ATS may request the generator. This
would cause the generator to start and put current to the cables that are
being disconnected.
 Ensure the ATS is locked in the inhibit position and the generator switch is in the
off position prior to opening the output breaker.

Rotating equipment
 Do not wear loose-fitting clothing, necklaces, earrings, etc.
 Keep hands clear of rotating parts on the generator and load bank.

Load bank cables carry high voltage and current across the ground.
 Ensure all cable connections are locked securely prior to placing load on the
load bank.
 Route cables to avoid having to step over them during testing; ensure they are
not blocking footpaths or egress routes.
Generator Safety
Load bank

Load banks are resistance banks that can be configured in series
or parallel. They cannot be exposed to liquids.
 Ensure the weather will remain clear, and no rain or snow is forecasted,
prior to starting a load bank.
 Ensure no automatic watering heads are located near the load banks.
 Ensure load bank cables are not run in any location that may be exposed
to water during the test.

Load banks are heavy.
 Employ a 2-man lift for any load banks weighing over 70 pounds.
 Lift with your legs, and not with your back.
Generator Safety
Maintenance

Noise hazard
 Wear hearing protection while the generator
is operating.
 Maintain distance from the generator whenever
possible.

Rotating equipment
 Do not wear loose-fitting clothing, necklaces,
earrings, etc.
 Keep hands clear of rotating parts.

Hot surfaces
 Generator operation produces many hot surfaces on the generator and the
radiator. Avoid contact with any hot surfaces.
 Generator operation results in coolant under pressure. Ensure the cooling
system sufficiently cools down prior to removing the radiator cap.
Generator Safety
Maintenance

Running generators move high volumes of air
 Ensure all access panels are securely shut prior to starting the generator.
The high volume of air moved by the radiator fan can cause doors to swing
open and shut with significant force.
 Ensure any loose clothing, hats, glasses, etc.
are properly secured when working in the
vicinity of an operating generator.

Batteries contain acid
 Ensure proper chemical-resistant gloves
and a face shield are used when checking
battery acid levels.
 Ensure the ready availability of a spill kit
when dealing with generator batteries.
Generator Types
Diesel



Most common generator type
Diesel engine drives the generator
Cummins, Detroit Diesel, Generac, Caterpillar
Natural gas

Same principle of operation as a diesel, except fuel used is
natural gas as opposed to diesel fuel.
 Typically less able to handle block loading without a short
voltage drop.
Generator Types
Flywheel







Not a true generator in the conventional
sense
Typically tied directly into the DC bus
fed from the UPS
Consists of a spinning, weighted flywheel
(600+ lbs) placed in a vacuum to minimize
friction. Keeps spinning at ~7700 RPM.
When power is lost, flywheel is
transformed from a motor to a generator
that provides current to the DC bus.
Can provide power for several minutes at full load before slowing to the point
where it cannot maintain the required voltage.
Minimal maintenance. An annual inspection and vacuum pump oil change, plus a
bearing change every three years.
Generator Types
Turbine



Turbines are driven by natural gas
Constant operation, high efficiency
Consists of small kW units that can be paralleled together to
support any load size
 Some configurations allow use of exhaust to heat water or
provide heat during the colder months
Generator Characteristics
 Most generators used in critical power situations are 3-
phase, 480 volts.
 While generators come in various sizes, the sizes used
are typically in the 50kW – 3MW range.
 The most common type of generator used in critical
power solutions contains diesel-driven, water-cooled,
turbo-charged engines.
Generator Operation
Generator Components
Generator Principles of Operation
Drive end

Diesel engines






Only air is initially introduced into the combustion chamber.
The air is compressed via a compression ratio typically between 15:1 and 22:1. This results in a
40-bar (4.0 Mpa; 580 psi) pressure.
The high compression heats the air to 550°C (1022°F).
At the approximate top of the compression stroke, fuel is injected directly into the
compressed air in the combustion chamber. Depending on engine design, the fuel may go into
a void at the top of the piston or a pre-chamber.
The fuel injector ensures the fuel is broken down into small droplets and is distributed evenly.
The heat of the compressed air vaporizes fuel from the surface of the droplets.
Generator Principles of Operation
Drive end

Diesel engines





The vapor is then ignited by the heat from the compressed air in the combustion
chamber.
The start of vaporization causes a delay during ignition which results in the
characteristic diesel knocking sound.
The droplets continue to vaporize from their surfaces and burn. They keep getting
smaller until all the fuel in the droplets has been burned.
As the vapor reaches ignition temperature, there is an abrupt increase in pressure
above the piston.
The rapid expansion of combustion gases then drives the piston downward, supplying
power to the crankshaft.
Generator Principles of Operation
Drive end

Turbine engines





A gas turbine is a type of combustion engine. It has an upstream rotating compressor
coupled to a downstream turbine. A combustion chamber sits between the compressor
and the turbine.
Energy is added to the gas stream in the combustor when fuel is mixed with air and
ignited. Combustion of the fuel increases the temperature.
The by-products of the combustion are forced into the turbine section. The high velocity
and volume of the gas flow is directed through a nozzle over the turbine’s blades.
This spins the turbine which powers the compressor.
The energy released from the turbine results from a combination of temperature
reduction and pressure of the exhaust gas.
Generator Principles of Operation
Generator end

Requirements for producing current




Current-carrying conductor
Magnetic field
Relative motion between conductor and field
At generator startup, a small amount of magnetism in the armature produces a small
amount of current that flows into the field.
Generator Principles of Operation
Generator end



As the field current increases, the current output of the armature also increases until
the generator reaches saturation and the current levels out to equal the output rating
of the generator.
As load is added, more energy input from the motor end is needed to overcome the
magnetic resistance at the generator end.
Each pole of the armature creates a single phase of current. The poles are offset by
120 degrees to produce a 3-P alternating current.
Generator Components
Drive end


Typically diesel-driven
Consists of fuel delivery system, combustion chambers, oil system, cooling system and
air intake
Generator Components
Generator end

Mechanical


Rotor: Attached to the driveshaft from the engine end
Stator: The stationary part of the generator end
Generator Components
Generator end

Electrical


Armature
 The power-producing component of an electrical machine. In a generator,
alternator or dynamo, generate the electrical current.
The armature can be located on either the
rotor or the stator.
 Consists of 6 poles which, when presented with
a rotating magnetic field, induce 3-phase current.
Field
 The magnetic field component of an electrical machine. The magnetic field of
the dynamo or alternator can be provided by either electromagnets or
permanent magnets mounted on either the rotor or the stator.
 Field windings are typically located on the rotor, and the armature windings are
typically located on the stator.
Generator Configurations
You will encounter many different configurations. Configurations are based on customer needs and the
make of the generator.
In single-generator sites, one generator supports the entire critical load.
Multiple-generator solutions come in different forms:
 Parallel systems include multiple smaller generators to support the entire critical load.



These are usually connected via an ATS.
The ATS will wait for each generator to come online, reach steady voltage and synchronize prior
to transferring to its emergency position.
Redundant parallel systems also include multiple generators, each of which is capable
of supporting the entire critical load.



These generally include multiple ATSs that parallel to a common AC bus.
Critical load is transferred to the first generator able to reach 480V steady state.
The other generators are then either paralleled onto the bus when they reach 480V, or are run in
standby unloaded in case the on-line generator experiences a failure.
Generator Operations
Starting the generator


Generators can be started from the control panel on the generator or remotely via the
ATS.
To transfer critical load to the generator for maintenance:






Start the generator from the generator’s control panel.
Wait for the generator to attain an acceptable level for transferring the load.
Perform an operational inspection of the generator.
Have the ATS transfer the critical load on the generator. Refer to the ATS manual for the exact steps
used to complete this transfer.
Be sure to lock the ATS in the inhibit transfer position to prevent the ATS from transferring back to the
normal power source.
The ATS will constantly monitor utility power. Once it determines utility power is
available and stable, it will attempt to normalize.
CAUTION! The ATS is unaware if the UPS is online. If you attempt a transfer while the UPS is in bypass,
you will drop the load!
Generator Operations
Shutting down the generator




When maintenance is complete, you will want to transfer the critical
load back to normal on-line operations.
Depending on the ATS, an automatic or manual countdown needs to
be initiated. When the countdown expires, the ATS will return to
normal power and the generator placed on a cool-down timer.
Once the cool-down timer expires, the ATS sends a message to the
generator to shut down.
You need to manually shut down the generator at its control panel,
since it was started manually. Wait until the ATS has finished all
countdowns prior to turning the generator switch to Auto.
Generator Operations
Considerations



Be sure to ask for permission from the customer prior to conducting any
power transfers. Inform the customer that the ATS makes a loud sound
when a transfer is initiated. Avoid alarming the customer.
When transferring the critical load via the ATS, be aware that any loads not
protected by the UPS (cooling, lighting, office computers at some locations,
etc.) will experience a temporary loss of power.
Ensure the generator enclosure is free of debris.
Generator Operations
Load banking







Load banking should be performed annually on generators to prevent wet
stacking. Wet stacking is a buildup of carbon inside the generator that can
result in issues relative to block loading.
Load banking generally takes about 4 hours.
The generator’s output breaker is disconnected from the building.
Load banks are connected to the generator.
Ensure phases are properly connected.
The cables running to the ATS must be properly insulated and capable of
handling a full load.
Load banking involves running different load sizes for a specified time
duration. The percentage of load is increased periodically.
Generator Operations
Spill containment



Ensure a spill kit is readily available.
Acid spill containment should be available for generator batteries.
MSDS should be available for all chemicals (anti-freeze, batteries, fuel, etc.)
Follow all precautions outlined in MSDS.
Generator Troubleshooting












Ensure the fuel filter is not clogged and is full.
Ensure the throttle control leakage is not damaged or stuck.
Check the fuel level.
Check fuel quality; analysis can be performed to determine if fuel polishing is needed.
Check belt for damage.
Ensure the battery charger is operating properly.
Ensure the generator is properly wet stacked.
Check for signs of seal leakage.
Inspect the radiator for clogs.
Ensure all alarms are clear on the control panel.
Ensure the generator has been properly primed.
Check the exhaust for the presence of heavy black smoke upon starting. This is a good
indication of the onset of wet stacking.
QUESTIONS
?
Technical Overview:
Fire Suppression System
Fire Suppression System
Types of Fire Suppression Systems
Wet pipe system


System is pressurized with water all the way to the sprinkler heads.
A glass bulb in the sprinkler head will shatter when fire heats it (to greater than ~165°F).
Dry pipe system



Used in locations where ambient
temperatures go beyond 40°F
(unheated buildings, garages, etc.). Not usually in data centers.
Water does not go all the way to the sprinkler head. Stopped at a dry pipe valve, which is a
type of check valve. The rest of the line is filled with air at higher than the water supply
pressure.
When fire activates the sprinkler head(s), the air leaks out through the heads until the
differential pressure causes the dry pipe valve to open. This allows water to flow to the
sprinkler heads.
Types of Fire Suppression Systems
Deluge system




Water supply pressure is blocked by a
mechanically latched deluge valve until a
signal is received from a detector or
pull station.
When the signal is received, the valve
can be manually activated or activated via
the fire panel.
One big difference between a deluge system
and a dry pipe system is that the sprinkler
heads in a deluge system do not contain a device for
inhibiting initiation (e.g., a glass bulb).
Once the deluge valve is opened, the system fills
and water immediately shoots from the sprinkler heads.
Types of Fire Suppression Systems
Pre-action system



Because data centers contain an incredible
amount of expensive and vital computer
equipment, an accidental fire sprinkler
initiation would be catastrophic in terms
of potential damages.
Therefore, data centers are likely to use a
pre-active sprinkler system, which includes
characteristics of all previous systems.
A pre-active system can be single or double interlocked:


In a single interlock system, there must be a detection event (similar to the deluge valve) prior
to the pre-action valve opening and admitting water to the system. Unlike the deluge valve, the
pre-action valve cannot be actuated by a differential pressure caused by a leaky sprinkler head.
In a double interlock system, both the detection event and the automatic operation of a
sprinkler head are required before the pre-action valve will admit any water into the system.
Types of Fire Suppression Systems
Gaseous system (FM 200, HFC125, Halon, Inergen, etc.)


Also called a “clean agent” system. Uses an
inert gas or chemical agent to stop a fire by
isolating one or more components required
for a fire to propagate
(i.e., fuel, heat, oxygen, and chain reactions).
These systems use one of these methods:



Total flooding: completely flooding the space
upon initiation, cutting off accessibility of the
reactants from one another.
Local application: applying an extinguishing
agent directly into the fire and its surrounding area.
Safety concerns


Clean agent systems carry the risks of asphyxiation (displaying the oxygen needed to sustain
life) and barotrauma (positive pressure released onto a space that can cause damage to
humans and structures).
When a clean agent system is activated, an alarm will sound and light will flash. Be familiar
with these indicators at your facility, because they require that all personnel completely
evacuate the space or risk death.
Fire System Devices
Fire alarm system sub-panel



Located in the critical environment area, reports to the building Fire Alarm
Control Panel (FACP).
Be sure to preclude any undesired outputs by de-activating their controls (e.g.,
HVAC EPO) before performing any maintenance activities. Restore protective
functions after performing maintenance.
Batteries should be replaced approx. every 4 years. Test at every PM visit.
Fire System Devices
Pre-action and deluge valves


Mechanically latched valves that control agent release.
Trip-test these valves annually at full-flow in accordance with the manufacturer’s
instructions. If water cannot be discharged unless IT equipment is shut down,
conduct the trip test in a manner that activates the valve without charging the
lines.
Fire System Devices
Actuator solenoids

Electronically operated mechanisms that open pre-action valves.

Must be removed or de-activated for maintenance to prevent accidental
discharge of water or fire suppression agent.
Pull stations

Located near the entrance/exit to the
protected area; activated by hand for
instant discharge.
Fire System Devices
Clean agent release abort button


Located on main clean agent system panel.
Verify proper Abort and Time Delay operations. Time delays vary by
geographic region.
FM 200 tanks and nozzles


Check the agent level and cylinder pressure semi-annually.
Ensure tanks and discharge nozzles are not damaged or obstructed.
Fire System Devices
VESDA (Very Early Smoke Detection Apparatus) system



Aspirating smoke detection technology offers the
earliest
possible warning of a potential fire.
During most maintenance situations, the VESDA system should be isolated to
prevent component activation or false alarms.
Check batteries and filters routinely. Batteries should be replaced every 3-4 years.
Fire System Devices
Photo-electric smoke detector and heat detector


Verify correct zoning and/or correct location.
Perform a test of the detection system by introducing a simulated fire condition
and causing the actuating controls to move to the “discharge” position.
Fire System Devices
Fire extinguisher




Verify tank has current inspection tag and/or
extinguisher is full.
Wall mounted; an unrestrained extinguisher is
potential hazard.
Ensure 3 feet of unobstructed access on all sides.
Types: Halotron, CO2, Dry (ABC)
Fire alarm annunciator

Verify correct zoning and/or correct location.
a
Fire System Devices
Sprinkler heads



Verify detector and sprinkler head locations correspond to the as-built drawings.
Ensure sprinkler heads are not damaged or obstructed.
Code requires availability of spare sprinkler heads and wrench.
Fire alarm horns and strobe lights

Ensure all horns and strobe lights are operational by simulating a cross-zone fire
detection.
Fire System Operation

Contact building engineer to request the fire system be placed in
test during maintenance activities.

Ensure system is restored after completing maintenance.

Verify with building engineer that alarms are being received as
expected.

In the event fire protection capability is lost, tour spaces
periodically with a fire extinguisher.
Fire System Troubleshooting

Check panel indication and react accordingly.

For an alarming smoke head with no indication of fire, check air
flow, dust and filters.

Perform a visual inspection to ensure there is no fire.
QUESTIONS
?
Technical Overview:
HVAC
Heat Exchanger
A heat exchanger is a device that facilitates the transfer of heat
from one medium to another.



By making a medium cooler, we say we’ve transferred heat out of it. Within a heat
exchanger system, that medium is the heat source.
By making a medium warmer,
we say we’ve transferred heat
into it. That medium is the heat
sink or heat reservoir.
HVAC operates on the principle
of transferring heat from one
medium to
another, with the
goal of providing climate
control in terms of heat and humidity.
The Refrigerant Cycle




Low-pressure liquid expands, absorbs heat and evaporates into a
low-pressure gas at the evaporator outlet.
The compressor pumps low-pressure gas through the accumulator and discharges high-pressure
gas into the condenser (the accumulator protects the compressor by preventing slugs of liquid
from entering).
In the condenser, heat is removed from
the gas. This condenses it into a
high-pressure liquid.
An expansion device (valve/orifice)
between the condenser and the
evaporator helps regulate the flow into
the evaporator (a strainer is located
just before this device to prevent clogging.
The temperature of the air passing through
the evaporator is controlled by a
thermostat that either starts or stops the
compressor to begin or end this cycle.
The Refrigerant Cycle
HVAC Types
DX (direct expansion or split) systems – in which refrigerant
is used as the main cooling medium
HVAC Types
Chilled water systems – in which chilled water is used as the
main cooling medium
HVAC Manufacturers
Stulz
Liebert
Mcquay
Trane
Uniflair
Dankin (YORK)
KYOTO COOLING
DX System
Characteristics
DX = A general term applied to computer room air conditioning systems
that are air, glycol or water cooled and include a self-contained
refrigeration system.







Smaller systems for buildings with limited space designated for cooling equipment.
System using refrigerant gases to transfer heat through compartments.
Evaporator unit in data center location with condenser located outside.
May contain one or more compressors, depending on the size of the system.
May be a combination of a building chilled water loop and an internal tube with a small
internal refrigerant coil to cool water before it goes into the evaporator.
Require most data centers to isolate their fire system during maintenance.
Can cause fire suppression system to discharge if refrigerant leak occurs (high pressure
refrigerant entering space obscures laser internal to smoke head).
DX System
Components




Compressor: An essential component in the refrigeration cycle. Uses mechanical energy to compress (or
squeeze) gaseous refrigerant. It allows an air conditioner to absorb heat at one temperature (like 70ºF /
21ºC). and exhaust it outdoors at a potentially higher temperature (like 100ºF / 38ºC).
Evaporator coil: An essential component in the refrigeration cycle. Looks like an automobile radiator. This
part of the system gets cold to the touch (about 45ºF / 7ºC) during normal use. Usually found in the space
from which heat is removed.
Refrigerant loop: Closed cycle of evaporation, compression and condensation. Has the net effect of
moving heat energy away from one environment and into another. The refrigerant changes its physical
state from liquid to gas and back to liquid. While changing from liquid to gas, heat energy flows into the
refrigerant from the area to be cooled (the IT environment, for example). Conversely, while changing
from gas to liquid, heat energy flows away from the refrigerant to a different environment (outdoors or to
a water source).
Reheat module: A heating coil installed in a computer room air conditioner or air handler to assist in
dehumidification of the discharge air stream.
DX System
Components (continued)




Condenser coil: One means of heat rejection commonly used in an air conditioning system.
Typically located on an outdoor pad or rooftop. Looks like an automobile radiator in a cabinet.
Usually hot to the touch (about 120ºF / 49ºC) during normal use. Transfers heat energy from the
refrigerant to the cooler surrounding environment (usually outdoor). The related dry cooler or
fluid cooler acts and appears similar. The difference is that the condenser coil uses hot refrigerant
which changes from a gas to a liquid as it moves through the coil. The dry and fluid coolers use hot
liquid, such as water or a water-glycol mix.
Evaporator fan/blower: The fan that either pulls or pushes air over the evaporator coils so that it
can be distributed throughout the data center.
Condenser fans: The fans on the condenser unit that blow air across the coils to reject the heat
pulled from the data center.
Humidifier module: The device used to provide humidification in the data center or IT room.
Humidifiers either use heat or rapid vibrations to create water vapor. The moisture is usually
added to the air stream exiting the air conditioner or air handler.
DX System
Components (continued)




Expansion valve: An essential component used in the refrigeration cycle. Regulates the flow of
high-pressure liquid refrigerant into the evaporator coil. Designed to open just enough to let the
refrigerant flow, while maintaining a
high-pressure differential from its inlet to its outlet. The
pressure at the valve’s exit is low enough that it initiates a phase change in the liquid refrigerant
to a vapor.
Controller: A computer logic-based system that monitors, controls and reports data on
temperature, humidity, component performance, maintenance requirements and component
failure.
Leak detection: A device used in IT rooms and data centers to sense the abnormal presence of
liquid water due to a leak or condensation. This can be a rope or puck-type device that has
electrodes hanging from the bottom. When water makes contact across the electrodes it closes
the circuit and causes an alarm.
Condensate pump: Connected to the evaporator coil drain pan and the humidifier unit, the pump
removes excess water from the system by employing a drain pipe.
DX System
Configurations





Perimeter cooling: CRAC units located around the perimeter of the data center or
IT room, directed inwards.
In-row cooling: An HVAC solution that shares rack space with the IT components.
Ceiling mounted: An HVAC solution mounted overhead, generally above an
acoustic drop-ceiling.
Up-flow: Supply air from the air conditioner that is directed toward the top of
load equipment or captured by an above-ceiling plenum and directed through
ducted diffusers.
Down-flow: Supply air from the air conditioner that is captured by under-floor
plenum and directed to load equipment through grated tiles and/or floor
diffusers.
DX System
Operation
DX System
Operation (continued)







Air conditioners use chemical refrigerant, which is converted from a gas to a liquid and back again. Heat is
transferred from the inside air to the outside air. Air conditioners have three main parts: a compressor, a
condenser and an evaporator.
The compressor and condenser are usually located on the outside air portion of the air conditioner. These are
often referred to as coolers or condensing units. The evaporator is located inside the CRAC or is a separate piece of
equipment called an air handler, air handler unit (AHU) or fan coil.
Refrigerant arrives at the compressor as a cool, low-pressure gas. The compressor squeezes its fluid content to
create a hot, high-pressure gas.
The gas flows into the condenser, where it cools and becomes a liquid.
The liquid enters the evaporator through a tiny, narrow hole. Inside, the liquid’s pressure drops and it begins to
evaporate into a gas.
While this is happening, heat is extracted from the surrounding air. By the time the gas leaves the evaporator, it
has become a cool, low-pressure gas.
The gas is then sent back to the compressor to begin the same process anew.
DX System
Operation (continued)









A fan connected to the evaporator blows the air across the evaporator fins.
Because hot air is lighter than cold air, the hot air in the room rises.
Vents in or near the ceiling suck the hot air into the air conditioner.
The hot air is used to cool the gas in the evaporator.
As heat is removed from the air, the air cools.
The cool air is blown into the data center, either under a raised floor or via ducts directly into the
room above the floor.
This continues repeatedly until the desired room temperature is attained.
When the thermostat senses the room temperature has reached its setting, the thermostat turns
the compressor off.
As the room warms, the thermostat turns the compressor on again.
Note: Set points for temperature and humidity are programmed into a controller on each unit. Reheat
and humidifier modules may be used to control humidity.
DX System
Safety








Disable the fire system to avoid an accidental discharge during maintenance
and
troubleshooting. This is often dependent on customer and site specifications (pre-determined
during kickoff or agreed to at the start of the maintenance visit).
Wear proper PPE to include (but not be limited to) safety glasses, hearing protection, fireretardant clothing, rubber gloves and cut-resistant gloves.
Refrigerant gases are extremely cold and can cause severe burns.
Refrigerant gases are under pressure and can cause severe damage if skin contact is made.
Opening or removing some panels exposes high-voltage conditions.
Smoke detectors usually installed.
Water leak detection usually installed in unit.
Follow MSDS for refrigerant. Refrigerant work requires elevated maintenance controls.
Understand these controls fully before engaging in any maintenance activity where the refrigerant
boundary may be compromised.
Chilled Water System
Characteristics

CW = A type of precision cooling system used in mid to large IT environments in which cold water
is pumped from a chiller to computer room air handlers. The water may be provided via a building
utility, or dedicated chillers can be installed.
Chilled Water System
Characteristics (continued)
CW = A type of precision cooling system used in mid to large IT environments in which cold water is
pumped from a chiller to computer room air handlers. The water may be provided via a building utility,
or dedicated chillers can be installed.





Uses water or glycol media to transfer heat outside the data center for heat rejection. Glycol is
used in areas where the chilled water loop may be exposed to freezing conditions.
Should have an under-floor rope leak detection system for monitoring, due to the increased
potential for leaks.
Requires a chiller to produce large volumes of chilled water.
Requires monitoring of chemical properties of water loop.
Requires cleaning of cooling towers on certain systems to remove scale build-up and assure
proper flow.
Chilled Water System
Components







Air handler: A device that moves air.
Cooling tower/chiller: A cooling tower is a device used to continuously refrigerate large
volumes of water. A chiller produces large volumes of chilled water (typically 45-48ºF / 79ºC) and distributes the chilled water to Computer Room Air Handlers (CRAHs), which are
designed to remove heat from the IT environment.
Evaporator: Contains the evaporator coil and fan/blower. Removes heat from the space
(cold air enters the space and hot water returns to chiller to be re-cooled.
Evaporator coil: An essential component in the refrigeration cycle. Looks like an automobile
radiator. This part of the system gets cold to the touch (about 45ºF / 7ºC) during normal
use. Usually found in the space from which heat is removed.
Evaporator fan/blower: The fan that either pulls or pushes air over the evaporator coils so
that it can be distributed throughout the data center.
Chilled water valve: A device, often controlled by an electric motor, used to regulate the
flow of water or glycol through the coil and/or heat exchanger in a computer room air
conditioner or air handler.
Chilled water loop: The water loop flow from the chiller to the evaporator coil.
Chilled Water System
Components (continued)

Chilled water pump package: The enclosure containing pumps used to circulate condenser water or glycol
mixture. Pump packages are specified based on desired flow rate and acceptable flow characteristics for
each application.

Water treatment package: A pump system used to regulate the chemicals in the chilled water system to
minimize damage and scale build-up in the system.

Storage tank: Holds the chilled water while the system is being repaired or maintained. Sometimes holds
additional chilled water for power outages or component failures.

Condenser water loop: The water loop that runs from the chiller’s heat reservoir.

Condenser pump package: The enclosure containing pumps used to circulate condenser water or glycol.
Pump packages are specified based on desired flow rate and acceptable flow characteristics for each
application.

Make up water: The water required for some open chillers to maintain the proper fluid levels in order for
the system to operate.
Chilled Water System
Components (continued)

Reheat module: A heating coil installed in a computer room air conditioner or air handler to assist with
dehumidification of the discharge air stream.

Humidification module: Humidity cannot be too low in a data center, due to the risk of static charge
buildup. This device provides humidification in the data center or IT room. Heat or rapid vibrations are
used to create water vapor. The moisture is usually added to the air stream exiting the air conditioner or
air handler.

Controller: A computer logic-based system that provides thermostat-based HVAC control. It also monitors
and reports data on temperature, humidity, component performance, maintenance requirements and
component failure.

Condensate pump: Connected to the evaporator coil drain pan and the humidifier unit, the pump
removes excess water from the system by employing a drain pipe.

Leak detection: A device used in IT rooms and data centers to sense the abnormal presence of liquid
water due to a leak or condensation. This can be a rope or puck-type device that has electrodes hanging
from the bottom. When water makes contact across the electrodes it closes the circuit and causes an
alarm.
Chilled Water System
Configurations

Perimeter cooling: CRAH units located around the perimeter of
the data center or IT room, directed inward.

Up-flow: Supply air from the air handler that is directed toward
the top of load equipment or captured by an above-ceiling
plenum and directed through ducted diffusers.

Down-flow: Supply air from the air handler that is captured by
under-floor plenum and directed to load equipment through
grated tiles and/or floor diffusers.
Chilled Water System
Operation
Chilled Water System
Operation (continued)






Operates similar to a DX system, except either water or a glycol mix is used as the system’s medium for rejecting
the heat.
The cooled liquid flows through building pipes
and passes through coils in air handlers,
fan-coil units or other systems.
The air in the building is cooled and usually
dehumidified.
There are two types of chillers: air-cooled and
water-cooled.
Air-cooled chillers are usually located outside
the building and consist of condenser coils
that are cooled by fan-driven air (dry coolers).
Water-cooled chillers are usually located inside the building. Heat from these chillers is carried by recirculating
water to a heat sink (i.e., outdoor cooling tower).
Note: Set points for temperature in the CW loop are programmable. Controllers inside the units regulate a chilled water
valve to maintain the desired temperature. Reheat and humidifier modules may be used to control humidity.
Chilled Water System
Troubleshooting












Check whether meter on make-up water has changed. This would indicate a potential leak
in the system.
The refrigerant gauges must be connected to verify the proper charge levels of the
refrigerant system for the chiller.
Check the chilled water valve for proper operation and signs of leaks.
Inspect the visible piping connections for potential leaks.
Inspect the chilled water pump seals and the condition of the bearing.
Begin troubleshooting efforts at the panel (check alarm history and current parameters).
Check for damaged/wet lagging.
Check nozzles for proper water flow. Reduced flow could indicate a piping restriction.
Check condensate pump drain and float operation.
Verify proper input/voltage and amp values.
Verify all fans are running.
Verify belts are in good condition.
Chilled Water System
Safety









Wear proper PPE to include (but not be limited to) safety glasses, hearing protection, fireretardant clothing, rubber gloves and cut-resistant gloves.
Refrigerant gases are extremely cold and can cause severe burns.
Refrigerant gases are under pressure and can cause severe damage if skin contact is made.
Opening or removing some panels exposes high-voltage conditions.
Serious leaks under floor can increase risk of electrical shock when working under floor.
Smoke detectors usually installed.
Water leak detection usually installed in unit.
Rope leak detection usually installed under floor to detect leaks.
Follow MSDS for refrigerant. Refrigerant work requires elevated maintenance controls.
Understand these controls fully before engaging in any maintenance activity where the
refrigerant boundary may be compromised.
HVAC Information Gathering
Equipment information












Manufacturer
Description (DX or chilled water)
Model number
Serial number
Building number (special information about build specifications)
Capacities of system
Diagram of piping, valves, pumps and tanks
Location of all equipment in the system
DX: evaporator units, condenser units, piping runs
Chilled water: evaporator units, chiller yard, piping runs
General appearance
Digital pictures
HVAC Information Gathering
Safety equipment provided by site




Location of fire extinguishers
Location of eye wash station
Location of emergency shower, if required
Refrigerant recovery cylinders
Access to equipment







Chiller yard
Cooling units
Condenser units
Pump packages
Isolation valves
Electrical isolation breakers
Passwords
HVAC Information Gathering
Egress route


Egress route internal to building
Egress route outside building in chiller yard
Maintenance information





Scope of maintenance activity
Maintenance window
Current maintenance schedule frequency
Special site requirements
Previous maintenance reports
HVAC Information Gathering
Spare parts storage and supplies










Refrigerant
Water treatment chemicals
Refrigerant oil
Refrigerant gauges
Refrigerant recovery tasks
Spill clean-up supplies
Contactors
Belts
Filters
PC boards/Controls
QUESTIONS
?
Technical Overview:
UPS
UPS Systems
UPS Safety









Arc flash/NFPA70E: There is always the potential for arc flash when working with energized equipment. PPE must include
flame-resistant clothing, safety glasses, hood or face shield with baklava, ear protection, insulated gloves and cotton under
garments. Remove all metal jewelry from every part of your body.
Electrical safety: Appropriate precautions must be taken to avoid electrical shock.
Housekeeping: Keep tools and materials outside the work area, and particularly away from energized equipment, to prevent
trips.
2-man work rule: When removing panel covers, one worker firmly holds the cover with two hands while a second worker
removes the screws. This minimizes the risk of slips that could result in opening breakers or making contact with energized
parts.
Muscle pulls and strains: Use proper lifting methods when moving equipment.
Cuts: Check panels, trays and other equipment for sharp edges and avoid them.
Trapped key interlocking: Provides pre-defined scenarios of operation (i.e., opening one switch to release a key that allows
closing a second switch).
Switching: Switching operations should always be performed with the doors closed.
LOTO: Use specified LOTO practices and procedures to safeguard technicians from unexpected energizing or startup of
machinery and equipment.
UPS Vendors
Schneider Electric
Eaton (PowerWare)
Emerson (Liebert)
JT Packard (3rd Party)
DC Group (3rd Party)
Various Other (3rd Party)
UPS Manufacturers
Schneider Electric… APC (Symmetra)
Schneider Electric… MGE (Galaxy, Comet, EPS)
Eaton (PowerWare)
Emerson (Liebert)
Mitsubishi
General Electric (GE)
UPS Types



Standby: Very basic. Provides only surge protection and battery backup.
Line interactive: In addition to surge protection and battery backup, provides the ability to tap
into its autotransformer to compensate for continuous under-voltage and over-voltage conditions.
Standby-ferro: Operates similar to a standby
UPS. While a standby UPS is always online,
a ferro-resistant transformer filters output and
attempts to minimize transients cause when switching
from the main power source to battery power.
UPS Types





Double-conversion online: While similar in many ways to standby and line-interactive UPSs,
double-conversion includes a rectifier and an inverter.
The term “double-conversion” refers to the fact that the rectifier directly drives the inverter.
The batteries in this type of unit are always connected to the inverter, so no power transfer
switches are necessary.
Upon power loss, the rectifier drops out of the circuit and the batteries supply the power.
When power is restored, the rectifier begins supplying the power while charging the batteries.
UPS Types

Delta-conversion online: Considered an improvement over the double-conversion
design. Uses a Delta converter to direct much of the input power directly to the
output, thereby making it more efficient.
UPS Characteristics
Static



Double-conversion online, Delta-conversion online, flywheel
Most popular
High reliability and availability
Rotary



Motor generator/battery UPS, engine-coupled UPS
Desired for non-linear/non-balanced loads
Mechanical components call for additional maintenance
Hybrid

Motor generator used as a post conditioner for an inverter that can operate from rectified
utility power or battery
UPS Components
Rectifier/charger



Changes AC to DC
Feeds the batteries
Feeds the inverter
Inverter

Changes DC to AC
Static switch

Transfers output power from inverter to alternate source without power dips or sags
Manual bypass switch


Permits UPS isolation for maintenance
Internally or externally mounted
Batteries


Provide backup power in the event of utility loss
Wet cell, Valve Regulated Lead Acid (VRLA)
UPS Configurations
Single unit configuration
Parallel configuration: Two or more UPS modules employed to share
the load through complex power control circuitry. Applied to increase
the capacity and/or redundancy of the system.
UPS Configurations
Serial redundancy: One UPS connected to the reserve/bypass input of
another UPS. The two UPSs operate normally under normal conditions, with the
downstream UPS supplying the entire load. When one UPS experiences a
problem, the load is still fully protected by the inverter and batteries of the
other UPS.
Iso-redundant: Prevents the critical load from direct exposure to the utility
source, even during a UPS module failure or maintenance. The critical load is
divided into isolated segments, and the failure of one segment cannot affect the
other load segments. Example: Bypass of the primary UPS fed by the secondary
UPS.
UPS Principles of Operation





The utility AC source provides power to the rectifier/charger in the UPS
module.
The rectifier/charger converts the utility AC power to DC and supplies DC
power to the UPS module inverter while simultaneously float-charging
the battery plant.
The UPS inverter converts DC power to AC and furnishes AC power to
the critical bus.
If the utility source power fails or is determined to be outside the
acceptable range, the battery plant becomes the primary supplier of DC
power to the inverter.
After utility power is restored, or an alternate power source becomes
available, each rectifier/charger slowly activates to once again power the
inverters and recharge the battery plant.
UPS Operation
Identify the current status of the UPS

Critical bus OK, battery available
Monitor the power flow through the UPS

Input, battery and output voltage, frequency and current readings
Execute operational procedures

Perform transfer/retransfer between the UPS and the bypass line, startup and
shutdown
Access history

Archive alarm conditions that have occurred
Make adjustment to programmable parameters

Set date/time, select number of auto transfer attempts
UPS Troubleshooting
Alarm sounding / red or amber lights
Check UPS screen
Connect to laptop
Remote monitoring indication/email generated
QUESTIONS
?
Technical Overview:
Switchgear
Switchgear
Switchgear Safety










Arc flash/NFPA70E: There is always the potential for arc flash when working with energized equipment. PPE must
include flame-resistant clothing, safety glasses, hood or face shield with baklava, ear protection, insulated gloves and
cotton under garments. Remove all metal jewelry from every part of your body.
Electrical safety: Appropriate precautions must be taken to avoid electrical shock.
Housekeeping: Keep tools and materials outside the work area, and particularly away from energized equipment, to
prevent trips.
2-man work rule: When removing panel covers, one worker firmly holds the cover with two hands while a second
worker removes the screws. This minimizes the risk of slips that could result in opening breakers or making contact
with energized parts.
Muscle pulls and strains: Use proper lifting methods when moving equipment.
Cuts: Check panels, trays and other equipment for sharp edges and avoid them.
Trapped key interlocking: Provides pre-defined scenarios of operation (i.e., opening one switch to release a key that
allows closing a second switch).
Remote racking systems: Allows an operator to rack switchgear from a remote location without having to wear a
protective arc flash hazard suit.
Switching: Switching operations should always be performed with the doors closed.
LOTO: Use specified LOTO practices and procedures to safeguard technicians from unexpected energizing or startup
of machinery and equipment.
Switchgear Manufacturers
Square D
Cutler Hammer
General Electric
Siemens
Uniswitch
R&B Switchgear
Switchgear Classification Criteria
1.
Current rating
2.
Interrupting rating (maximum short circuit current that can safely be
interrupted by the device)
3.
Voltage class (low, medium, high voltage)
4.
Construction type (indoor, outdoor, utility)
5.
Interrupting device (fuse, circuit breaker)
6.
Operating method (manually, solenoid operated)
7.
Type of current (AC, DC)
Switchgear Characteristics
 Accepts incoming power from the transformer
 Provides safety shutoff via breakers
 Allows cross-connection of power between sources
 Distributes power to PDUs and other equipment
Switchgear Components

Busbar: Electrical conductor, maintained at a specific voltage, that is capable of carrying a
high current; usually used to make a common connection between several circuits in a
system.

Circuit breaker: Device used to prevent excessive electric current – as caused by a short
circuit – from damaging apparatus in the circuit or from causing a fire.

Relays/overload circuit: A device that breaks a connection in one circuit to protect the other
devices in the same circuit when a change of current or voltage occurs in another circuit.

PLC: Programmable Logic Controller. Digital computer used to automate such
electromechanical processes as breaker manipulation.

Meter: Installed in panel fronts to monitor such readings as voltage, amperage and kW.

Surge suppression (optional): Protects electrical devices from voltage spikes by limiting the
voltage supplied to a circuit.
Switchgear Configurations
 Site-specific
 Designed before any planned construction, renovation,
or upgrade periods
Switchgear Principles of Operation
Protection

Interrupts short circuit and overload fault currents
Isolation

Isolates circuits from power supply
Redundancy

Allows more than one source to feed a load
Switching

Local or remote switching
Switchgear Operation
Local

Manual onsite opening or closing of breakers
Remote

Opening or closing of breakers controlled by an automated
computer system in a remote location
Protective

Breakers and overloads operate automatically to protect
dedicated circuits and the system as a whole.
Switchgear Troubleshooting
 Thermal imaging: Assess the state of the system and
predict failures before they occur
 Internal arc containment
 Breaker injection testing
 Recertification
 Visual inspection
QUESTIONS
?
Technical Overview:
Batteries
Batteries
Battery Types
VRLA (Valve Regulated Lead Acid)
Flooded (wet cell)
Battery Manufacturers
C & D (Max Rate, Dynasty)
GNB (Sprinter)
Exide (Data Safe)
CSB
East Penn (Unigy)
Power
Battery Characteristics
VRLA










Sealed for safe use in data centers without need for hydrogen sensing.
Expected life is approximately 3-5 years of use. In third year, operating at
approximately 80% of battery capacity. The failure rate after three years increases
considerably.
Failure of these batteries can occur at any time with little or no warning.
Multiple strings generally required to support DC system.
Lower initial cost for installation.
Testing with Alber or equivalent to assess internal condition of the battery.
Testing is an important tool for determining condition of sealed batteries.
Testing of these batteries is recommended on a quarterly basis.
Less maintenance required due to sealing.
Electrolyte is immobilized (absorbed glass mat or gel).
Battery Characteristics
Flooded







Vented battery requires monitoring for hydrogen gases.
Clear cases for viewing the internal conditions of the battery and
electrolyte levels.
Clear cases allow visible inspection of the plates for shedding, hydration
and sulfating. These serve as early problem indicators with the jar.
Electrolyte levels can be adjusted to within the upper- and lower-level
indicators marked on the cases.
Electrolyte can be tested for specific gravity levels and temperature.
Higher initial cost for installation.
More space required for placement.
Battery Characteristics
Flooded








Expected life depends on the installation, use (cycles) and quality of the
maintenance program.
Battery testing should be performed on a quarterly basis.
Flooded jars require more maintenance than VRLA batteries. Monthly preventive
maintenance checks are recommended.
During preventive maintenance: The electrolyte levels need to be monitored, the
jars cleaned and a visual inspection conducted.
Jars need to be filled with either distilled water or de-ionized water from a
filtering system.
An annual exercise and equalization change is required.
Non-sealed system for serviceability.
Usually too heavy to be lifted manually.
Battery Components








Resilient container (plastic)
Positive and negative internal plates (lead)
Plate separators (porous synthetic material)
Electrolyte: dilute solution consisting of sulfuric acid
and water (battery acid)
Terminals: The battery’s connection points (lead)
VRLA
Generally mounted inside
enclosures, but some
applications use open racks
for easier access to batteries
for testing and replacement.
Single sealed jar with
exposed posts for
termination.





Flooded
Mounted on multi-tier racks
(2 or 3 levels high) due to size
and weight.
Jar
Busbars used for
interconnection of multiple
internal cells.
Flame arrestors with cap.
Fill or access cap.
Battery Configurations
VRLA

Generally require multiple strings to support the required DC of the UPS.
Battery Configurations
VRLA

Batteries can be located adjacent or non-adjacent according to site needs.
Battery Configurations
Flooded

Usually only a single string is required to support the DC system.
Battery Configurations
Flooded

A separate room should be used for these batteries due to their gaseous
emissions.
Battery Operations
VRLA

Limited to either online or offline.

The breaker in the string is either open or closed.
Flooded

Limited to either online or offline.

The breaker in the string is either open or closed.
Battery Troubleshooting
VRLA

The simplest troubleshooting is a visual inspection: look for cracks,
leaks, bulging, corrosion and obvious problems.

The most effective way to determine battery health is by testing
batteries with an Alber battery tester. This displays the voltage and
the internal resistance of each battery.

Test the AC ripple of the string.

Test the DC bus voltage at the beginning and end of the string, as
opposed to the DC bus inside the UPS.
Battery Troubleshooting
Flooded

The simplest troubleshooting method is conducting a visual inspection: look
for cracks, leaks, bulging, terminal corrosion, excessive shedding of plates,
and sulfation and/or hydration forming on the plates.

Use an Alber battery tester.

Test the AC ripple of the string.

Test the DC bus voltage at the beginning and end of the string, as opposed to
the DC bus inside the UPS.

Monitor the electrolyte levels to ensure all cells and jars are using
approximately the same amount of electrolyte.

Use a digital hydrometer to check specific gravity.
Battery Safety
VRLA









Use proper PPE when working around batteries (safety glasses, appropriate
shoes/boots, appropriate gloves, etc.)
DC batteries CANNOT be turned off
at any time.
Use proper lifting techniques to avoid
personal injury.
Turn the rectifier charger off and
place the UPS in bypass to minimize
DC
system potential.
Use certified insulated tools.
Employ two-man rule when lifting heavy weights.
Clean only with a water-dampened pad (NO chemicals).
Spill kit is required.
Ensure availability of Safety Data Sheets (SDA).
Battery Safety
Flooded

Use proper PPE when working around batteries (safety glasses, appropriate
shoes/boots, appropriate gloves, face shield, acid apron, etc.)

DC batteries CANNOT be turned off at any time.

Use proper lifting techniques to avoid personal injury.

Turn the rectifier charger off and place the UPS in bypass to minimize DC
system potential.

Use certified insulated tools.

Use mechanical lift to load and unload batteries.
Battery Safety
Flooded



Use a fiberglass ladder if batteries are stacked 3 tiers high.
Use a hydrogen monitoring system.
Employ two-man rule when
lifting heavy weights.
 Clean only with a
water-dampened pad
(NO chemicals).
 Spill kit and wash station are
required.
 Understand and satisfy spill
containment requirements as dictated by your facility.
Battery Principles of Operation
VRLA and Flooded

Batteries are connected in a string to provide an adequate supply
of DC voltage and amperage to support the needs of the UPS for
the critical load during loss of utility power.

Battery run-time is dictated by the needs of the customer.

Refer to one-line UPS schematics for battery connectivity.
Battery Information Gathering
Battery data











Manufacturer
Description (VRLA or Flooded)
Date code
Amp hour rating
Number of batteries in string
Number of strings in system
Racks or enclosures
Cable or busbar interconnects
Adjacent or non-adjacent to UPS
Location of DC disconnect
Previous battery testing data, if available
Battery Information Gathering
Safety equipment








Location of fire extinguishers
Location of eye wash station
Location of emergency shower, if required
Spill mediation (perimeter barrier or spill containment around each
string)
Spill clean-up cart or supplies
Mechanical lift for handling flooded batteries
De-ionized water filter system for refilling flooded batteries
PPE board with face shields, safety glasses, acid aprons and rubber
gloves
Battery Information Gathering
Access


Access to building
Ramp, loading dock, loading dock with leveling ramp, doors (roll-up door, double
doors, single door width)
 Raised floor strength

Can the raised floor support the weight of the batteries?
 Hall access width




Protective covering required for transport through facility
Data center or battery room
Door (single- or double-width), ramp onto raised floor, rating of raised floor
Elevator
 Width of door opening, capacity of elevator, freight or general access
Battery Information Gathering
Egress routes



How many routes of egress exist?
Are the egress routes clear?
Where do the egress routes lead to?
Battery Information Gathering
Maintenance




When does the work have to be done?
What is the current maintenance schedule frequency?
Testing equipment
Special site requirements
Battery Information Gathering
Spare battery stock






Charger type
Charger voltage
Rack or enclosure
Types
Quantity
Date codes
Battery Information Gathering
Flooded cell spare parts







Flame arrestors
Flame arrestor caps
Fill caps
Busbars
Hardware
No-oxide grease
Scotch brite pads
QUESTIONS
?
Technical Overview:
Data Center Cleaning
Data Center Cleaning Safety






Vacuum extension cords present a trip hazard. Ensure cords are routed in a safe
manner (along walls, where possible) and the closest available plug is utilized to
minimize cord length.
Chemicals used for spot cleaning and equipment wipe-downs may cause eye and
nose irritation. Anyone handling chemicals should wear safety glasses and
appropriate rubber gloves.
Ensure removed floor tiles are reinstalled to prevent trip hazards and fall hazards.
Data center personnel should be warned beforehand when tiles are being
removed for cleaning purposes.
Spilled mop water can be a shock hazard. Ensure minimal amounts of water are
used in mop buckets and applied to the floor.
A wet floor can also be a slip hazard. Prior to mopping, place “wet floor” signs at
all entrances to the data center.
Wear safety glasses when removing tiles. Air movement causes airborne
particulates.
Data Center Cleaning Types
Surface cleaning


Performed quarterly
Consists of sweeping, spot removal with chemical agent, damp mopping and
equipment wipe-downs
Under-floor cleaning


Performed during the annual cleaning service ticket
Floor tiles removed and sub-floor vacuumed. Perforated tiles are shaken to
remove dirt and debris. All vacuums use HEPA filters.
Workstation cleaning


Performed quarterly
Consists of workstation wipe-down, monitor cleaning and vacuuming of office
space. Generally only performed in NOC areas.
Equipment cleaning


Performed quarterly in conjunction with surface cleaning
Server racks are dusted and CRACs wiped with chemical cleaner
Data Center Cleaning Operations





The fire suppression system should be called out for cleaning. The
accumulation of dirt or water on an under-floor smoke head can cause the
smoke head to signal an alarm.
The VESDA should be disabled prior to cleaning. Dust in the VESDA can cause
a false alarm. Vacuum motors may burn out and trip the VESDA.
Some customers will request that leak detection be disabled during mopping
to prevent nuisance alarms. If water leaks onto a leak detection cable, it may
take several hours for the cable to dry enough to stop sending alarm signals.
Plug into a non-critical power supply.
Plug into the outlet via GFCI pigtail.
QUESTIONS
?
Site Awareness
Site Awareness
Security protocols
 The nature of our work demands we follow any number of
security protocols that may be specified by our customers.
 The information and equipment we support is critical to
their business operation, and they are certainly within
their rights to take whatever precautions are necessary to
control access to it.
 Consider the following security measures:
Site Awareness
Security protocols
Badging:
 One of the simplest ways a customer can limit access is through badging.
A picture badge may be used to denote whether an individual has access
to different areas, needs an escort or is even an employee. Some badges
contain embedded information or barcode that requires scanning and
“intelligence” approval to gain access.
 The keys to badging are knowing the requirements and
process (including time limitations for executing the process)
involved with obtaining a badge (i.e., security questionnaire,
fingerprinting, urinalysis), what access the badge provides,
who to contact about misplaced badges, and whether
vendors will also be badged or require escorts.
Site Awareness
Security protocols
Locks:
 This method can be as simple as providing a key that fits a lock on a chain, or as
involved as using a biometric (fingerprint, retina, etc.) scanner.
 You will need to understand the lock mechanisms employed (key, barcode,
biometric, buzz-in) and the person to contact for obtaining access.
Site Awareness
Security protocols
Time limitations:

A customer may set limitations regarding which hours you will be allowed into
their facility, as well as how much advance notice is required prior to your arrival.

The same limitations can be applied to deliveries. Some customers have internal
policies requiring them to refuse packages
that arrive without advance notice. The
customer may also require deliveries to be
X-rayed prior to acceptance.
Site Awareness
Security protocols
Drug tests:

In addition to having a strict policy regarding drug use by our employees, we will
satisfy the demands of customers with a zero-tolerance drug policy who require
drug testing of anyone working within their facilities. In other words, drug testing
may be a prerequisite.
Site Awareness
Security protocols
Site contacts:

Know who to call regarding any issue that may arise. When it comes to security
issues, this may fall outside the scope of your normal site point of contact. If
security issues arise and you do not have a security point of contact, refer the
issue to the NOC or to your project coordinator.
Site Awareness
Customer expectations
Your customer expects you to “own” his site. The definition of the term “own” is dependent
on the customer. Some will give you full and free reign regarding PM. Others will often lean
on you for advice. Still others will expect you to fill specific roles and will manage your
actions similar to the way they would manage any other vendor.
You need to understand your customer’s expectations and your communicated scope of
responsibility, and apply that knowledge in such a way as to satisfy your customer’s needs
and keep your back-office team informed. It is important that you know the following:
Site Awareness
Customer expectations

What the customer expects when he’s away vs. when he’s onsite.

How often you should do walk-throughs.

What are the customer’s “hot” buttons?
In other words, what types of responses
from you have elicited positive or
negative reactions from him? What
data center events will put the customer
or others in their company on edge?
Site Awareness
Customer expectations

How flexible and reasonable is the customer when it comes to getting hard-tofind items shipped to the site in an emergency situation?

What are the customer’s expectations regarding set points, generator run times
and other equipment-specific actions?

What are the customer’s expectations
regarding humidity and temperature thresholds?
Site Awareness
Customer expectations

What are the customer’s expectations regarding escalation procedures? In other
words, who should be contacted when various situations occur?

Who does the customer prefer as their
primary point-of-contact?

Does the customer expect you to
communicate directly with building
management, when necessary, or do
they plan to act as a liaison?
Site Awareness
Project Management team






While acting as the face of our company to your customers, your customers will also
need to interface directly with other company employees on a regular basis. Keep the
flow of communication constant among everyone assigned to your account.
Know who is the project manager for your account, and how/when to contact them.
Know who is the scheduler for your account, and how/when to contact them.
Understand what happens at kickoff meetings and how the outcome of these meetings
may affect you and your job.
Know who your available resources are within our service group, and how/when to
contact them.
Know who are the key members of your vendor organizations, and how/when to
contact them.
Site Awareness
Site layout


In addition to becoming familiar with how all of the equipment works
together in the facility, you should become familiar with the entire makeup
of the customer site.
In order to properly address everyday operations, emergency situations,
planning, time estimates and making safety recommendations, you should
understand:




General layout
Safety hazards
Location of key components
Useful indications
Site Awareness
Site layout







Availability of site drawings, one-line diagrams, maps
Markings on equipment (know the labeling method and how to utilize
power/flow paths on equipment)
Evacuation routes/exits
Customer-specific procedures
(turning off fire system, EPO)
How to prepare the site for maintenance
Room layout of the entire site
Room access (requirements,
contacts for obtaining access)
Site Awareness
Equipment integration
In order to operate effectively and be prepared to combat unknown occurrences, you must
become familiar with how different items of equipment are integrated.
Take it upon yourself to become aware of the following:

How piping is routed throughout the facility. This is helpful should you need to isolate a leak
or understand a
low-flow condition.

Origins and inputs (feeds) of your
high-energy sources. This will help
isolate equipment using logout/tagout
or when troubleshooting during an
emergency.

Know where to find relevant equipment
manuals and drawings.
QUESTIONS
?
Systems Integration
Systems Integration
Power distribution flow path
Systems Integration
Power distribution flow path



High voltage utility power is reduced to medium and then low
voltage via a succession of step-down transformers. It is then
isolated using medium and low-voltage switchgear.
This utility power is fed through the facility’s internal distribution
system (downstream of the switchgear).
The critical load is protected by an uninterruptible power supply
(UPS). Backup power for the site is provided by an emergency
generator via an automatic transfer switch (ATS).
Systems Integration
Emergency generator/ATS
Systems Integration
Emergency generator/ATS




Upon loss of utility power, the ATS will start the generator and
transfer the critical load to emergency power.
The ATS will seek to restore its normal condition (fed from utility) by
detecting the availability of utility power.
This automatic action may or may not be desired, depending on
whatever operation is underway (troubleshooting, special
maintenance procedures, changing power lineup before/after
loadbank, etc.
The ATS should include an “inhibit” function (usually a switch inside
the ATS) to prevent this action from happening.
Systems Integration
Bypass arrangements in power distribution


Static bypass: Allows the load to be transferred safely to the utility source should
the UPS module experience internal problems.
Maintenance bypass: Allows a UPS module to be safely serviced without
requiring shutdown of the load. Ideally, the maintenance bypass is housed in
its own external enclosure
to allow complete isolation
of the UPS for emergencies
or maintenance, but this isn’t always the case.
Systems Integration
Affect on fire system




VESDA: An active system that samples the air for signs of smoke.
Smoke heads (including inside equipment, ducting, over-head or under-floor): Passive
detection.
Both of these systems can cause initiation of a fire suppression system, if configured to do so.
However, erroneous initiations
are also possible (changes in air
speed/direction can
take dust
and other particulates airborne,
a high-pressure refrigerant leak
might fool
the system into
detecting smoke, etc.).
Systems Integration
Are there automatic shutoffs based on ambient
temperature or temperature at a particular component?
Fire/smoke dampers shutting
will stop HVAC air flow
Systems Integration
Automatic shutdown of some AC units in the event of fire
to stop air movement.
Systems Integration
Power line-up

Switchgear set up for handling multiple sources
Systems Integration
Power line-up

Prevent multiple UPSs from failing into one another.
Systems Integration
Redundancy

Redundancy is the duplication of possible points of failure

Minimizes the effect of any single failure on operations

Allows periodic maintenance without a complete shutdown, thereby decreasing the chance
of load loss

Data center redundancy

Redundant communications networks

Redundant power supplies

Redundant fire protection systems

Redundant cooling solutions/components
Systems Integration
Tier ratings
Criticality
Business Characteristics
Effect of System Design
1 – Lowest
• Typically small businesses
• Numerous single points of failure in all aspects of design
• Mostly cash-based
• No generator if UPS has 8 minutes of backup time
• Limited on-line presence
• Extremely vulnerable to inclement weather conditions
• Low dependence on IT
• Generally unable to sustain more than a
power outage
• Perceive downtime as tolerable inconvenience
2
10-minute
• Some amount of on-line revenue generation
• Some redundancy in power and cooling systems
• Multiple servers
• Generator backup
• Phone system vital to business
• Able to sustain 24-hour power outage
• Dependent on email
• Minimal thought to site selection
• Some tolerance to schedule downtime
• Vapor barrier
• Formal data room separate from other areas
Systems Integration
Tier ratings
Criticality
Business Characteristics
Effect of System Design
3
• Worldwide presence
• Two utility paths (active and passive)
• Majority of revenue from on-line business
• Redundant power and cooling systems
• VoIP phone system
• Redundant service providers
• High dependence on IT
• Able to sustain 72-hour power outage
• High cost of downtime
• Careful site selection planning
• Highly recognized brand
• One-hour fire rating
• Allows for concurrent maintenance
4 – Highest
• Multi-million dollar business
• Two independent utility paths
• Majority of revenues from electronic transactions
• 2N power and cooling systems
• Business model entirely dependent on IT
• Able to sustain 96-hour power outage
• Extremely high cost of downtime
• Stringent site selection criteria
• Minimum two-hour fire rating
• High level of physical security
• 24/7 onsite maintenance staff
Systems Integration
Sequence of operations (open discussion)

How does each item of equipment in the data center function in
conjunction with other equipment?
 How can each item of equipment be properly and safely
maintained without incurring a load loss?
Systems Integration
EPO hook-up
 Most data centers contain one or more Emergency Power
Off (EPO) buttons or switches.
 The EPO buttons/switches are typically wall-mounted at
convenient and strategic locations in the data center.
 Activating any EPO switch/button shuts
down the power to the data center in the
event of an emergency situation
 Prevent serious injury or loss of life to
personnel
 Prevent catastrophic damage to the facility
or equipment
QUESTIONS
?