Executive Summary 3
Distributed / Centralized - System description 3
Centralized or Distributed?
4
1 - Focus on Reliability 5
1.1 So, does this mean that such MTBF calculations are of no use? 5
2 - Focus on Withstand Rating (short-circuit capacity) of the bypass 7
3 - Emerson Network Power Distributed and Centralized Designs for Modular UPS Architectures 8
9
9
The calculation method 9
I) Elements in Series
II) Elements in Parallel
III) Power Availability
IV) The Reliability Block Diagram (RBD) of a single UPS module
V) Parallel Redundancy and Connection 11
Additional considerations on MTBF values - Maintenance and Human Error 14
1) Maintenance 14
9
10
9
9
2) Human Error 14
15
2

When designing a power protection scheme for data centers, IT and facility managers must ask themselves whether a distributed or centralized backup strategy makes more sense. Unfortunately, there is no easy answer to that question. Companies must weigh each architectural advantages and disadvantages against their financial constraints, availability needs and management capabilities before deciding which one to employ.
This white paper explores the principle of centralized versus distributed bypass and will apply it equally to standalone monolithic and integrated-modular UPS architectures, especially trying to clarify the differences in two major areas:
Reliability (in terms of comparative Mean Time Between Failure “MTBF”) and hence availability
Fault clearing capacity and short-circuit withstand rating
By following the suggestions in this white paper, data center operators can simplify their decision-making process by receiving an overview of weaknesses and capabilities of both system designs, whichever strategy they ultimately select.
In a distributed bypass architecture, each UPS module has its own internal static switch (Fig. 1), rated according to
UPS size, and each UPS monitors its own output. If the UPS system needs to transfer to bypass, each static switch in each module turns on at the same time and they share the load current amongst themselves. Here below a single line diagram is shown for clarity. On the contrary, in a centralized bypass system (Fig. 2), there is one large common static switch (also known as “MSS” Main Static
Switch) for the entire UPS system, according to the size of the known final size of the system. If the UPS system needs to transfer to bypass, the load current is then fed through the MSS. Here below a single line diagram for a distributed parallel architecture is shown.
The centralized bypass is an alternative to the distributed bypass. Technically, the two solutions fulfill the same purpose (to guarantee power continuity for example) but have different architectures. Whilst it is true that distributed bypass solutions are the most common because of their flexibility of use and low initial cost, it is also true that, in the medium/large data center market, centralized bypass solutions could be preferable in terms of reliability, performance, footprint and sometimes cost; even more so in the case of large installations where the number and type of protections as well as system wiring have a big impact. It is therefore important to respond to the various requirements with flexible solutions, which are able to adapt to the growing demands of the market in terms of availability, capacity and performance requirements.
Distributed Bypass Architecture
Input
Rectifier
Battery
Inverter
Rectifier
Battery
Inverter
Rectifier
Battery
Inverter
Rectifier
Battery
Inverter
Figure 1. Example of a distributed bypass architecture design.
Output
3

Rectifier
Battery
Inverter
Rectifier
Battery
Inverter
Centralized Bypass Architecture
Rectifier
Battery
Inverter
Rectifier
Battery
Inverter
Main Static
Switch
Input
Output
Figure 2. Example of a centralized bypass architecture design.
new UPS system, upgrading or changing their existing electrical infrastructure.
Large organizations need tailored configurations that meet availability and manageability requirements.
The choice of the configuration is also affected by the existing situation, whether the customer is getting a
In Table 1 below, some of the major points which need to be considered when evaluating the right setup for a parallel UPS system are listed.
Distributed Bypass Centralized Bypass
System
Maintenance bypass
UPS 1
SS
UPS 2
SS
UPS 3
SS
UPS 1 UPS 2 UPS 3 Main
Static
Switch
System
Maintenance bypass
System scalability – fuzzy growth plans
Lower up-front cost
Robust design / higher fault clearing capacity
System scalability – clear growth plans
Typically smaller footprint Higher system reliability for large system installations
Easier installation and deployment (simple switchgear) Higher up-front cost
More complex system management
(no single point of control)
Typically applied for small/medium business space
(power module up to 200 kVA)
Typically larger footprint
More complex installation and deployment
(possibility to integrate MSS into switchgear)
Possibility to monitor and control the system from a central point (MSS)
Typically used for large enterprise space
(multiple power module > 200 kVA)
Table 1. Considerations about distributed and centralized parallel system configurations.
4

Here below some of the major arguments of the above comparison which need to be cited separately:
Invest-as-you-Grow (minimized capital expenditure)
– the restriction being that in a centralized bypass architecture the static switch (rated for the maximum load) had to be purchased ‘up-front’ even if the load was forecast to grow over time; while distributed bypass architectures help save on Capex by adding
UPS incrementally each time the load increases, instead of making large upfront investments in centralized configurations with more capacity than they initially need.
Typically many manufacturers of critical power equipment, such as UPS, publish ‘reliability’ data for their products in the form of an MTBF. Allied with an estimate of the Mean
Down Time (MDT) or Mean Time To Repair (MTTR), which should include an element of ‘travel time’ and ‘spares logistics’ which both reflect the anticipated reaction time of the service organization to respond to a site emergency, a simple calculation can produce the commonly used (and often abused) metric of ‘reliability’ – that of availability.
Single-point-of-failure (SPoF) – the idea being that you may have an N+1 modular redundancy but only a single N rated single static-switch compared to a distributed system which has N+1 bypass redundancy as well as UPS module redundancy.
There is an often ignored issue which works slightly against the argument of ‘invest as you grow’, in that with both systems the client still needs to install the full supporting infrastructure of transformer, power cabling and LV switchgear (and even standby generation) regardless of the planned phased load growth.
Only a small number of manufacturers produce MTBF data from actual field measurements and, when they do, the absolute number is based on cumulative field service hours run compared to the number of failures of the output bus voltage. Taking the cumulative hours run, rather than the hours run between failures on an individual single module/ system, can give a distorted view of an individual system
MTBF as it shares multiple failures over a diverse group of individually isolated systems. On the other hand, when redundant power architectures are taken into account, the ‘theoretical’ module MTBFs can be used to generate system level MTBFs of such magnitude that the absolute value is meaningless, or, at best, unimaginable, and claims for UPS system MTBFs of more than 100 times the plant life expectancy are commonplace.
The second point (a SPoF) is debatable on a technical basis although it was and still is hardly ever actually
‘debated’: for ‘open circuit’ failure modes, where the switch simply fails to operate when instructed to do so, it is clear that N+1 is favorable to N as you maintain the N+1 redundancy in both module and bypass. The technical caveat is always that the reliability of the bypass paralleling control and communication bus is at least one order of magnitude higher than the staticswitch themselves.
However, it is the ‘short circuit’ failure mode that needs to be carefully considered. In this case, the N+1 architecture could become “multiple SPoFs” and the single switch can be considered more robust. A shortcircuit failure of a static-switch is one which connects the grid to the output bus of the UPS system and raises the chance of the output bus voltage to be disrupted long enough to negatively affect the critical load. The frequency with which this failure mode is open (fail to safe) or closed (fail to short-circuit) is open to analysis, and the MTBF comparison that is developed later in this paper takes this difference as a “likelihood of occurrence” between the two paralleling architectures to the heart of the calculation.
Building a critical facility which may have a design life of around 25 years and trying to compare competing power systems with claimed MTBF’s of over 100 years when it is clear that the anticipated economic service life of the plant is around 15 years is an exercise fraught with difficulty and based on assumptions.
1.1 So, Does This Mean That Such MTBF Calculations
Are of No Use?
Not necessarily: if used with care and when the input data is based on the same set of underlying assumptions, the concept of ‘comparative’ system reliability is useful. This white paper will demonstrate that reliability calculation techniques, such as the
Reliability Block Diagram (RBD) method (please see Appendix at the end of this document), can be effectively used as a tool for ‘comparison’ of alternative power architectures. The results should be used in comparison and should not be used to state or infer absolute MTBFs. In other words, a system ‘A’ which has a calculated MTBF of 5,000,000 hours can be regarded as twice as likely to fail (at any time) compared to a system
‘B’ of MTBF 10,000,000 hours – whilst we should ignore the ‘lower’ absolute number of 5,000,000 hours (571 years) which may appear to be more than ‘sufficient’ for the application.
5

Some UPS manufacturers will state the MTBF of a single module and, often, will derive from that a ‘system’
MTBF that applies only to the equipment that they are providing. If the power system designer takes this figure in isolation, they would be in danger of dramatically overstating voltage reliability at load terminals because both modules would be fitted with paralleling controls as well as communication hardware and software that ensures parallel load sharing and output synchronization. The communication cable would most often be connected in a ring such that it could sustain damage at one point and not impinge upon correct operation; nevertheless, this additional control set has a finite reliability associated with its function.
On the power output side, each machine would be connected to the output switchboard through isolators to a common busbar. It is possible (and therefore has to be taken into account) that a failure in one UPS module
(for example a short circuit at the output) could adversely affect the remaining module(s) – negating the ‘independence’ of their combined reliability.
Following the RBD technique, calculations and considerations shown in the Appendix, for a different parallel redundant example, we can immediately understand that this method can be easily applied to both centralized and distributed architectures.
In the case of a distributed bypass, each module has an internal automatic bypass which avoids the need to have a common centralized bypass.
As you can see in the Appendix from the way the RBD has been constructed whilst, physically, the bypasses are in parallel, for reliability they are in series since a failure in any one will (for a short circuit failure) produce a fault in the critical bus. Thus, even if the centralized bypass architecture might be seen as a common point of failure, it’s indeed the preferable solution for large system installations since a distributed bypass design would produce one common point of failure in every module resulting in multiple SPoFs.
For example, if we consider large system installations, the positive impact is higher with a fewer number of modules in parallel, as can be seen in Fig. 3.
Relative MTBF for Bypass Architecture
8.00
7.00
6.00
5.00
4.00
3.00
2.00
1.00
0.00
Centralized
Distributed
N
1.00
1.00
1+1
7.46
5.44
2+1
5.68
3.62
3+1
4.59
2.72
4+1
3.85
2.17
Modular Redundancy
5+1
3.31
1.81
Figure 3 . Relative MTBF for centralized and distributed bypass architecture.
6+1
2.91
1.55
7+1
2.59
1.36
8+1
2.34
1.21
9+1
2.13
1.09
6
Centralized
Distributed

It should now be clear that the MTBF of a single system depends upon the failure rate in the short-circuit condition of its automatic bypass and that having a distributed bypass architecture only serves to increase the potential number of points of failure in the system with a reduction in system MTBF.
blowing the closest fuse to the faulty branch circuit) within the typical hold-up time required by the rest of the (healthy) load (e.g. 10 - 20 ms).
There is an important precaution that has to be taken into account in the distributed bypass design (Fig.
4), as the slightest difference in impedance of each parallel path will cause any short-circuit current to flow in a highly unbalanced fashion and will usually cause overload damage and cascade failure.
For large multi-module systems, a centralized bypass architecture is preferable because it is considered most reliable, thus, the only conclusion that can be reached is that, despite the apparent attractiveness of multi-module redundancy of bypasses for large system installations, the reliability aspect is in favor of a single centralized bypass.
When a short-circuit occurs downstream the UPS,
(in a section of the load itself or in a part of the fixed distribution wiring) the voltage collapses to near zero and the source current rises and flows to the fault. The rate of rise and peak value of the fault current depends upon the sub-transient reactance (output or ‘forward transfer’ impedance) of the source. The reaction of the
UPS to the collapse in voltage is to transfer the output bus to the bypass so as to utilize the short-circuit clearing capacity of the grid transformer and clear the fault (by opening the appropriate circuit breaker or
Ensuring that the cable lengths are closely matched will largely mitigate the problem but systems with long cable run differences between modules must be carefully treated. Once the path impedance is matched, we have a situation where the withstand rating of the individual bypass static switches will each take a share of the transformer sub-transient current. In fact, as UPS bypass static switches are in N+1 redundant array, then they will, in theory at least, have a higher withstand than the alternative central bypass.
However, as the UPS systems become larger, the impact of the size of the transformer has a greater influence on the design of the UPS bypass. Take as an example a 3
MW N+1 UPS fed by a 4 MVA transformer (3.2 MW at
0.8PF, 5,774 A full-load at 400 V). A transformer size such as this would typically have an output impedance
(or sub-transient reactance) of 8 percent and would be capable of delivering 72 kA short-circuit current.
Distributed Bypass Architecture
Input
Rectifier
Battery
Inverter
Rectifier
Battery
Inverter
Rectifier
Battery
Inverter
Rectifier
Battery
Inverter
Output
Figure 4. Example of the current path over a distributed bypass architecture design.
7

It is, therefore, quite straightforward in a centralized bypass solution for the 6000 A static switch to be rated in excess of the 72 kA, otherwise, a centralized bypass solution can easily be customized to match particular requests in terms of short-circuit current capability.
However, we should also consider the alternative - the distributed bypass solution - in, for example, a 6 x 600 kW module array. Each module internal bypass would need to be rated for 1/5 th of the 72 kA (around 15 kA) and this would be the ‘normal’ rating that a single module of 600 kW would be fitted with.
We need to remember that typically, a multi-modular distributed solution (in relatively small steps, e.g. 600 kW) may have been chosen rather than a big monolithic one, because of two possible drivers:
Considering large T-free modular UPS architectures:
Trinergy™ up to 1,6 MW in a single unit offers as standard a distributed bypass solution with the bypass static switch fitted in each single core as shown below in Fig.5. There is also the possibility to configure the system for a centralized bypass architecture, enabling the UPS with inhibited bypass to be connected in parallel with an external central piece of equipment up to 5000 A rating (MSS) to address higher availability or short-circuit withstand capacity requirements.
Input/output connections
Three phases of inverter
Three phases of rectifier
‘install as you grow’ approach
‘right-sizing’ the UPS system to maximize operating efficiency (since partial load is endemic in a high proportion of enterprise data centers).
Static bypass
In both cases, some of the modules will be off-line, either physically not there yet, or deliberately switched into standby because the load is partial. Now we have the situation where the transformer sub-transient current is shared by as few as two 600 kW modules,
36 kA each, thus it’s uneconomic and relatively impractical (size) to provide that level of fault-current withstand in each static-switch.
200 kVA 200 kVA I/0 Box Battery connection
Booster
Figure 5. Trinergy (up to 1,6 MW in a single unit) system architecture.
Trinergy Cube up to 3 MW (Fig. 6) in a single unit, represents an evolution of the Trinergy UPS leveraging
Trinergy successes, experience and clients’ feedbacks in order to perfectly address large system installation requirements. It offers a centralized bypass solution up to 5000 A rating as standard with the centralized bypass located in the I/O box, as largely preferred by customers and consultants for large data center installations.
The problem is exacerbated in modular UPS designs where the module building block is small in relation to the system size, e.g. 300 kW blocks in a 3 MW system.
The advantage of the modularity is that efficacy can be maximized (close to the single module efficiency) down to 10 percent system load when the load is partial, but the result could be that just two 300 kW modules are in operation when the load is less than 10 percent (giving
N+1) but the available short-circuit current would be
>70 kA, far too high to be practical for a 400 A module.
The central bypass fitted in a completely independent segregation is concurrently maintainable thanks to the input and output bypass switches located in the I/O box in order to ensure maximum system availability at all times while granting a very high fault current capacity up to 85 kA.
Core switches
Centralized bypass
Input/output connections
Core switches
When choosing Emerson Network Power UPS systems to better address large datacenter Tier III and Tier IV power requirements, the user may select between different UPS architectures (monolithic or modular scalable) all compatible with both bypass system designs (centralized or distributed).
Core
I/0 Box Core
Figure 6. Trinergy Cube (up to 3 MW in a single unit) system architecture.
8

the output switchgear and cabling infrastructure that connects the UPS with the critical load.
Going for a distributed or centralized backup strategy is a decision that IT and Facility Managers must take when designing a power protection scheme for their data center. However, there is no single answer to that question as architecture advantages and disadvantages together with financial constraints or management capabilities must be taken into consideration and evaluated by companies before deciding which way to go. The answer to the five simple questions below might help taking the right decision:
I) Elements in Series
When power flows through a unitary chain the resulting combined reliability can be calculated from the MTBF’s
& MDT’s of the two elements, A and B;
Considering A and B:
A with MTBF (A) and MDT (A)
B with MTBF (B) and MDT (B)
What are the scalability plans expected in the near future?
The MTBF of the combined unitary string is:
Which strategy makes better financial sense?
MTBF (A+B) = [MTBF(A)*MTBF(B)] / [MTBF(A)+MTBF(B)]
Which bypass architecture will better meet my availability requirements?
And the MDT of the combined unitary string is:
What are the impacts on system reliability?
MDT (A+B) = [[MDT(A)*MDT(B)]+[MTBF(A)*MDT(B)]+
[MTBF(B)*MDT(A)]] / [[MTBF(A) + MTBF(B)]
Which is the most fault tolerant solution for my data center?
As you would expect, if the two elements are identical the overall result for MTBF is half that of one element and the MDT is the same as one element.
Where large multi-module arrays are planned in high power UPS systems, the provision of a centralized bypass is preferable to a distributed architecture for both maximizing the system MTBF and safeguarding the operation during short-circuit withstand scenarios.
This is particularly specific to modular UPS systems where partial load is expected and modules are switched off to match the UPS capacity to the load.
II) Elements in Parallel
Where power flows through two elements that are joined at the output and only one element is required for the output to function (parallel redundancy) then the formulae are as follows:
Considering A and B:
Choosing Emerson Network Power UPS systems, IT and
Facility Managers can find all the answers to the above questions by selecting the backup strategy which makes more sense for each specific scenario, in order to ensure that the critical load is always protected by the most reliable solution without compromises in terms of availability, capacity and efficiency.
A with MTBF (A) and MDT (A)
B with MTBF (B) and MDT (B)
The MTBF of the parallel string (A+B) =
[[MTBF(A)*MDT(B)]+[MTBF(A)*MTBF(B)]+[MTBF(B)*
MDT(A)]] / [MDT(A) + MDT(B)]
And MDT (A+B) = [MDT(A)*MDT(B)] / [MDT(A)+MDT(B)]
The Calculation method
Building from basic principles, many models for subsets of the power train (between MV supply network and the LV load terminals) can be developed – whilst the inherent dangers of oversimplifying the RBD’s
(making them ‘look’ like the single line diagram of the plant that they model) should be considered. It should also be noted that theoretical MTBF data sets provided by UPS system manufacturers rarely take into account
Now the situation is reversed: If the two elements are identical, the resultant MDT is half that of one element but the combined MTBF becomes very long.
III) Power Availability
A good measure for comparing different power system architectures is power Availability, as described here below.
9

Availability, normally expressed as a percentage, is a simple relationship between MTBF and MDT (or MTTR):
Availability = MTBF / (MTBF + MTTR) x 100%
For example, if the MTBF was calculated to be 87,600 hours (ten years) and the MTTR was estimated to be 4 hours, then the availability would be 99.9954 percent.
It is important to note that for an availability figure to be stated, both the MTBF (usually calculated) and the
MTTR (usually an assumption) must be known. Hence, very high availability numbers can be stated simply by assuming (and not stating) a very short MTTR. It is far more realistic to use MTBF as the MDT is so subjective and open to abuse.
IV) The Reliability Block Diagram of a Single UPS
Module
In Fig. 7 an RBD model of a single module UPS is shown, neglecting most of the ‘short-circuit’ failure modes of sub-assemblies.
As we are going to use the same base model in both the distributed and centralized architectures, the absolute results for MTBF (with and without bypass and for three example grid power quality MTBFs) are only to be used in comparison. It is worth noting that:
The grid quality hardly affects the module output
MTBF if the bypass is ignored
When the bypass is included, the module MTBF is dominated by the grid MTBF.
Grid Supply
250
0.001
> 100ms delta
RDB for generic IGBT/IGBT Transformer-free Static UPS
APB S/C failure rate 30 Grid
<10ms delta
150
0.001
Rectifier
150,000
12
249,58
0.02
Battery
250,000
24
Auto bypass static switch
500,000
12
Boost & Filter
200,000
12
185,704
11.14
2.60E+60
0.02
111,111
17.33
Inverter
250,000
12
106,554
11.51
76,923
15.69
149.96
0.00
C/B
250,000
6
74,711
9.86
58,824
13.41
S/C failure mode
Auto bypass static switch
1.50E+07
12
74,341
9.87
Critical Supply Reliability with varying grid power quality
Poor grid quality
100
0.010
h h
MTBF
MTTR
Without auto bypass
71,628
9.46
99.98680% h h
MTBF
MTTR
Availability
With auto bypass
262,991
0.011
99.999996% h h
MTBF
MTTR
Availability
Average grid quality
250
0.001
h h
MTBF
MTTR
Without auto bypass
74,711
9.86
99.98680% h h
MTBF
MTTR
Availability
With auto bypass
657,597
0.004
99.999999% h h
MTBF
MTTR
Availability
High grid quality
400
0.001
h h
MTBF
MTTR
Without auto bypass
75,523
9.97
99.98680% h h
MTBF
MTTR
Availability
With auto bypass
1,051.783
0.007
99.999999% h h
MTBF
MTTR
Availability
Figure 7. Reliability Block Diagram of a single UPS module.
10

It can also be demonstrated that the overall module output bus voltage MTBF is highly dependent upon the battery MTBF as well as the grid MTBF, which is somewhat intuitive as, clearly, the more reliable the source of energy (and with redundancy between battery and grid), the higher the UPS MTBF can potentially be.
The yellow shaded cells are those where the MTBF and
MTTR can be set in the original spreadsheet for each element and the smaller font number-sets below each sub-assembly are the numeric result of the previous assembly in the chain and the current one.
allowed in the operation. UPS OEMs publishing an MTBF number should always be able to state the base criteria for grid power quality, battery MTBF and utilization of bypass power used in their MTBF or availability calculation.
V) Parallel Redundancy and Connection
The diagram on Fig. 8 shows the RBD technique applied to a four module redundant UPS system with a single automatic bypass to the LV supply network.
For example the output Circuit Breaker (C/B) has an
MTBF of 250,000 h and MTTR of 6 h and when fed by power from the IGBT inverter with MTBF of 250,000 h and MTTR of 12 h the resultant power has an MTBF of
74,711 h and MTTR of 9.86 h.
Published component MTBF data, such as MIL-Std data, can be used to build up models for the individual elements such as the rectifier, etc., but often ‘in the order of’ estimations can be used as long as the assumptions are consistent and equally applied.
The models can be used for sensitivity analysis purposes to good effect.
To represent the chance of failure in one machine (for example a short circuit at the output), causing a failure in another, it is suggested that a series element is added to the calculation – an element whose MTBF is related to the MTBF of the module. By trial and error, it appears that a factor between 20 and 40 produces a result, which, whilst proving the parallel system, is far more reliable than a single module, moderates the enormous
MTBF numbers generated by straight arithmetic.
A study carried out in Europe between 1991 and 1997
(albeit very limited in scope by the very few failures that occurred in a population of only a few hundred systems) came to the conclusion that 25:1 to 30:1 was a reasonable estimate.
A consistent approach across competing systems will produce the desired ‘comparison’ on a level playing field – whilst the absolute value of the MTBF is of less importance.
It is important to note the addition on the output side of the RBD chance of ‘closed’ or ‘short-circuit’ failure of the static bypass switch. Normally, this element is not included as the incidence of the failure mode (the grid being connected, asynchronously, to the output bus under a control of component fault scenario such that any protection is not activated typically in less than 10 ms) is much lower than the ‘open circuit’ static switch failure rate. However, this element has an increasing influence in distributed bypass architecture, with a negative effect of very high modularity on the system
MTBF when distributed bypasses are applied.
The RBD shown in Fig.8 would be used as a subset of a single block on a power distribution system
RBD. It shows four UPS modules running on battery
(MTBF=58,824 h, MDT=13.41 h). The reason for this is due to the fact that we cannot utilize the supply network more than ‘once’ in the calculation. After each module, an element has been added (in series) for the paralleling control (MTBF=500,000 h, MDT=15 h) and the ‘parallel redundant combination’ has been calculated in the spreadsheet. There are several methods that can be used for this “one out of N” redundant situation, but the result is an extremely long MTBF.
The inclusion of the output circuit breaker in the RBD is not common but as it produces a ‘glass ceiling’ on the module MTBF (of any type of UPS regardless of technology or topology), it is a worthwhile addition when modeling for comparative purposes. Note that, including routine maintenance, the MTBF of nearly all circuit-breakers is published as 250,000 h so no UPS can exceed that theoretical upper limit. In the example above, the single UPS module without the circuitbreaker and without recourse to the raw grid power has a very typical MTBF of around 100,000 h.
However, by the time we have placed the four series elements which represent the chance of failure of each module affecting the group (e.g. a factor of 25 times the individual module MTBF), the result has reduced to a somewhat more realistic and understandable
MTBF=328,000 h (about 37 years).
It is worth noting that ‘the MTBF’ of a single module can vary widely (>10:1) depending upon the grid and battery MTBF and whether or not bypass operation is
Finally, on the output side of the RBD, we can see the effect of short-circuit failure of the bypass on the overall system MTBF (again using the factor of 25x for the incidence) reducing the MTBF by circa 2 percent.
11

10ms Grid
150
0.001
RDB of a (3+1) redundant distributed bypass system
Relative MTBF of S/C failure mode compared to O/C MTBF= 25
ABP 1
500,000
12.00
Bypass Path
149.96
0.004
Module
58,824
13.41
Module
58,824
13.41
Module
58,824
13.41
ABP N+1
500,000
12.00
P1/1
500,000
12
52,632
13.26
P2/1
500,000
12
// Group
1.38E+11
13.26
P1/4
1.32E+06
13.26
1,315.777
13.26
P3/1
500,000
12
P2/4
1.32E+06
13.26
657,892
13.26
P3/4
1.32E+06
13.26
438,595
13.26
P4/4
1.32E+06
13.26
328,947
13.26
Module
58,824
13.41
P4/1
500,000
12
3,717.726
0.004
P1/4 SC
1.25E+07
12.00
2,865.481
2.75
P2/4 SC
1.25E+07
12.00
2,331.102
4.48
P3/4 SC
1.25E+07
12.00
1,964.708
5.66
P4/4 SC
1.25E+07
12.00
1,697.846
6.52
MTBF // 3,717.726
MTTR //
Module Path
328,947
13.265
0.004
Critical Supply Reliability
With distributed auto bypass to grid
MTBF=
MTTR=
1,697.846
6.52
h h
Availability = 99.9996%
With centralized auto bypass to grid
MTBF=
MTTR=
2,865.481
2.75
h h
Availability 99.9999%
Figure 8. Reliability of a 3+1 system with both distributed and centralized bypass design.
We can compare the centralized and distributed architecture in both these examples.
In the case of distributed bypass each module has an internal automatic bypass, which avoids the requirement of having a common centralized bypass.
As you can see from the way the RBD has been constructed whilst, physically, the bypasses are in parallel, for reliability they are in series since a failure in any one will (for a short circuit failure) produce a fault on the critical bus, therefore distributed bypass technologies produce one common point of failure in every module with the result of multiple SPoFs.
12

We can go further and take the same principle of distributed bypasses to systems of, for example, 10 modules as shown in Fig. 9 below, demonstrating that the system MTBF depends upon the failure rate in the short-circuit condition of all automatic bypasses and thus a distributed architecture only serves to increase the potential number of points of failure in the system with a reduction in system MTBF.
RDB of a (9+1) redundant distributed bypass system
Relative MTBF of S/C failure mode compared to O/C MTBF= 25
10ms Grid
150
0.001
ABP 1
500,000
12.00
ABP N+1
500,000
12.00
Bypass Path
149.96
0.004
Module
58,824
13.41
P1/1
500,000
12
Module
58,824
13.41
P2/1
500,000
12
Module
58,824
13.41
P3/1
500,000
12
// Group
1.34E+11
13.26
P1/10
1.32E+06
13.26
1,315.696
13.28
P2/10
1.32E+06
13.26
657,871
13.28
P3/10
1.32E+06
13.26
438,586
13.28
P4/10
1.32E+06
13.26
328,942
13.28
P5/10
1.32E+06
13.26
263,154
13.28
P6/10
1.32E+06
13.26
219,296
13.28
P7/10
1.32E+06
13.26
187,968
13.28
P8/10
1.32E+06
13.26
164,472
13.28
P9/10
1.32E+06
13.26
146,198
13.28
P10/10
1.32E+06
13.26
131,578
13.27
MTBF // 1,487.128
MTTR // 0.004
Module
58,824
13.41
P4/1
500,000
12
1,487.128
0.004
P1/10 SC
1.25E+07
12.00
1,329.015
1.26
P2/10 SC
1.25E+07
12.00
1,201.292
2.31
P3/10 SC
1.25E+07
12.00
1,095.966
3.15
P4/10 SC
1.25E+07
12.00
1,007.621
3.57
P5/10 SC
1.25E+07
0.00
932,456
3.58
P6/10 SC
1.25E+07
0.00
887,727
3.33
P7/10 SC
1.25E+07
0.00
811,401
3.12
P8/10 SC
1.25E+07
0.00
781,941
2.93
P9/10 SC
1.25E+07
0.00
718,165
2.76
P10/10 SC
1.25E+07
0.00
679,146
2.51
Module
58,824
13.41
P5/1
500,000
12
Module
58,824
13.41
P6/1
500,000
12
Module
58,824
13.41
P7/1
500,000
12
Module P8/1
Critical Supply Reliability
With distributed auto bypass to grid
MTBF=
MTTR=
697,146
2.61
h h
Availability = 99.9996%
With centralised auto bypass to grid
MTBF=
MTTR=
1,329.015
1.28
h h
Availability = 99.9999%
Figure 9. Reliability of a 9 +1 system with both distributed and centralized bypass design.
13

If we use a considered and consistent approach to compare competing power systems then we should be able to demonstrate that one solution (when compared to the other) is theoretically ‘more reliable’, i.e. statistically less liable to fail. However, as in all (relatively simplistic) reliability calculations, two limitations always apply: routine maintenance is included and the human factors surrounding the system operation are excluded.
1) Maintenance
All calculated MTBF data assumes that the equipment is fully maintained in good working order and that the risks associated with the actual maintenance itself are excluded from the calculation. When considering long periods of continuous power supply to critical loads, it has to be accepted that items of plant have to be shut down for routine (preventative) maintenance or replacement. With MTBF targets of more than 10 years, and most critical plant needing, at least, an annual intervention, we need to be careful that the system design meets the demands of the user when it comes to shutdown strategy and risk.
MTBF even more. Without timely cell replacement, the single-string MTBF will be in the 8 - 9 years region of service life, but if the user installs some redundancy
(either in battery strings or UPS modules) and replaces the cells before the beginning of the eighth year, the
MTBF achieved will be much higher. How much higher is a subject for another white paper, but the solution lies in the introduction of a ‘factor for maintainability’ based upon cell quality (predictability), monitoring and capacity testing frequency in the last year of service. Results can be shown that for a 10 year design life battery, properly maintained and with a redundant string, an overall system MTBF of 250,000 h is in the right order. By varying the battery quality, temperature and testing regime, the battery MTBF can be seen to vary from 50,000 h to over 1,000,000 h. It should be stressed again that using consistent values for MTBF in comparing systems will, to a large extent, negate over
(or under) estimation in individual component MTBF values.
A good example of the care that should be taken with estimating MTBF if calculated or measured values are not available is that of batteries. Cells in a battery are a
‘wear item’, such that replacement when their capacity falls below 80 percent can be regarded as planned maintenance and there is therefore a divergence between actual life and MTBF based upon the service regime. For example, consider a battery constructed with cells with a manufacturer’s design life of 10 years that is engineered, installed and maintained in accordance with the OEMs instructions, including a temperature controlled environment.
2) Human Error
Many sources estimate that 60 - 70 percent of all data center ICT service interruptions are due to human error.
However, no designer can plan for every eventuality
(including an operator pressing the Emergency Power
Off - EPO - button) and it is often very difficult to make subjective comparisons between systems on operational issues. Simple steps taken with operator training, documentation, mimic diagrams, clear labeling and color-coded plant, etc., can pay dividends.
However, it must be accepted that as the system becomes more complex (to achieve higher levels of availability for example), the chance of operator error increases. Although it is difficult to put a finite value against this tradeoff between simplicity and performance, the designer must always be aware of the dangers of complexity.
After a constant life of float duty and the occasional discharge-recharge cycle, the failure mechanism will be a blend of internal corrosion and drying-out such that, in the last 3 - 6 months of service life, the capacity will fall from roughly 100 percent to less than the critical threshold of 80 percent. Each ‘failed’ cell will exhibit either a short-circuit failure, where the cell conducts current from terminal to terminal but adds no energy, or an open-circuit failure where the whole string will be compromised. In each string perhaps 5 percent of cells can fail short, but only one can fail open for the entire string to ‘fail’. The designer can, of course, influence the MTBF by redundant strings, etc., but the planned-maintenance regime can influence the system
For example, if we consider two identical systems each with a modest MTBF of one year and a MTTR/MDT of 2 hours – the MTBF (reflecting the chance of coincident failure when it only takes 2 hours to fix a fault in one element) is a staggering 2,190 years. So, is this realistic?
If the systems were truly independent of each other, in other words, there were no possible fault scenarios where one element could have an adverse effect on the other, and if the MDT was realistic, then, yes – the result would have meaning. Although it would still be
‘meaningless’ in the context of the expected lifespan of the equipment it would reflect a ‘more reliable’ system than, say, a system whose MTBF was half of that, where, one system was half as likely to fail as the other.
14

MV = Medium Voltage
LV = Low Voltage
RBD = Reliability Block Diagram
MSS = Main Static Switch
MDT = Mean Down Time
MTTR = Mean Time To Repair
MTBF = Mean Time Between Failures
SPoF = Single Point of Failure
15

About Emerson Network Power
Emerson Network Power, a business of Emerson (NYSE:EMR), is the world’s leading provider of critical infrastructure technologies and life cycle services for information and communications technology systems. With an expansive portfolio of intelligent, rapidly deployable hardware and software solutions for power, thermal and infrastructure management, Emerson Network Power enables efficient, highly-available networks.
Learn more at www.EmersonNetworkPower.eu
While every precaution has been taken to ensure accuracy and completeness herein,
Emerson assumes no responsibility, and disclaims all liability, for damages resulting from use of this information or for any errors or omissions.
Specifications subject to change without notice.
Locations
Emerson Network Power
Global Headquarters
1050 Dearborn Drive
P.O. Box 29186
Columbus, OH 43229, USA
T +1 614 8880246
Emerson Network Power
Europe Middle East And Africa
Via Fornace, 30
40023 Castel Guelfo (BO) Italy
Tel: +39 0542 632 111
Fax: +39 0542 632 120
ACpower.Networkpower.Emea@Emerson.com
Emerson Network Power
United Kingdom
George Curl Way
Southampton
SO18 2RY, UK
T +44 (0)23 8061 0311
F +44(0)23 8061 0852
Globe Park
Fourth Avenue
Marlow Bucks
SL7 1YG
Tel: +44 1628 403200
Fax: +44 1628 403203
UK.Enquiries@Emerson.com
EmersonNetworkPower.co.uk
Follow us on Social Media:
Emerson. Consider it Solved, Trinergy TM , Emerson Network Power and the Emerson Network Power logo are trademarks and service marks of Emerson Electric Co. or one of its affiliated companies.
©2015 Emerson Electric Co. All rights reserved.