Three Case Studies of Commercial Deployment of
400V DC Data and Telecom Centers in the EMEA
Region
Sara Maly Lisy
Mirna Smrekar
Emerson Network Power
Warrenville, USA
Sara.Lisy@Emerson.com
Emerson Network Power
Zagreb, Croatia
Mirna.Smrekar@Emerson.com
copper cabling and installation costs of power distribution
infrastructure.
Abstract— Interest and adoption of 400V DC power has been
growing over the past decade. After many years of studies and
trials, commercial implementation of 400V DC power in
production telecommunications and data center sites is becoming
a reality. This adoption is driven by multiple motivators,
depending on the site-specific application and layout.
Because there is limited availability of 400VDC-powered
telecom equipment today, an attractive distribution architecture
is to distribute 400V DC power with long cable runs and
convert 400V DC to -48V DC near the -48VDC-powered
equipment loads. This allows users to achieve substantial cost
savings on distributing 400V DC over long cable runs, while
still feeding standard telecom equipment.
This paper will present case studies of the implementation of
400V DC power at three sites in Europe and Africa. It will
examine the decision making process behind 400V DC adoption
from the end users’ points of view, and detail the deployment
architectures.
This paper will review case studies of two data center
applications and one telecom application, detailing the different
400V DC adoption motivators and deployment architectures.
The first case study shows deployment of a small scale
400VDC data center in 2N concept powered solely by 400VDC.
400VDC power is used not only for IT equipment, but also for
cooling and lighting. Solar energy supplied from PV panels
through solar MPPT units is connected directly to the 400VDC
bus.
II. CASE STUDY 1
A. Background
Bachmann GmbH & Co. KG is a German-based
manufacturer of electronic components and systems. The
company has been focused on developing innovative power
distribution equipment to enable data center and industrial sites
to maximize energy efficiency. Bachmann was building a new
data center for its internal operations and wanted to design it as
a showcase of energy efficiency. 400V DC power was chosen
as the main power distribution architecture in order to
minimize the number of conversions between AC and DC
power and to enable the integration of renewable energy
sources that generate DC power.
The second case study describes the deployment of a
traditional data center with 2N concept. Both AC and 400VDC
architectures were built in parallel to allow for future modular
extension of either AC or 400VDC powered equipment
depending on future needs.
The third case study demonstrates how an existing -48VDC
large telecom site is powered by 400VDC distribution. 400V DC
power cabinets distribute power over long cables to 400VDC/48VDC conversion cabinets that are located near the -48VDC
loads.
B. Solution
To ensure maximum availability, the data center was
designed with a 2N configuration, with dual paths for power
conversion, distribution, and cooling, Fig. 1. At the site
entrance, an automatic transfer switch (ATS) connects both
utility grid power and a back-up generator. The output of the
ATS provides the 400VAC input to the two 400V DC power
systems. A 230VAC feed from the ATS also supplies power
to auxiliary AC powered loads, such as the security system
and electrical sockets.
I. INTRODUCTION
In data centers, the primary drivers for 400V DC power are
the simplicity of the power chain in comparison to AC power
and the ability to provide redundancy and scalability in a very
modular approach. By eliminating conversions back and forth
between AC and DC power, 400V DC has the potential to
decrease capital cost and footprint while increasing efficiency
and availability. In addition, the use of DC power simplifies
the integration of renewable energy sources.
Compared to -48VDC power architectures traditionally
used at telecom sites, 400V DC can significantly reduce the
978-1-4799-6582-3/15/$31.00 ©2015 IEEE
6
Fig. 1. Electrical drawing of Bachmann site.
Integration of renewable energy was a major driver of the
power architecture at this site. Bachmann engaged with the
European Union DC Components + Grid (DCC+G) microgrid
demonstration project to serve as a demonstration site for a
version of a 400V DC power system that was developed for
DCC+G [1]. Fig. 2 shows a system diagram.
Fig. 2. - Schematic of 400V DC power system with integrated rectifiers, solar
MPPT, and batteries.
This 400V DC power system contains both rectifiers that
convert utility AC power to 400V DC power, as well as solar
MPPTs that convert variable DC power from solar PV panels
to regulated 400V DC power. Both the rectifiers and solar
MPPTs are hot-swappable modules that can be integrated into
the same power system. The peak efficiency of the rectifiers is
97% and the peak efficiency of the solar MPPTs is 98%.
Bachmann utilizes two 400V DC power systems, one for each
side of the power distribution path, that are each populated with
one 15kW rectifier and one 15kW solar MPTT. The power
system cabinets also include two integrated battery racks each,
to provide approximately 10 minutes of battery backup. These
systems can be expanded with the installation of additional
rectifiers, MPPTs, and battery trays.
Solar panels are installed on the roof of the newly
constructed building, which was designed to optimize the
position of the solar panels relative to the sun, shown in Fig. 3.
There are two arrays containing 21 panels, capable of
producing up to 5250W. Each 400V DC power system is fed
by one solar panel array and by the output of the ATS.
Fig. 3. Solar panels on the roof of newly constructed Bachmann building.
The output of each 400V DC power system is connected to
a power distribution cabinet. These cabinets feed the circuits
for facility loads and connect to 400V DC power distribution
unit (PDU) power strips that were installed in IT racks.
Bachmann designed and manufactured the 400V DC PDUs,
shown in Fig. 4, which include connectors for power cables to
servers protected by ABB 400VDC-rated circuit breakers.
The PDUs power 400V DC servers, manufactured by HP, in
IT racks in the data center. To show versatility of 400V DC as
a main distribution voltage, some racks of AC-input servers
are also included in the site. These are powered by in-rack
400VDC-to-AC inverters from local manufacturer, Schaefer.
Fig. 4. 400V DC power distribution unit.
ETSI EN 301 605 recommends two potential grounding
schemes for 400V DC applications: TN-S and IT earthing
topologies [2]. The Bachmann installation utilizes TN-S
grounding, also referred to as negative pole grounding (NPG),
for the 400V DC power distribution. The Bachmann site
includes both 400V DC powering of data center and adjacent
office building equipment, and TN-S grounding is commonly
7
adopted by building installations because it can be less
complex than implementing IT grounding.
The data center is cooled by two redundant 16kW
400VDC-powered CRAC units from local supplier, Weiss
Klimatechnik. High efficiency LED lighting from Philips is
powered by 400V DC and installed in both the data center and
adjacent office building. Because LEDs are inherently
powered by direct current, the use of 400V DC for powering
eliminated a power conversion in the lighting ballast and
significantly reduced the size and complexity of components in
the ballast. The complete data center is shown in Fig. 5.
C. Results
The 400V DC data center and office microgrid have been
operational since March 2015, with reported uninterrupted
operation. The installation continues to serve the dual purposes
of supporting Bachmann’s operations and demonstrating a
complete miniaturized data center implementing the latest
innovative and energy efficient technologies. The German
Data Center Award gave Bachmann 3rd place prize for energy
efficiency at the Future-Thinking Data Center Conference in
Darmstadt in April 2015 [3].
Fig. 5. Bachmann data center. Left to right: 2 racks of AC servers with
inverters in bottom; rack of 400VDC-powered servers; IT cross connection
cabinet; 2 400V DC power systems. Overhead: 400VDC-powered LED lights.
Each side of the AC power distribution is protected by a
250kVA UPS system. Each side of the DC power distribution
includes a 120kW capacity 400V DC power system, also
known as a DC UPS, which converts AC power to 400V DC
power and provides backup via batteries, shown in Fig. 6 and
Fig. 7. The 400V DC power systems were initially populated
with three 15kW high efficiency rectifier modules each, and
can be expanded with up to eight rectifier modules per bay as
well as additional power bays for more capacity.
III. CASE STUDY 2
A. Background
Coromatic AB, a leading data center integration firm, was
contracted by a Scandinavian company to build a new data
center to consolidate its data center operations that had
previously been distributed across a number of international
sites. The company placed a high priority on energy efficiency
and asked Coromatic to design a cutting-edge, high efficiency
data center with high availability.
The company was inspired by the Green.ch data center in
Zurich, Switzerland, which implemented both AC power and
400V DC power in different sections of the data center.
Installed in 2012, that project demonstrated capital and energy
savings of 400V DC power distribution in a direct comparison
to the AC section [4]. The Scandinavian company decided on
a similar concept, with parallel AC and 400V DC
infrastructures within its data center. It views 400V DC power
as a green technology to enable energy efficiency via fewer
conversions between AC and DC power and ease of potential
future integration of renewable energy sources.
Fig. 6. 400V DC power system diagram.
B. Solution
Coromatic designed the site architecture with separate AC
and 400V DC architecture powering separate IT equipment
loads. Both the AC and 400V DC power paths were designed
with 2N redundancy. The site contains primary and secondary
switchgear. The primary switchgear accepts 400VAC input
from both the utility and backup generators, and feeds the
400V DC power systems, the secondary switchgear, and
auxiliary AC-powered equipment. The secondary switchgear
powers the site’s AC UPS systems and provides a maintenance
bypass path for the UPS.
Fig. 7. One of two 400V DC power systems initially installed in data center.
8
The 400V DC power systems are connected to strings of
(28) 12V batteries, for 336V nominal and 378V float bus
voltage.
Remote distribution cabinets provide power
distribution to circuits for both the AC and 400V DC power
integrated side by side. AC and 400V DC power are
distributed to IT racks, shown in Fig. 8., that contain an AC
power strip on one side and a 400V DC power strip on the
other. Both AC-input and 400VDC-input IT equipment may
be used in the same rack, although individual servers accept
either AC or 400V DC input, not both. The 400V DC
equipment loads include servers from both HP and Cisco, as
seen in Fig. 9. The cooling, lighting, and other auxiliary loads
are powered off of the AC infrastructure.
Fig. 9. Cisco 400VDC-powered server.
For the 400V DC distribution, the site implemented an IT
earthing topology, also referred to as high resistance mid-point
grounding (HRMG). This grounding scheme, which inserts
high impedance resistors between each pole and ground, is
recommended for data center applications to enhance
continuity of service [5]. In this approach, a single line-toground fault does not result in disruption of service and allows
equipment to continue operating while the fault is identified.
IV. CASE STUDY 3
A. Background
An African telecommunications provider has a large
network of telecom central offices which use primarily
-48VDC-powered equipment.
As the telecom operator
continues to expand its operations, however, the distribution
cabling and support infrastructure for -48V DC power has
proved problematic at times.
C. Results
Initial equipment loads include approximately 80kW of AC
power load and 30kW of 400VDC-powered loads. The AC
and 400V DC power paths are being monitored to assess their
respective power consumption, energy efficiency, and
performance. Near future expansion plans call for 120-240kW
of 400V DC powered loads and 300kW of AC-powered loads.
The data center plans for an eventual total capacity of 1.5MW.
The 400V DC power systems were chosen in part for their
ability to expand in increments from small sub-100kW loads to
much larger capacity. The company also plans to install up to
200kW of solar PV power at the data center site in the future.
The telecom operator wanted to replace aging, inefficient
-48V DC power systems and needed to expand the equipment
in one of its central offices in South Africa. The building was
set up with a power room with -48V DC power systems and
batteries on the ground floor and -48V DC networking
equipment on the floors above. Because of space constraints,
many of the -48V DC distribution cables from the power plants
to the remote distribution cabinets (RDCs) were run outside of
the building. Expanding the -48V DC distribution would have
been expensive and posed additional cable routing and support
infrastructure challenges. The operator wanted to use a
centralized power plant and batteries for ease of maintenance
and separation of batteries from critical networking equipment.
In order to upgrade this existing site, the telecom operator
made the decision to replace the -48V DC power systems with
a centralized 400V DC power system, and convert 400V DC to
-48V DC near the equipment loads. This would alleviate the
significant cable management issues existing in the -48V DC
distribution, reduce expansion costs, and allow the use of
existing equipment in their -48V DC core site.
B. Solution
The goal of this site was to minimize the copper required
for long cable runs between the power room and equipment
floors, while enabling the use of existing -48V DC powered
equipment and RDCs. An additional benefit is to future proof
for potential transition to 400VDC- powered loads.
The site currently supports approximately 80kW of load,
with 2N power distribution. Initially, two 120kW 400V DC
power systems will be installed in an A + B configuration to
support up to 2000A of -48V DC end loads. Each power
system is comprised of eight 15kW modular rectifiers. The
systems could be expanded further in the future with additional
Fig. 8. IT rack with AC power strip on one side and 400V DC power strip on
the other side.
9
power bays. Four 336V battery strings each made up of 28
12V cells will be connected to each power system to provide
up to four hours of back up time in the event of AC grid failure.
Both the 400V DC power systems and batteries will be located
on the second floor in the power room, shown in Fig. 10.
Each 400V DC power system contains a distribution
section with 400VDC-rated circuit breakers. The distribution
breakers will be connected to cabling that runs to
400VDC/-48VDC conversion systems located on the first floor
of the building. These distribution cable lengths vary up to 77
meters.
The 400VDC/-48VDC conversion systems will be located
on the same floor as the -48V DC equipment loads. This
enables the distribution of 400V DC power over long cable
runs while allowing the use of existing -48VDC-powered
equipment. The converter systems utilize 3.5kW converter
modules, with peak efficiency of 97%. Each converter system
has a maximum capacity of 105kW of -48V DC output per
bay. In the initial site design, two converter systems will be
populated with conversion capacity of 91kW and 56kW,
respectively. Although typically mounted in an open relay
rack, the 400VDC/-48VDC converter systems will be
integrated into data center racks in accordance with the user’s
preference, as shown in Fig. 11. -48V DC power will be
distributed from the conversion systems via bulk 600A outputs
that connect to -48V DC RDCs. These RDCs will distribute 48V DC to the -48V DC equipment loads via cabling.
Fig. 10. Telecom power room with A+B layout. Initial 400V DC power
systems in light blue; future expansion 400V DC power systems in pink;
battery strings in blue.
C. Results
As of this writing, the installation on site is pending. Prior
to making a decision, a comparison was performed to assess
the cost of equipment and installation of -48V DC power
distribution
versus
400V
DC
distribution
with
400VDC/-48VDC conversion. Previous studies have shown
the potential to decrease capital costs by 25% or more using
400V DC distribution to 400VDC/-48VDC conversion, but the
exact savings are highly dependent on site layout [6].
The telecom operator made calculations based on
preliminary estimates of a typical layout and included the cost
of power systems, batteries, conversion systems, cabling, and
installation. This assessment found an estimated 25% savings,
at which point the operator decided to move forward with
designing the site implementation. Once the specific site
layout was finalized, calculations showed that the 400V DC
solution reduced the required cabling from battery plants and
power systems to the RDCs to approximately 3,500 meters
from the 10,500 meters required for -48V DC power
distribution.
Fig. 11. 400VDC/-48VDC converter system mounted in data center rack,
preparing for installation.
The results of these commercial deployments demonstrate a
number of key aspects of 400V DC power. Installations were
completed with full safety certifications in Europe. No safety
issues were experienced in the deployment or operation of the
sites. The 400V DC infrastructure throughout the power chain
was assembled from commercially available 400V DC
equipment. The power distribution systems have demonstrated
stability, without typical AC power issues of harmonics or
phase/voltage balancing. All three sites have planned for
potential future expansion of the 400V DC power through the
use of modular, scalable 400V DC power systems. The first
case study demonstrated straightforward integration of solar
energy that was simpler than the use of an inverter with an AC
power distribution topology.
V. SUMMARY
The above case studies reviewed 400V DC power
implementation at three separate sites. Each case represented a
unique combination of adoption drivers and installation
architecture. Table 1 summarizes the different scenarios.
10
TABLE 1. SUMMARY OF CASE STUDY APPLICATIONS
End User
Typical Power
Type
Application
Primary 400VDC
Driver
Power Distribution
Grounding
Bachmann
Data center
AC Power
Elimination of
conversion stages;
integration of solar power
400V DC distribution to 400V
DC loads; AC loads powered
via inverters
TN-S
(negative pole
ground)
Scandinavian
company
Data Center
AC Power
Elimination of
conversion stages
Parallel power paths
distributing both 400V DC
and AC power
IT
(high resistance midpoint grounding)
African telecom
Telecom
-48V DC
Cable reduction: cost and
cable management
400V DC distribution to 48VDC conversion and loads
IT
(high resistance midpoint grounding)
REFERENCES
VI. CONCLUSION
This paper has detailed case studies of 400V DC power in a
variety of applications covering both data center and telecom
sites. In each application, 400V DC power delivers benefits
compared to traditional power architectures, with different
motivations for each site [7]. After years of trials, these
commercial deployments in Europe and Africa demonstrate
increasing maturity and acceptance of 400V DC power. As
400V DC compatible equipment availability continues to
expand, the pace of commercial deployments is expected to
accelerate [8].
ACKNOWLEDGMENT
The authors would like to thank their former colleague,
Marek Szpek, for his diligent work on all three 400V DC
implementations in this paper and for his years of dedication to
the study and advancement of 400V DC power.
11
[1]
B. Wunder, L. Ott, M. Szpek, U. Boeke, R. Weis, “Energy efficient DCgrids for commercial buildings,” IEEE/Intelec Vancouver 2014.
[2]
ETSI EN 301 605 – Environmental Engineering (EE); Earthing and
bonding of 400 VDC data and telecom (ICT) equipment.
[3]
http://www.future-thinking.de/startseite-drzp/preistraeger/
[4]
G. Ailee, W. Tschudi, “Edison Redux: 380 Vdc brings reliability and
efficiency to sustainable data centers”, IEEE Power and Energy
Magazine, 2012, Volume 10, Issue 6.
[5]
K. Hirose, T. Tanaka, T. Babasaki, S. Person, O. Foucault, B.
Sonnenberg, M. Szpek, “Grounding concept considerations and
recommendations for 400VDC distribution system,” IEEE/Intelec 2011.
[6]
S. M. Lisy, B.J. Sonnenberg, J. Dolan, “Case study of deployment of
400V DC power with 400V/-48VDC conversion,” IEEE/Intelec
Vancouver 2014.
[7]
M. Szpek, B.J. Sonnenberg, S. M. Lisy, “400VDC distribution
architectures for central offices and data centers,” IEEE/Intelec
Vancouver 2014.
[8]
D. Geary, D. Mohr, D. Owen, M. Salato, B.J. Sonnenber, “380V DC
eco-system development present status and future challenges,”
IEEE/Intelec Hamburg 2013.