533582178
E Powers
El Chivato Public Service Company
Backbone Gigabit Ethernet Proposal
By
Eric Powers
Term Project
EETS-8391 Gigabit Ethernet
Summer 2002
Page 1 of 6
533582178
E Powers
Problem Statement
El Chivato Public Service (CPS) has it headquarters located in downtown Albuquerque, New Mexico. The
company’s Valley Service Center is located about three miles north in the same city, while its Energy
Management Systems (EMS) dispatch center for the entire statewide power grid is located about three
miles south of the downtown Albuquerque Headquarters. CPS’s generating capacity is located in
Farmington, New Mexico. Other field offices are located around the state in Bernalillo, Santa Fe, Las
Vegas, Deming and Belen.
The company has its own fiber infrastructure, with excess dark fiber capacity, connecting the majority of
these sites, installed along its powerline right-of-ways. Because of the small incremental cost off adding
additional fibers at the time of installation, excess fiber capacity was installed, with the thought of possibly
reselling the excess capacity. The fiber was originally installed to support voice telephone traffic, video
teleconferencing and to provide reliable communications channels for system-related telemetry and control.
The fiber initially helped to eliminate the toll charges for the inter-office telephone traffic between these
disparate sites. The fiber routes are mostly provisioned as DS-3s, with some older technology telemetry
and relaying equipment, which uses dedicated fibers.
More and more, the company’s key systems, including the power dispatch system, are migrating to network
based applications. The company’s data traffic load has increased significantly in the past ten years. The
company has contracted with a consulting firm to upgrade and consolidate their data communications
systems into an integrated IP based network. In the process, the company would like to incorporate some
form of redundancy also.
Objective
To design a Gigabit Ethernet Based MAN in the greater Albuquerque area with WAN broadband optical
links connecting the more distant locations. This new network is to leverage CPS’s existing fiber
infrastructure where possible, and utilize leased links as necessary to meet the stated goals. The final
design should include consideration for quality of service (QoS) for future voice traffic and video
teleconferencing, plus redundancy to enhance reliability.
Page 2 of 6
533582178
E Powers
Functional Requirements
Applications supported over this Intranet will include email, file transfer, computer aided drafting (CAD),
Internet access, and company Web-based applications supporting time reporting, billing and documentation
systems. Also, CPS is upgrading to a new Energy Management System that is designed to utilize an IP
network. Typical prior generation systems utilized dedicated circuits. This system monitors, manages and
controls the statewide electrical power grid.
The company has a need for video teleconferencing, especially between the Albuquerque Headquarters
location and the remote Farmington location, thereby minimizing the need to travel the two hundred miles
between the two sites.
VOIP was considered for this upgrade. It was determined that because of the company’s fiber
infrastructure and installed DS-3 equipment, voice could be left on the company owned and provisioned
fiber circuits. Capacity will be freed when the inverse multiplexed data traffic is moved over to the new IP
network. Eventually, however, the company does plan to convert the voice traffic, r atleast portions of it,
over to VOIP, so equipment installed in this upgrade should be capable of supporting VOIP.
Performance Requirements
To ensure acceptable Class of Service (CoS) transmission over the packet switched network, the network
backbone will be based on Gigabit Ethernet and the switches and routers will be appropriately sized to keep
peak bandwidth utilization under 60%. The edge of the network will feature 100 Mbps (Fast Ethernet)
equipment, much of it already existing. This approach will allow backward compatibility between the
WAN connections and the existing, installed base of 10BASE-T and 100BASE-T equipment.
The Gigabit channels that comprise the new backbone provide a significant channel bandwidth increase on
the links connecting the remote sites. This capacity, given the current traffic patterns, should more than
adequately provide for the Internet Protocol (IP) teleconferencing proposed between Headquarters and
Farmington.
As can be seen from Figure 1, this is a statewide network, with the longest Gigabit Ethernet link at 200
miles.
Page 3 of 6
533582178
E Powers
Management Requirements
CPS has an existing staff used to install and maintain its statewide microwave and fiber network. The
addition of the Gigabit Ethernet based Intranet will require cooperation with the IT staff, but no more than
what already occurs with the existing system, since there is some inverse multiplexing of the data network
over the DS-3 systems already. The biggest impact for this crew will be the new Inline Optical Amplifiers
situated along the Albuquerque to Farmington route. This represents new remote “sites” for future
maintenance and repair calls. The Inline Optical Amplifiers will be installed at appropriate locations along
the 200-mile (321 kilometer) Albuquerque to Farmington link.
Information Technology (IT) staff exists which is dedicated to maintaining the company’s historically
separate and discrete data networks. In addition to a large IT staff contingent located in the Albuquerque
area, there are also dedicated IT personnel assigned at each of the remote locations. This staff already
provides on-site support 8am-5pm Monday through Friday, with on-call response after hours on a 24hours
a day, 7 days per week basis. This staff can assume the additional responsibilities associated with this
Gigabit Ethernet upgrade with some additional training on the new routers. The Gigabit Ethernet portion of
the network follows basically the same management principles as the existing 100BASE-T and 10BASE-T
networks. The network management system will be expanded to monitor and control the configuration and
reconfiguration of the MAN and WAN.
Equipment
The major sites, including the remote Farmington site, will be linked with Gigabit Ethernet over the
company’s existing excess single mode fiber. All links will utilize two single mode fibers, one for each
direction. The switches will be full duplex with flow control capability. VLAN capability will also be a
requirement, even though CPS is not initially using that feature. Switches shall be capable of link
aggregation and load balancing to allow for future alternatives. The new switches shall be non-blocking
and feature hot-swappable multi-rate Network Interface Cards (NICs). All switches and critical file
servers shall be equipped with uninterruptible power supplies and/or back up generators for increased
availability.
At three miles, the Service Center link is the only link within the five-kilometer limitation for 1000BASELX. The other links will require Gigabit Ethernet Link Extenders, which extend that distance to 120
kilometers. The Farmington link, at 200 miles (322 kilometers), will require several Inline Optical
Amplifiers appropriately spaced along the route as determined by optical link budgets.
Page 4 of 6
533582178
E Powers
Block Diagram
Santa Fe
Farmington
Campus Distributor
100/1000 Mbps
Routing-switch
All Fiber Links are
Full Duplex
Gigabit Ethernet
Single Mode Fiber
Campus Distributor
100/1000 Mbps
Bernalillo
Campus Distributor
Routing-switch
100/1000 Mbps
GELE
Routing-switch
GELE = Gigabit
Ethernet Link
Extender
GELE
GELE
IOA
Valley Service Center
Inline Optical
Amplifiers
200 miles
Campus Distributor
100/1000 Mbps
15 miles
IOA
GELE
Routing-switch
60 miles
GELE
GELE
Typical for Las Vegas,
Deming and Belen
3 miles
Campus Distributor
ROUTER
Variable, all greater than 100 miles
Campus
Distribution
Router
Albuquerque
Headquarters
EMS
3 miles
Leased T1 Circuits
Campus Distributor
100/1000 Mbps
Routing-switch
Figure 1. Proposed CPS Gigabit Ethernet Wide Area Network
All links are Gigabit Ethernet, full duplex over dedicated single mode fiber. The fiber links
between the Switches and the Gigabit Ethernet Link Extenders (GELEs) are 1000BASE-LX.
Page 5 of 6
533582178
E Powers
Future Migration Plan
The network is being installed with several features that will not be utilized immediately, but will be
available for future applications, thus facilitating future migrations. The Gigabit Ethernet links are
sufficient for today’s known traffic loads plus some growth.
The router selected for use at Headquarters supports Quality of Service (QoS). This feature will help at that
time which it is deemed economically and technologically feasible to transfer all or a portion of the voice
telephone traffic over to voice over IP (VOIP). QoS allows prioritization of packets based on the
application. Thus, higher priority may be assigned to such latency sensitive applications.
All new equipment will also have VLAN capability. Even though priority frame transfer is not planned
immediately, this capability will assist in the future, both in adds, moves and changes and in assisting
routing protocols.
As the company replaces, upgrades or even install new power transmission lines, fiber too will be installed
along these routes, adding new network topology possibilities. Eventually, the leased T-1 circuits to Las
Vegas, Belen and Deming will be replaced with Gigabit Ethernet over fiber links. A new line, currently
under environmental review, will link Farmington to Santa Fe. This new fiber optic link will allow a
looping of the northern New Mexico system, thus enhancing reliability by providing alternate routing in
case of single point of failure.
Page 6 of 6