BEGINNING AND INTERMEDIATE STEPS
ON PREPARING BLACKBOARD SYSTEM CONNECTIVITY
FOR AUTOMATED REDUNDANCY/FAIL-OVER
CAPABILITY
STRATEGY, IMPLEMENTATION, DOCUMENTATION
ABOUT THIS PRESENTATION
This session will show you what GW has done to prepare
its connectivity for redundancy and fail-over. Much of what
we discuss can be applied at schools looking for more
robust and modularized connectivity but who are not
planning on implementing a fail-over strategy soon. We
will go over different versions, permutations and reasoning
behind our current setup as many of these can offer useful
insight for the wide range of circumstances that Bb
schools represent. This session is more or less a sequel
to the fail-over server presentation given by GW last year.
The George Washington University
The “one-card” system had been in place for seven years with
a large existing infrastructure, multiple privileges & accounts
and approximately 1000 devices including IPCs, terminal
servers
Security 379
Activity 43
POS68
Vending 57
MDTs120
Laundry 51
VTS 8
Copiers/Microfiche 48
Interfaces: Follett JDA Bookstore POS, Pharos Uniprint, MDT,
Catapult POS, Campus Video Solutions TIA
Blackboard Transaction System –UNIX Edition,
OUR SERVER
Just kidding!
Why should you work on your
connectivity.
 Money
 Money
 Money
 Face
Money Money Money
How much does your system
take-in in one day.
Look at it backwards. What is real cost of
precautions in terms of daily revenue.
What happens to that money if system is downit doesn’t get spent through you.
Liability and safety concerns
Face
Imagine how you will look if you haven’t been
pushing for upgrades, redundancy, stable and reroutable communications.
Imagine how your boss will look.
Do you want to be the one that didn’t seem to be
concerned?
DIVERSITY OF LEGACY
COMMUNICATIONS
AND THEIR PROBLEMS
DIVERSITY OF LEGACY COMMUNICATIONS
AND THEIR PROBLEMS








Terminal Servers
Line Drivers
Dry pairs
Modems
RS 485
RS 232
Serial FDIs (BSI)
Private
connections
DIVERSITY OF LEGACY COMMUNICATIONS AND THEIR
PROBLEMS











Special connectors- inventory and repair procedures
Proprietary devices – inventory and repair cycling
Multiple service cost models –
IP
dedicated unconditioned house pairs (dry
pairs)
Dedicated unconditioned Phone Co. pairs
(dry
LADA circuits)
Analog house pairs (dial tone).
Analog Phone Co. pairs (dial tone).
Documentation/accounting difficulties.
Specialized Technical training/familiarization
Lots of single points of failure.
THE QUICK
DISCONNECT
THE QUICK DISCONNECT
This was our first step to making the
system more modular. We had
bought a second server and it
was configured. Data migration,
patching, database update
intervals were big issues but not
the focus here.
We realized that even though we
had plans to cut down the times
and uncertainty of these other
issues, to swap out our server,
even if we put it in the same
place would take us a long time
of plugging and unplugging.
The potential for mistakes was big.
THE QUICK DISCONNECT
QUICK DISCONNECT CABLE TO INFRASTRUCTURE
LOCAL COPPER
F I B E R
- -----25 PR.---COPPER
(AMPHENOL)
ANALOG
DATACARD
FIREWALL
NET SWITCH
UNIPRINT
UNCOND.
PERMANENT INTERNAL CABINET INTERCONNECT
WIRING
So we came up with this plan of consolidating all of the outside
connections onto one point.
In order to do this we had to add something that is the bane of any
technicians existence. We had to add extra connection points.
Normally this would have been a really bad thing, but in this case, it
cleaned up all of our wiring and made out switchover time really
short -- 10 minutes.
THE QUICK DISCONNECT
QUICK CONNECTION WIRING DIAGRAM
Connection mounting board inside of Server 1and Server 2 cabinets
QUICK DISCONNECT CABLE TO INFRASTRUCTURE
LOCAL COPPER
F I B E R
- -----25 PR.---COPPER
(AMPHENOL)
ANALOG
DATACARD
FIREWALL
NET SWITCH
UNIPRINT
UNCOND.
1
2
3
4
5
6
7
8
9
ANALOG CONNECTION BLOCK
DRY CIRCUIT CONNECTION BLOCK
MEDIA CONVERTER
MEDIA CONVERTER
DATACARD JACK
NETWORK SWITCH JACK
UNIPRINT GATEWAY (LOCAL-SERIAL)
SERVER1 FIREWALL CONNECTION
SERVER2 FIREWALL CONNECTION
ANALOG CONNECTION BLOCK (RJ12)
1-8 MDT MODEM
9 BLACKBOARD SUPPORT - SERVER1
10 BLACKBOARD SUPPORT - SERVER2
11 TEST MDT
12-13 OCP MODEM
PERMANENT INTERNAL CABINET INTERCONNECT
WIRING
DRY CIRCUIT CONNECTION BLOCK (RJ12)
1-4 GWORLD LOOP LAD CIRCUITS
5 BSI
6 TEST LAD
THE QUICK DISCONNECT
WHAT’S SO GREAT ABOUT
IP?...
AND WHY SHOULD I PAY FOR IT?
WHAT’S SO GREAT ABOUT IP AND WHY SHOULD I PAY FOR IT.
Your networking people will
understand it.
Your connections can be
repaired and replaced more
easily
Many of your circuit paths may
already exist.
Better reporting by Bb IP
devices and network devices
You may be able to share costs
with other departments for
Techs, equipment and circuit
paths.
WHAT’S SO GREAT ABOUT IP AND WHY SHOULD I PAY FOR IT.
Fewer service cost models.
You can incorporate or help
develop universally applied
documentation.
Potential for redundancy and
incorporation into
Networking’s overall plans.
Potential for sharing exclusive
connections with other vital
services.
Get yourself in line with where
most of the future
development is going.
It’s all going that way anyway.
WHAT’S SO GREAT ABOUT IP AND WHY SHOULD I PAY FOR IT.
Universal edition is coming to a Campus near you.
One of the great features of the coming Universal edition is the fact
that you will no longer need an NP. The NP is an expensive,
proprietary, hard to configure and connect, potential single point
of failure. Bb, I’m sure, and GWU in any case will be working
on redundancy and failover strategies for the Universal Edition’s
new NP configuration. Dependency on serial communications
can prevent you from being able to take advantage of some of
these features.
WHAT’S SO GREAT ABOUT IP AND WHY SHOULD I PAY FOR IT.
Incentives- Methods of attack





Plans can coincide w/future upgrades of system
Propose and if needs be – let it get shot down
Mention other systems may be able to take
advantage – get allies.
IP -Its all gonna go that way anyway.
Maybe you can start a wider initiative and make
yourself look really good.
CONVERSION TO
IPCONVERTERS
CONVERSION TO IPCONVERTERS
BJK 09/29/01
QUICK SWITCH GWORLD IN-BUILDING CONNECTION SCHEME (GENERAL)
LAUNDRY
CAMPUS BUILDING
GWORLD HUB
COPIER
OTHER
485 CONV.
QUICK SWITCH INVOLVES
REMOVING 485
CONVERTER’S RJ45 PLUGS
FROM JACKS FROM AND
INSERTING PLUGS FROM IP
CONVERTER
LINE DRIVER
RJ45
JACK
RJ45
JACK
REMOTE IP CONV.
IP
RJ45
JACK
SECURITY
SECURITY
SECURITY
RJ45
JACK
RJ45
485
BLDG
NET
PATCH CABLES
MEDIA CONV.
FIBER LINE TO CAMPUS FIBER NET
CONVERSION TO IPCONVERTERS
For conversion of our serial, RS485 circuits over to IP connections
using IP Converters, we used a staged approach where, with a
very short down time, we first converted the existing connections
to be modular . We created some inexpensive intermediate
connector assemblies that allowed us to continue running over
the old connectivity system until the pieces for the IP connectivity
were in place. Then, with another very short downtime, we
switched over. The hidden beauty of this conversion method is
that if for any reason you have a problem with the switch over to
IPConverters, you can fall back to your original hardware in
seconds.
CONVERSION TO IPCONVERTERS
GWORLD REMOTE HUBSITE RJ45 INSTALLATION
(PRE-CONVERSION)
485 CONVERTER’S SINGLE TB10 CAN HAVE MULTIPLE
DEVICE CABLES CONNECTED.
TERMINALS 1-5 AND 6-10 ARE DUPLICATE SETS.
STANDARD WIRING
TB10 PIN
1/6
2/7
3/8
4/ 9
5 /10
CAT 5 COLOR
WH/OR
OR/WH
GRN/WH
WH/BU
BU/WH
BELDEN COLOR
WHITE
GREEN
SHEILD
RED
BLACK
DEVICE CABLES
FUNCTION
RR+
SHEILD
TT+
10
4
9
3
8
2
7
6
5
1
485 CONVERTER’S TB10
CURRENT RS485 CONVERTER HOOK UP
CONVERSION TO IPCONVERTERS
Create intermediate
cable assemblies.
TB10 PIN
1/6
2/7
3/8
4/9
5 /10
CAT 5 WIRE COLOR
WH/OR
OR/WH
GRN/WH
WH/BU
BU/WH
FUNCTION
RR+
SHEILD
TT+
RJ45
PLUGS
10
4
9
3
8
2
7
6
1
485 CONVERTER’S TB10
5
CONVERSION TO IPCONVERTERS
Make it modular.
RJ45 jacks should be set as uniformly as
possible (preferably in rows) and
separated by loop and marked for loop
number and address(s). Wires can be
either cat 5 or Belden type. Color codes
will have two standards.485 converter
pins can hold more than one wire as
depicted, RJ45 jacks should not (one
cable per RJ45 jack).
RJ45 PIN
1
2
4
5
6
3*
CAT 5
WH/OR
OR/WH
BU/WH
WH/BU
GRN/WH
N/A
BELDEN FUNCTION
WHITE
RGREEN R+
BLACK
T+
RED
TSHEILD SHEILD
N/A
N/A
DEVICE CABLES
RJ45 SCREW
TERNINAL
JACK
RJ45 SCREW
TERNINAL
JACK
1
8
2
7
3
6
4
5
CONVERSION TO IPCONVERTERS
BJK 09/29/01
QUICK SWITCH GWORLD IN-BUILDING CONNECTION SCHEME (GENERAL)
LAUNDRY
CAMPUS BUILDING
GWORLD HUB
COPIER
OTHER
485 CONV.
QUICK SWITCH INVOLVES
REMOVING 485
CONVERTER’S RJ45 PLUGS
FROM JACKS FROM AND
INSERTING PLUGS FROM IP
CONVERTER
LINE DRIVER
RJ45
JACK
RJ45
JACK
REMOTE IP CONV.
IP
RJ45
JACK
SECURITY
SECURITY
SECURITY
RJ45
JACK
RJ45
485
BLDG
NET
PATCH CABLES
MEDIA CONV.
FIBER LINE TO CAMPUS FIBER NET
The Meshed Network
The Meshed Network
The Meshed Network




Work around power outages and connectivity problems
Can be made to be self-re-routable
Allows relatively easy connectivity for location changes of
components of you system (server, carding, administrations
stations)
Network metrics, monitoring, balancing
IP routers in a meshed network
IP routers in a meshed network
The Meshed Network
With repeaters for common leg runs
The Meshed Network
With repeaters for common leg runs
The Meshed Network
With repeaters for common leg runs
IP Router switches with repeaters for common leg runs
Easy conversion process
CURRENT CONFIGURATION
Loop 1
COMMON HUB
OTHER HOPS
BALONEY HALL
SWITCH ROOM
Loop 2
EXISTING
SWITCH
Loop 3
Loop 4
FIRST STEP
Use one existing loop
connection that
has a common
path as the trunk
for the repeater.
Loop 1
COMMON HUB
OTHER HOPS
BALONEY HALL
SWITCH ROOM
Loop 2
EXISTING
SWITCH
Loop 3
REPEATER
Loop 4
FINAL CONFIGURATION
Loop 1
COMMON HUB
Loop 2
REPEATER
OTHER HOPS
BALONEY HALL
SWITCH ROOM
EXISTING
SWITCH
Loop 3
Loop 4
IP Router switches with repeaters for common leg runs
GWORLD MESHED NETWORK
REPEATER INSTALLATION STEP
DRAFT
BJK 12/07/05
1 Assign VLAN jurisdictions to switch/router and repeater locations.
2 Create schedule of IPC IP address changes that will conform to VLAN structure and submit to
GWorld Applications Administrators.
3 Create schedule of loops whose fiber cabling will be re-used for repeater connectivity. (See
attached drawings)
4 Submit list of backup and normal power requirements and their locations to Ron Schreiber for
approval and comment.
5 Install repeaters in racks of respective buildings.
6 Submit conversion schedule to GWorld Applications Administrators for approval.
7 Convert loops:

Document and confirm repeater fiber path for one repeater and stage parts for conversion.

Disconnect fiber of loop to be re-used, attach repeater, re-attach loop and check for proper operation.

Patch in remaining loops to be connected through repeater and check.

Remove all parts of obsoleted loop infrastructure.

Submit documentation of current and retired infrastructure to Ron Schreiber.
SYSTEM/BUSINESS CONTINUANCE
DOCUMENTATION
STRATEGIES AND PITFALLS
SYSTEM/BUSINESS CONTINUANCE DOCUMENTATION
STRATEGIES AND PITFALLS
When adding, giving back or
fixing circuits,
documentation can save
you time and money.
 Accounting
 Work description
 Diagnostic aid
SYSTEM/BUSINESS CONTINUANCE DOCUMENTATION
STRATEGIES AND PITFALLS
For disaster recovery,
documentation could
make a huge difference
in the time it takes to get
your system back into a
working condition
SYSTEM/BUSINESS CONTINUANCE DOCUMENTATION
STRATEGIES AND PITFALLS
GWORLD MESHED NETWORK
REPEATER INSTALLATION STEP
For justifying decisions or
defending decisions,
documentation can really make
stating your case a lot easier.
DRAFT
BJK 12/07/05
1 Assign VLAN jurisdictions to switch/router and repeater locations.
2 Create schedule of IPC IP address changes that will conform to VLAN structure and submit to
GWorld Applications Administrators.
3 Create schedule of loops whose fiber cabling will be re-used for repeater connectivity. (See
attached drawings)
4 Submit list of backup and normal power requirements and their locations to Ron Schreiber for
approval and comment.
5 Install repeaters in racks of respective buildings.
6 Submit conversion schedule to GWorld Applications Administrators for approval.
7 Convert loops:
 Document and confirm repeater fiber path for one repeater and stage parts for conversion.
 Disconnect fiber of loop to be re-used, attach repeater, re-attach loop and check for proper operation.
 Patch in remaining loops to be connected through repeater and check.
 Remove all parts of obsoleted loop infrastructure.
 Submit documentation of current and retired infrastructure to Ron Schreiber.
SYSTEM/BUSINESS CONTINUANCE DOCUMENTATION
STRATEGIES AND PITFALLS
DISADVANTAGES
•Time consuming to create and maintain
SYSTEM/BUSINESS CONTINUANCE DOCUMENTATION
STRATEGIES AND PITFALLS
Three types of drawing
documentation used by GW
MSWORD
 Drawings very portable to just
about anyone.
 Not widely used for drawing.
 Easy to use but somewhat
methods are somewhat
arcane.
 Exportable into Visio (some
objects translate better than
others).
 Moderate learning curve.
SYSTEM/BUSINESS CONTINUANCE DOCUMENTATION
STRATEGIES AND PITFALLS
Three types of drawing
documentation used by GW
MSVISIO
 Requires that you purchase
Visio
 Not everyone has it (not as
portable)
 Focus of software is drawing
(more features).
 Moderate learning curve.
SYSTEM/BUSINESS CONTINUANCE DOCUMENTATION
STRATEGIES AND PITFALLS
Three types of drawing
documentation used by GW
AUTOCAD
 Drawings portable only to
other Autocad
users or through viewers
with varying editing features.
 Very expensive
 Big learning curve.
 Used by many
Architecture/engineering
firms
SUMMARY
SUMMARY
A few years ago we had a
system that
 could not be moved
 had all sorts of special
connections
 was understood by very
few people
SUMMARY
These steps we have taken
have made it so that, with
a little preparation, we
can put critical system
devices in a number of
places.
SUMMARY
GWORLD MESHED NETWORK
REPEATER INSTALLATION STEP
Documentation has given us
recognition as a team
that can play well with
others.
DRAFT
BJK 12/07/05
1 Assign VLAN jurisdictions to switch/router and repeater locations.
2 Create schedule of IPC IP address changes that will conform to VLAN structure and submit to
GWorld Applications Administrators.
3 Create schedule of loops whose fiber cabling will be re-used for repeater connectivity. (See
attached drawings)
4 Submit list of backup and normal power requirements and their locations to Ron Schreiber for
approval and comment.
5 Install repeaters in racks of respective buildings.
6 Submit conversion schedule to GWorld Applications Administrators for approval.
7 Convert loops:
 Document and confirm repeater fiber path for one repeater and stage parts for conversion.
 Disconnect fiber of loop to be re-used, attach repeater, re-attach loop and check for proper operation.
 Patch in remaining loops to be connected through repeater and check.
 Remove all parts of obsoleted loop infrastructure.
 Submit documentation of current and retired infrastructure to Ron Schreiber.
SUMMARY
We cut down on down time
for several kinds of
transitions.
QUICK DISCONNECT CABLE TO INFRASTRUCTURE
LOCAL COPPER
F I B E R
- -----25 PR.---COPPER
(AMPHENOL)
ANALOG
DATACARD
FIREWALL
NET SWITCH
UNIPRINT
UNCOND.
SUMMARY
We are poised to take
advantage of many of the
features that will come
with the Universal
Edition.
NP
END
FUNNY CLOTHES
GWORLD ASST.
DIR.
JAMES WHITE
GWORLD DIR.
KEN
PIMENTEL
Thank You!
Suzanne Werner
202-994-1297
werner@gwu.edu
Bio: Has been working at The George Washington University for three years as the Senior Applications
Administrator for the GWorld Card program. She has a BA in Computer Science from Rutgers University and
has over 8+ years experience working with various Unix systems. She is a New Jersey native with a passion for
writing and playing her own music and has traveled across the country playing in several bands.
Steve Nichols
202-994-0421
snichols@gwu.edu
Bio: Has been working at The George Washington University for three years as an Applications Administrator for
the GWorld Card program. An alumnus of the Virginia Military Institute, he has over 8+ years working in various
IT fields for both the government and private sector. A Virginia native, he enjoys the outdoors and is a die-hard
Washington Redskins fan.
Brendan Kelly
202-994-1711
bkelly@gwu.edu
Bio: Has been working at The George Washington University for seven years in various technical positions for
the GWorld Card program. He attended Northeastern University’s Electrical Engineering program and has over
20 years experience working with various device control and computer systems. He is a Connecticut native and
spent 15 years in and around Boston. He enjoys skiing and starting projects.