Documenting and Managing
Infrastructure Connectivity
David Cuthbertson
Square Mile Systems Ltd
david.cuthbertson@squaremilesystems.com
www.squaremilesystems.com
Square Mile Background
• Develop toolsets, training and
techniques for operational
management of complex IT
infrastructure
• Focus areas
–
–
–
–
–
Data center management
Connectivity management
System change impact analysis
Documentation techniques
Infrastructure visualisation
• All technologies!
Business Processes
Departmental, Company
Services
End user, infrastructure, supplier
Applications
PC, server, mainframe, SOA
Virtual Infrastructure
Network, Servers, Storage, DBMS
Hardware Infrastructure
Network, Servers, UPS, Storage, Other
Fixed Infrastructure
(Cabling, Power, Cabinets, Buildings)
Data Center Infrastructure
Session overview
• Communicate practices and techniques to
colleagues and customers that will aid
better management of infrastructure
connectivity
• Recognise good and bad methods of
labeling devices and cabling
• Take into account operational
management needs when designing or
installing cabling infrastructures
Is it a Problem?
• Standards already cover cabling design,
installation and labelling
l
l
a
– TIA568A, ISO 11801, EN50173,
TIA942
r
o
f
e
s
e
th administration
• Standards already cover
?
s
w
n
o
l
o
l
i
t
o
f
a
t
y
– ANSI/TIA/EIA-606,
EN50174-1,
n
n
e
a
m
m
e
l
w
ISO/IEC14763-1,
TIA942,
BS6701:2004
p
o
m
i
H
g
n
i
l
• Connectivity
is more than just about data
cab
cabling!
– Power is becoming just as complex
Changing Requirements
BEFORE
AFTER
No. of Servers per cabinet
Power Dissipated per cab.
Current service to cabinet
Types of Equipment
3-6
300-2000W
16A
Servers
Monitor
KVMs
Power Strips
UPS
Network types
No. of Cables
Power
(per server)
Network
Cabinet Total
100M
1 or 2
1 or 2
20-30
30-40
3kW - 25kW
2x32 A or 3 phase
Blade Servers
Power Distribution Units
MidSpan Boxes
Disk Arrays (Storage)
Smart Power Strips
Regular Power Strips
1G, 10G, SAN
2 to 6
5 to 10
300 - 400
Where Do We Focus?
•
•
•
•
•
•
•
•
•
Earthing and bonding
Containment
Firestopping
Spaces (rooms, racks etc.)
Vertical wiring
Horizontal wiring
Power
IT equipment
Other equipment
Which is Easiest to Document?
Standards Recommendations
1. Class or Hierarchy Structure
Class 1 – Single equipment room
Class 2 – Multiple rooms
Class 3 – Campus
Class 4 – Multiple sites
Example TIA606
Naming conventions and approach will differ
with administration systems.
Standards Recommendations
2. Naming conventions and examples
Country, site, floor, room, rack, unit, sub-unit, port
1A-AC01/A-01
Floor
1
Equipment
Room
A
Rack
AC01
Patch
panel
A
Port
01
Standards Recommendations
3. Standardised Naming & Coding
Glossary of terms and abbreviations
Symbols for drawing
Termination point colour coding
Patch cable coding
Standards Recommendations
4. Recommended data sets to maintain
Port type
- RJ45
Cable type - Cat6A
Cable length - 65m
User name - Daves PC
Test results - U:\Cabletest\1Atest.xls
Drawings
- Floor, room drawings
Work flow
- Work orders and changes
Why Does It Happen?
1. Standards have been
referenced for the design,
implementation and testing
of infrastructure
2. Good components have
been chosen - which rarely
go wrong
3. Is it only a people issue?
How Do We Manage Today?
•
•
•
•
Informal / formal processes
Site survey, pre-installation checks, audits
Ownership is often on a local basis
Create knowledge sets as individuals or
within teams – Excel, Visio, Word, Notes,
Sharepoint, Access
• Or give the problem to someone else
– Outsource, out task.
Different Teams, Different Focus
Business Processes
Departmental, Company
Services
Systems
End user, infrastructure, supplier
Applications
Service
Management
Applications
PC, server, mainframe, SOA
Virtual Infrastructure
Mid-range
PCs, Network, Servers, Storage, DBMS
Servers
Networks
LAN/SAN
Hardware Infrastructure
PCs, Network, Servers, UPS, Storage, Other
Desktops
IMAC
Data
Centre
Fixed Infrastructure
(Cabling, Power, Racks, Rooms, Buildings)
Adding a New Server?
KVM
Architecture
Structured cabling only
LAN diagrams
Inventory list
Storage diagrams
KVM
Asset list
WAN diagrams
Building wiring
diagrams
IIS Architecture
Patching
spreadsheets
Rack
Diagrams
Power architecture
Computer room
layout
Circuit breakers
Point to Point Cabling
Edge switches
Backbone switches
Power distribution
Blade switches
PDUs
Labelling standards
PABX port mapping Legacy systems
LAN Architecture
SAN
Architecture
Power strip
connections
Different Views
FEATURE
LINK 10/100
FEATURE
LINK 10/100
LAN
LAN
SERIAL
SERIAL
CURRENT
CURRENT
ON = I OFF = U
BLINK = REMOTE
ON = I OFF = U
BLINK = REMOTE
Copper
OUTLET #
OUTLET #
I /U TOGGLE
I /U TOGGLE
Fibre
Firewall
Power
Managing change at
equipment level
requires different
views of connectivity!
Server
STATUS
100240V
9
10
11
12
13
14
15
16
1
2
3
4
5
6
7
8
Storage
RESERVED
~
50~
60Hz
1.2A
KVM
Or monitoring results
from power, network
or storage can’t be
interpreted
Switch
Is It Just Too Difficult?
For many organisations, the internal culture does
not enable more coordinated team practices
Why is it difficult?
Creating a start point or baseline is costly
Defining update processes
Will involve multiple technical teams
Disinterest by project teams
Connectivity Costs!
It is easy to use technical teams and suppliers inefficiently!
–
–
–
–
–
–
–
–
–
–
–
Travel to check on existing connectivity or site survey
Meetings to communicate local cabling architecture
Reconciling differences in systems documentation
Creating build and patch instructions
Drawing diagrams which become out of date and uncontrolled
Audit connectivity to check on resilient paths / SPOF
Tracing connectivity to help with fault diagnosis
Manually produce reports on numbers of changes / installs
Switch port tracing to identify free and pre-patched ports
Producing risk and test plans as part of projects
Risk and recovery tasks creating additional knowledge sets
What’s the upside?
1. Reduced operational expenditure
– Optimise existing infrastructure
Power
2. Reduced cost of changes
– Planning, implementing, reporting
Time
3. Faster implementation of changes
– Reducing planning and communication
– Forward planning and capacity control
Project
SLAs
Year
4. Reduced risk of disruption
– Impact analysis is easier, less mistakes
5. Less effort to keep accurate documentation
What Types of Documentation?
Specific path connectivity
–
End Points and paths
Definitions
–
Types of ports, addressing, naming, labelling
Diagrams
–
Views of physcial locations, physcial and logical paths
Capacity reports
–
–
Fixed infrastructure (patch panels, power strips, PDUs)
Active infrastructure devices (network switches, SAN, PBX)
Management & Workflow
–
Reservation, build instructions, audit trail, changes
Example - Cable Labeling
A
B
Patch
Panel
SW01
SW02
SW03
C
01
Patch
Panel
F
E
D
Patch
Panel
Patch
Panel
02
13
14
Patch
Panel
Patch
Panel
Patch
Panel
03
23
G
24
UX01
UX02
UX03
Cable Labeling Examples
Possible Options - Some
1. No Label
2. Port number at each end
(SW01/P01)
3. Local end points of cable
(SW01/P01 – PP01/01)
4. Device end points of cable
(SW01/P01-UX01/ETH0)
5. Unique label on each cable
(04567489)
6. Common path label on cable
(1123238)
Benefits
No administration requirements
Easy to ensure cable is in right port
Easy to trace ends of patch cable
Easy to know impact of disconnect
Easy for software tools to reference
Easy for workflow and end path tracing
Recommended Labeling
• Our recommendation is;
– Unique labels at both ends of a patch cable
• Why
– No re-labelling if devices change names
– Cables can be re-used, pre-patched
– Easy to audit recent changes
– Needed by software tools as a reference
• But you need to look for a software tool
that accepts (and can create) cable labels
Reducing The Amount of Data
Before - uncoordinated data
Consistent views & reports
Capacity & audit trails
Workflow and reservation
Excel
VisioWord
Word
Visio
Excel
Excel
Visio
Word
Visio
After – Less data sources
Visio
Word
Excel
Word
Excel
Visio
Word
Word
Word
Excel
Excel
Visio
Excel
Word
Reporting
Word
Visio
Visio
Excel
Excel
Excel
Excel
Define the Level of Detail
2. End to End path
1. Local patch
Patch
Panel
Patch
Panel
3. All devices
connected to the
switch
Patch
Panel
Patch
Panel
Patch
Panel
Example – Do This Yourself
Network
Assets/Inventory
G10-3
SW-BHAMG10-4
CORE2
SFP1
Excel
11
SW-BHAM-05
9
Visio
SW-BHAM-04
SW-BHAM-02
27
1
ETH 0
23
SFP1
15
ETH 0
ETH 0
ETH 2
ETH 0
ETH 2
UK_BIRM_UX06 UK_BIRM_UX07 UK_BIRM_UX04 UK_BIRM_UX08 UK_BIRM_UX10
Port Connections
Power
Excel
1. Draw diagram using Excel data
2. Refresh Visio for updates
More Sophisticated
1. Paths
Physical
Logical
Device
2. Diagrams
Physical
Logical
Multi-technology
3. Capacity
Fixed infrastructure
Active components
Power
4. Workflow
Reserve, design
Build instructions
Specialist DC
Management Toolset
Outputs
Data Feeds
Monitoring, Discovery, Test
Results, Project Plans, Other
It is wise to develop your own
requirements before choosing
any specialist toolset!
Steps to Success - Build
• Ensure hand over documentation reflects the
built environment
• Insist that format and content are consistent with
the standards or systems adopted
• Why not get suppliers to deliver the operational
processes as well as infrastructure data?
– They do it already for HVAC and power systems
• Any further works should result in updates to
existing documentation sets
Steps to Success - Operate
1. Create an inventory
- All components involved in connectivity
2. Document the fixed infrastructure
- Backbone, power, SAN
3. Record the connectivity
- Paths, ports, labels
4. Create reports and diagrams to suit the need
- Capacity, topology diagrams
5. Embed in project workflow
Are We Working Too Hard?
•
•
•
•
Reverse engineering existing systems
Producing different views of connectivity
Travelling and meeting unnecessarily
Coping with inconsistent information
• Be smart – work more as a team!
– Save on cost, time and effort
– Increase the level of control
Thank you for your attention
Questions or feedback?
David Cuthbertson
Square Mile Systems Ltd
www.squaremilesystems.com
www.assetgen.com