CASE STUDY Scalable Key & Cer6ficate Lifecycle Management within Cisco Systems Alex Wight
Cisco Systems
Session ID: SPO1-303
Session Classification: Intermediate
1 Insert presenter logo here on slide master. See hidden slide 2 for direc6ons Agenda
§  Why the need?
§  Who does this impact?
§  How to fix it?
§  Scoping
§  Analysis & requirements gathering
§  Solution finding
§  Implementation
§  Where are we now?
§  Apply
2 Insert presenter logo here on slide master. See hidden slide 2 for direc6ons Why the need?
§  Certificate expirations caused noteworthy outages
• Sales quoting tool
• Document repository system
§  High cost of manual certificate management (more
on this…)
§  Compliance with corporate security policy
§  Detection mechanism for policy compliance (more on
this…)
3 Insert presenter logo here on slide master. See hidden slide 2 for direc6ons Why the need? (cont.)
Total Cost is High
$1.4M
Total
Cost
$144K
$288K
High Mgmt
Costs per Cert
•  4 hours/cert
•  $288/cert at
Cisco
500
1000
5000
Number of Certificates
Manual Mgmt =
Human Error
•  Security Risk
•  Operational Risk
Cost of Downtime
•  Hard to quantify
•  Always costly
(especially if you
make the news)
4 Insert presenter logo here on slide master. See hidden slide 2 for direc6ons Compliance and Detection
Certificate Inventory @ Cisco
§  Cisco needs inventory for a number of reasons:
1.  Determine if certificates comply with security policy
2.  Find out who runs services over SSL/TLS and thus needs
communication around new policies/processes
3.  Learn which platforms are using SSL/TLS to gauge potential
need for module development
4.  Pull in certificate expiration data for notification purposes
§  Number of employees and diverse businesses makes
communications about new services & process hard
§  Network complexity makes scanning very tricky…
5 Insert presenter logo here on slide master. See hidden slide 2 for direc6ons Cisco at a glance…
~99,000 employees
~10,000 routers
~20,000 switches
15 Global Points of
Presence
•  2000+ Extranet
connections
•  10k+ Telecommuters
•
•
•
•
•  115+ acquisitions
•  650+ active suppliers
•  Operating in 77 countries
6 Insert presenter logo here on slide master. See hidden slide 2 for direc6ons Who is impacted?
•  Scope limited to SSL cer6ficates for first phase •  Determine: -­‐ Number of SSL cer6ficates out there -­‐ Cer6ficate authori6es in use (External & Internal) -­‐ Types of plaSorms where certs are deployed Perform monitoring of other (non-­‐SSL) mission cri6cal cer6ficates through manual import 7 Insert presenter logo here on slide master. See hidden slide 2 for direc6ons How to fix? – Analysis & Requirements
§  We don’t know what we don’t know
§  Need some way to discover what’s out there
§  Review current policies and processes and determine
what improvements need to be made
§  Cisco runs a wide array of web server platforms
§  Solution must support multiple platforms
§  Must be modular for adding platform support as needed
§  There’s other critical certs besides in-house SSL
servers
§  Need monitoring and notification of other critical certs
§  Executive sponsorship! (top-down is ideal)
8 Insert presenter logo here on slide master. See hidden slide 2 for direc6ons Analysis & Requirements (cont.)
Cert CA Exp. Date abc.cisco.com MS 3/4/11 VRSN 4/5/11 MS 5/23/12 www.cisco.com 123.cisco.com Upon discovering a cert that may expire, PKI team becomes accountable to some extent •  PKI team is small •  Increase resources or distribute responsibility
•  Need to integrate with Cisco’s cross-­‐
charge system •  Requires an approval system for managers to approve the cross-­‐
charge •  Need to collect contact info and map to certs •  Educate cert owners on responsibility and risk •  Distribu6ng the responsibility to cert owners eases the load on PKI team and increases awareness 9 Insert presenter logo here on slide master. See hidden slide 2 for direc6ons Analysis & Requirements (cont.)
§  Policy doesn’t address preferred PKI providers
§  Create evaluation criteria and update policy
§  Multiple Certificate Authorities are needed
§  Solution needs to interface with different CAs
§  Service owners don’t always install certs correctly
§  Installations should be automatically verified for correctness
§  Don’t auto-restart after installation
§  Need to notify service owners and allow them to bounce
§  Should monitor service to ensure bounce happens prior to
expiration
10 Insert presenter logo here on slide master. See hidden slide 2 for direc6ons How to fix? – Finding a Solution
§  Looked into many certificate management protocols
§
§
§
§
§
KMIP
CMP
CMC
XKMS
1619.3
§  Very few tools which implement these protocols
§  Nothing that would scale to Cisco’s needs
§  Web servers don’t implement the protocols either
§  Non-modular designs make new platform support hard
11 Insert presenter logo here on slide master. See hidden slide 2 for direc6ons The Solution
Cisco Custom Portal •  Integrated with dept. cross-­‐charge system •  Integrated with managerial approval system •  Cer6ficate Management interface Manage, Report, etc.
Discover
Provision
12 Insert presenter logo here on slide master. See hidden slide 2 for direc6ons Where are we now?
§  Custom Cisco front-end portal
§  Venafi certificate mgmt. back-end
§  Policy changed - new processes
mandatory
§  In production for almost 2 years
§  Full certificate lifecycle management
on multiple platforms
§  Thousands of SSL certificates issued
13 Insert presenter logo here on slide master. See hidden slide 2 for direc6ons Apply – Take Aways
Next three months:
§  Get ahead of the ball!
§  Try to gauge your key & cert mgmt. needs
§  Act now before your needs grow
Next six months:
§  Define responsibilities
§  Get executive sponsorship - top-down works
best!
Over the coming years:
§  Keep an eye on key management protocol
standards and adjust as necessary
14 Insert presenter logo here on slide master. See hidden slide 2 for direc6ons Question and Answer
Panel Discussion
15 Insert presenter logo here on slide master. See hidden slide 2 for direc6ons