Listen. Plan. Deliver.
Service Level Agreements - SLAs
A Necessary Evil, and
Why they are important…
December 9, 2015
Project Delivery Summit
Project Delivery Summit – 12-9-2015
Introduction
Listen. Plan. Deliver.
Hali Reyes (Customer perspective)
Executive Director WCDS Consortium
30 years of experience in public sector IT
projects
Fully executed contract value $500M+
Frank Otto, PMP (Vendor perspective)
Managing Principal
NexLevel Information Technology, Inc.
More than 25 Years of State and Local
Government IT Projects
Helped many clients with SLAs
Project Delivery Summit – 12-9-2015
What we will cover
Listen. Plan. Deliver.
Customer and Vendor Perspectives
Which projects need SLAs?
Why are SLAs necessary?
How are SLAs developed?
How are SLAs managed and applied?
Q & A
Project Delivery Summit – 12-9-2015
What are SLAs?
Listen. Plan. Deliver.
Service Level Agreements are contract
obligations that describe the expected
level of service and how that service
will be measured and managed
throughout the contract
Project Delivery Summit – 12-9-2015
When do you need SLAs?
Listen. Plan. Deliver.
Large, custom-service contracts
Examples
• WCDS Consortium – CalWORKs Information
Network (CalWIN)
• DFW - Automated License Data System
• CHP – Computer Aided Dispatch System
Some services have existing SLAs:
Cloud providers
• Azure – Monthly uptime of 99.95%
Telecom providers
Project Delivery Summit – 12-9-2015
Why are SLAs necessary?
Listen. Plan. Deliver.
Provides mutual understanding of
service to be delivered
Defines expected quality and importance
of services
SLAs set clear expectations of the
services to be delivered
SLAs delineate responsibility
Consequence – Liquidated Damage
Project Delivery Summit – 12-9-2015
Defining the SLAs
Listen. Plan. Deliver.
SLAs are defined in the RFP
It is customary to leverage SLAs from past
projects that are similar in size and scope
During re-procurement, it is typical for the
RFP to be updated with new vendor
obligations based on the account’s history and
past vendor’s performance
Vendors will cost a RFP response based on
SLA obligations
SLAs are often a significant part of
contract negotiations
Project Delivery Summit – 12-9-2015
Managing the SLAs
Listen. Plan. Deliver.
The contract defines what needs to be
measured
The contract does not contain enough
detail to define how each SLA will be
measured
The vendor and customer need to
mutually agree upon how each SLA will be
measured
This process is very time-consuming
Be prepared to create a mutually respectful
and results-oriented environment
Project Delivery Summit – 12-9-2015
Managing SLAs
Listen. Plan. Deliver.
After agreement is reached about how
the SLAs will be measured, it is time to
define the process
What reports and monitoring tools will you
need?
Who will monitor?
How will it be validated?
What is the escalation process if you
disagree?
Project Delivery Summit – 12-9-2015
Common SLAs
Uptime
System Response Time
Onsite Response Time (Repair)
Call Answer Statistics
Project Deliverables on Time
Software Development
Batch Processing
Project Delivery Summit – 12-9-2015
Listen. Plan. Deliver.
Uptime Example
Listen. Plan. Deliver.
Where to measure
Server O/S, Database, Application
How to measure
When did downtime start (helpdesk ticket?)
When did downtime end?
What was down?
• Complete system, one module, one site…
Subtract scheduled downtime
Remove what you control
Network, building power, HVAC
Project Delivery Summit – 12-9-2015
Uptime Chart
Availability %
90% ("one nine")
97%
98%
99% ("two nines")
99.5%
99.9% ("three nines")
99.99% ("four nines")
99.999% ("five nines")
Project Delivery Summit – 12-9-2015
Listen. Plan. Deliver.
Downtime per
Year
36.5 days
10.96 days
7.30 days
3.65 days
1.83 days
8.76 hours
52.56 minutes
5.26 minutes
Downtime per
Month
72 hours
21.6 hours
14.4 hours
7.20 hours
3.60 hours
43.8 minutes
4.38 minutes
25.9 seconds
System Response Time
Listen. Plan. Deliver.
Where to measure
Database, Application
How to measure
Scripted tool
Stopwatch over a shoulder
Database transaction time
Remove what vendor cannot control
Network?
Project Delivery Summit – 12-9-2015
Onsite Response Time
(Repair)
Listen. Plan. Deliver.
Tracking time starts when call is placed
(trouble ticket)
Metropolitan areas
Rural areas
Tracking time ends when repair
personnel are onsite
Consider tracking time to repair
Issue - Need to review all repair trouble
tickets
Project Delivery Summit – 12-9-2015
Call Answer Statistics
Listen. Plan. Deliver.
Example: Outsourced Help Desk
Stats should come from ACD/PBX
Track:
Average call answer time
Average and maximum hold time
Count busy signals (at the switch)
Project Delivery Summit – 12-9-2015
Project Deliverables
Listen. Plan. Deliver.
Project Deliverables (e.g. Project Plan,
Scope Plan, Comm Plan, etc…)
Set turnaround times for Deliverable
Expectation Documents (review days
per number of pages)
Set dates for Deliverables
Set turnaround times for edits
(cosmetic vs. content)
Project Delivery Summit – 12-9-2015
Software Development
Release Quality
User Acceptance Test (UAT)
Features and Functions
Release Timeliness
Target Date
Deficiency Resolution
Timely
By Priority
Project Delivery Summit – 12-9-2015
Listen. Plan. Deliver.
Batch Processing
Listen. Plan. Deliver.
Timeliness
Example: System up by 6:00 am
Can impact performance
Can impact time-sensitive processes
Percent of jobs completing successfully
Cancelled jobs can impact client
Day, Week, Month, Quarter, Annual
Project Delivery Summit – 12-9-2015
Problematic SLAs
Downtime for workstations
Downtime from trouble tickets
Slow response time measured at
Database
Uptime measured at O/S
Project Delivery Summit – 12-9-2015
Listen. Plan. Deliver.
SLA Management
Listen. Plan. Deliver.
SLAs need to be monitored
Monthly meeting
Verification can be hard and time
consuming
Vendor report of all SLAs
Start in RFP
Project Delivery Summit – 12-9-2015
Philosophy of SLAs
Listen. Plan. Deliver.
Create an environment where:
Processes are defined
Monitoring occurs
Vendor and client mutually contribute
Core SLA Team
Works through issues
Keeps discussions to core members
Escalate to management as needed
Project Delivery Summit – 12-9-2015
Questions
Project Delivery Summit – 12-9-2015
Listen. Plan. Deliver.