IRT Operational Services
Service Level Agreement
IRT Infrastructure Services Group (ISG)
Data Center
June 22, 2007
IRT Operational Services - Service Level Agreement
Information Resources & Technology (IRT)
Table of Contents
Table of Contents.................................................................................................. ii
1. Scope of Document ....................................................................................... 1
2. General Term of the Service Level Agreement .............................................. 1
3. Warranty and Liability .................................................................................... 1
4. Services Provided .......................................................................................... 2
4.1. Data Center Services – Addendum1 .......................................................... 2
4.2. University Dependent Services .................................................................. 2
4.3. See customer specific document ............................................................... 2
5. System Availability ......................................................................................... 2
6. System Monitoring ......................................................................................... 2
7. Communications Methods ............................................................................. 3
8. Change Management Process ...................................................................... 3
8.1. Types of Requests ..................................................................................... 3
8.2. Systems Request Authority ........................................................................ 4
9. Problem Severity and Response Time .......................................................... 4
9.1. Severity Level 1: ......................................................................................... 4
9.2. Severity Level 2: ......................................................................................... 4
9.3. Severity Level 3: ......................................................................................... 5
9.4. Severity Level 4: ......................................................................................... 5
10.
Data Center Policies .................................................................................. 5
11.
Billing ......................................................................................................... 5
12.
Services and Pricing (see customer specific quotation) ............................. 5
13.
Security Audit Questions and answers (see customer specific document) 5
14.
Signatures (see customer specific document) ............................................ 5
Addendum 1: System Services Definitions: ......................................................... 6
Addendum 2: Services Performed for Hosted Servers ........................................ 7
Addendum 3: Services Performed for Managed Servers ..................................... 8
3/8/2016
Page
ii
IRT Operational Services - Service Level Agreement
Information Resources & Technology (IRT)
1.
Scope of Document
This document describes the IRT Operational Services provided to clients
in support of their ongoing business. This is used in conjunction with
customer specific agreements for particular services. It does not describe
other IRT services, such as application development or business systems
analysis, which are managed on a project basis.
2.
General Term of the Service Level Agreement
This Service Level Agreement (SLA) documents the agreement between
the client and IRT for delivery of services including services delivered,
levels of service, communications, and pricing. All terms are in effect until
modified by an amendment.
Amendments can be added to the agreement at any time that the parties
agree. If there are substantial service changes, then some time may be
required to implement. The timing of the amendment will be included in the
amendment. Changes to this agreement that result in changes in charges
may require 30 days to implement.
Either party can terminate this agreement in whole or in part with 30 days
notice. Billing rates may be adjusted based on service level changes. The
SLA is reviewed on its anniversary.
3.
Warranty and Liability
It is the mission of IRT to provide high quality, cost effective services to the
Bio-Medical Community at Stanford.
We commit to protecting the equipment and data supported under this SLA
from deliberate damage from IRT or other persons provided access to the
equipment by IRT. However, we will not be held liable for and damage to
equipment owned by the Department or data loss that occurs due to
accidental actions by IRT staff or other persons.
3/8/2016
Page
1
IRT Operational Services - Service Level Agreement
Information Resources & Technology (IRT)
4.
Services Provided
4.1. Data Center Services – Addendum1
4.2. University Dependent Services
There are several University services for which IRT is dependent and
reliant but has no operational control on delivery
o Power
o Air Conditioning
o Internet Access
o Infrastructure services

Stanford eCommerce/Verisign

Domain Name Services

IP Addressing

Web Authentication

Leland Email (SMTP - Outgoing Mail)

PeopleSoft/Directory/Registry

Etc.
4.3. See customer specific document
5.
System Availability
Systems will be available 7X24 except for regularly scheduled maintenance
downtime. The standard downtime maintenance schedule is 4am to 7am
on Wednesdays, other windows will be negotiated with each client and will
occur between 7am and 7pm. Clients will be given at least three (3)
business days notice of any scheduled downtime. There maintenance
windows scheduled by the University for network and firewall changes (i.e.
Thursday 4am to 7am).
IRT staff services will be available during normal business hours, Monday
through Friday, 8am to 5pm unless otherwise specified. Severity Level 1
outages will be responded to 7X24 for critical business systems.
6.
System Monitoring
Basic operational monitoring, periodic testing of systems for proper
functioning, is provided for all managed systems housed in the Data Center.
The monitoring system pages the on-call systems administrator when error
conditions are detected.
External operating monitoring can be arranged with the client paying the
fees (approximately $25/month/url) for this service.
3/8/2016
Page
2
IRT Operational Services - Service Level Agreement
Information Resources & Technology (IRT)
7.
Communications Methods
IRT will utilize an email list provided by each client (eg
client@lists.stanford.edu where the membership is set up and maintained
by the client.) This list will be used to provide information about
announcements, planned maintenance, and operational status of both
systems and applications.
8.
Change Management Process
All requests for changes to systems or applications, whether originated by
the client or by IRT Operations staff must go through the IRT change
management process for approval. The process starts with a request
submitted via HelpSU. Requests will be logged then sent via email to the
authorized Client for approval. The Client will return the request via email
with approval or denial of the request.
With the exception of emergencies, requests will not be done without Client
approval. In the case of an emergency, the client will be contacted as
quickly as feasible and informed of the changes.
8.1.
Types of Requests
8.1.1. Standard Customer Requests
All standard requests for account changes or other non-emergency
requests must be submitted via HelpSU http://Helpsu.stanford.edu.
The request must include:






Client Name
System Name
Application Name
Nature of the Request,
Date the Change is Needed
Problem Severity (level 1, 2, 3 or 4)
8.1.2. Emergency Customer Requests
Emergency requests must be submitted either in person or via the
Data Center hot line at (650) 725-8000. If the call transfers to voice
mail leave a message which includes your name and a call back
phone number.
The on call Systems Administrator will be
automatically paged within 5 minutes and will return your call.
8.1.3. Customer Escalation
If problems are not resolved to the client’s satisfaction by the above
methods, the client can escalate the response by contacting IRT
management in the following order:
1. Associate Director, Data Center
2. Director, ISG
3/8/2016
Page
3
IRT Operational Services - Service Level Agreement
Information Resources & Technology (IRT)
3. Chief Information Officer.
8.2.
9.
Systems Request Authority
We grant authority for changes based on a customer specific
addendum.
Problem Severity and Response Time
IRT Data Center will respond to problems according to the following
severity levels:
Problem Severity
Initial Response Time
Follow-up
w/Client
Level 1 – normal Respond to client within 30 minutes of Hourly
business hours
notification 100% of the time.
Level 1 - off hours
Respond to client within 1 hour of Hourly
notification 95% of the time
Level 2 - normal Respond to client within 4 hours of Daily
business hours
notification 100 % of the time
Level 3 – normal Respond to client within 1 working day of Weekly
business hours
notification 100% of the time
Level 4 –normal Respond to client within 3 working days of Monthly
business hours
notification 100% of the time
9.1.
Severity Level 1:
Major Business Impact – defined as a problem that causes complete
loss of service to the Client production environment and work can not
reasonably continue. Workarounds to provide the same functionality are
not possible and can not be found in time to minimize the impact on the
Client’s business. The problem has one or more of the following
characteristics:


9.2.
3/8/2016
A large number of users cannot access the system.
Critical functionality is not available.
The application cannot
continue because a vital feature is inoperable, data cannot be
secured, backed up, etc.
Severity Level 2:
Significant Business Impact – this classification applies when processing
can proceed but performance is significantly reduced and/or operation of
the system is considered severely limited. No workaround is available,
however operation can continue in a restricted fashion. The problem
has one or more of the following characteristics:
Page
4
IRT Operational Services - Service Level Agreement
Information Resources & Technology (IRT)



9.3.
Internal software error, causing the system to fail, but restart or
recovery is possible.
Severely degraded performance.
Some important functionality is unavailable, yet the system can
continue to operate in a restricted fashion.
Severity Level 3:
Minor Business Impact – a problem that causes minimal loss of service.
The impact of the problem is minor or an inconvenience, such as a
manual bypass to restore product functionality. The problem has one or
more of the following characteristics:



9.4.
A software error for which there is a Client acceptable workaround.
Minimal performance degradation.
Software error requiring manual editing of configuration or script files
around a problem.
Severity Level 4:
No Business Impact – a problem that causes no loss of service and in
no way impedes use of the system. The impact of the problem has one
or more of the following characteristics:


A software enhancement for which there is a Client acceptable
workaround.
Documentation error.
10.
Data Center Policies
See IRT Data Center Policies document for all policies including Security,
Change Management, Scheduled Maintenance, Backup and Restore
Procedure, Appropriate Use Policy, and Hardware Requirements. The
document
can
be
found
at
the
following
url:
http://med.stanford.edu/irt/datacenter/policies.html
11.
Billing
IRT Operations bills on a monthly basis, directly charging the appropriate
client account with the agreed upon charges. (See the customer specific
agreement for detailed charges.)
12.
13.
Services and Pricing (see customer specific quotation)
Security Audit Questions and answers (see customer specific
document)
14. Signatures (see customer specific document)
3/8/2016
Page
5
IRT Operational Services - Service Level Agreement
Information Resources & Technology (IRT)
Addendum 1: System Services Definitions:
Systems Administration
Client IRT
Hardware and Software Procurement
1. Recommend new or replacement hardware.
2. Receive all hardware and software after assets have been
registered.
Warranty/Maintenance Management
3. Understand all hardware warranty requirements/restrictions.
4. Resolve or coordinate with third-party vendors to resolve hardware
problems.
5. Ensure that hardware warranties are not violated.
6. Manage spare parts inventory.
Server Operation
7. Configure and install hardware in accordance with documented
standards.
8. Operate hardware in accordance with manufacturer requirements
and the documented standards that will ensure that the client’s
business objectives are met.
9. Perform routine housekeeping and system maintenance activities
as required and approved to ensure optimization of performance
within the client’s environment.
10. Start and stop processes and tasks as requested by the client and
as required by documented procedures.
Operating System
11. Maintain & enforce security best practices, including HIPAA data
12. Notify the client contact of the availability of new releases of
Windows Systems Group supported operating system and
supporting software.
13. Install new releases, upgrades and patches in a test environment.
14. Run test suites scripts and evaluate results
15. Certify that testing of new releases, upgrades and patches has
been performed successfully.
3/8/2016
Page
6
















IRT Operational Services - Service Level Agreement
Information Resources & Technology (IRT)
Addendum 2: Services Performed for Hosted Servers
Security Audit
Maintain Security
Monitor cameras
Monitor card access
Server audits
Recommend Patches
Recommend SW Upgrades
Scheduled Escort - supervision of hardware installs
Monitoring network
Monitoring System Access
Monitoring Air Conditioning
Provide standard 30A 208V Power
Provide Network
Provide, configure and change Firewalls
Review Bandwidth usage and headroom
Project Management
Scheduled Escort when required
3/8/2016
Page
7
IRT Operational Services - Service Level Agreement
Information Resources & Technology (IRT)
Addendum 3: Services Performed for Managed Servers
System Administration
O/S, Patches, Backup, basic admin
8-5 M-F response to system failures
Assistance with trouble shooting problems with these applications
Basic O/S installation, regular patching and security updates
Daily security updates
Occasional user requests
Standard Monitoring
Tomcat (1 instance included in base)
Web Servers with apache (1 instance included in base)
Webauth / ldap lookup
7x24 best efforts response to system failures
Additional monitoring for these applications
addressing performance issues
installation and debug of applications
Installation, and debug of database
setting up functions that allow users to run their scripts with root
privilege (sudo)
viewing files
7x24 response to system failures
Frequently changing role of a system
Managing frequent account changes
Managing SAN disk upgrades / migration
Multiple functions on one server (web, app, db)
Non-standard monitoring that requires special scripts
Non-standard setup
Non-supported O/S platform
tuning O/S level parameters
Database Administration
8-5 M-F support
basic database maintenance and patching
perform daily export backup
perform operational monitoring
setup a database account
setup database
setup monitoring
Single instance database (Oracle, SQL, Mysql)
user does schema, tables, views and procedures
user has full responsibility to run the database
24x7 best efforts support
Assist with minor Data Migrations / Updates
Assist with releases twice per quarter
3/8/2016
Page
8
IRT Operational Services - Service Level Agreement
Information Resources & Technology (IRT)
Assist with testing of scripts
Migration from test to production
Occasionally assist with scripts
24x7 support
Assist and troubleshoot job failures
Assist with Data Migrations / Updates
Audit backups
backup all transaction logs
help rebuild environments
Monthly changes to vendor scripts
Point in time recovery
3/8/2016
Page
9