KT v01.14
EDUCATION
The Storage Capacity Design Dilemma
an ITIL approach
LeRoy Budnik
Knowledge Transfer
KT v01.14
SNIA Legal Notice
EDUCATION
• The material contained in this tutorial is copyrighted by
the SNIA and portions are subject to other copyrights1.
• Member companies and individuals may use this material
in presentations and literature under the following
conditions:
– Any slide or slides used must be reproduced without modification
– The SNIA must be acknowledged as source of any material used
in the body of any document containing material from these
presentations.
– This specific legal notice shall not be removed.
• This presentation is a project of the SNIA Education
Committee.
The Storage Capacity Design Dilemma © 2007 Storage Networking Industry Association. All Rights Reserved.
1
© 1997-2007 Knowledge Transfer
2
KT v01.14
Abstract
EDUCATION
The Storage Capacity Design Dilemma
As architects, we must continually discover capability limits, constraints,
and patterns and match requirements, capabilities and cost to provide
an effective design. When we add the business requirements, it
becomes the core mantra of ILM. Yet translating these requirements
into hardware and software is not easy. In the process, we must
continuously choose between complex alternatives, some of which
seem equally unacceptable, and hence the dilemma.
In this session,
we will introduce fundamentals of storage infrastructure design from the
perspective of ILM requirements. We will introduce key formulas, the
science of storage capacity planning and follow a case study to
demonstrate application in host, fabric and array design, to meet the
requirements. The dilemmas of design decisions clear up once you
know the formulas. It will change your thinking.
The Storage Capacity Design Dilemma © 2007 Storage Networking Industry Association. All Rights Reserved.
3
KT v01.14
Objectives
EDUCATION
In this session, we have a dual focus on the science and consultative
process of developing a mature discipline of storage capacity
assessment, planning and design. The IT Information Library (ITIL) is
the foundation of the methodology. Essential formulas come from a
variety of sources.
Objectives:
•
•
•
Know basic storage capacity and performance formulas
Design for capacity and scalability in hosts, SAN, storage arrays and
backup
Understand how to break out of the most common storage design
dilemmas
The Storage Capacity Design Dilemma © 2007 Storage Networking Industry Association. All Rights Reserved.
4
KT v01.14
TLS Inc.
EDUCATION
The Storage Capacity Design Dilemma © 2007 Storage Networking Industry Association. All Rights Reserved.
1
© 2000-2007 Knowledge Transfer
5
KT v01.14
Capacity Management
EDUCATION
No
text
Goal
Scope
Why?
Business
To
understand
Strategy
the and
future
Plan,
business
IT Strategy
Capacity
Management
ensures
that IT
requirements,
and
Planand
including
the organization’s
allcapacity
applications,
operations
process
storage
provisioning
and
hardware,
IT evolving
infrastructure
O/S, business
networking,
to ensure
peripherals
thatinalla cost
and
match
demand
current
human
and
effectiveresources
andfuture
timelycapacity
mannerand
performance aspects of the business
requirements are provided cost effectively.
The Storage Capacity Design Dilemma © 2007 Storage Networking Industry Association. All Rights Reserved.
1
© Crown Copyright with value added status
6
KT v01.14
Process
EDUCATION
No
text Capacity
Inputs
Service
Resource
Capacity
Management
Management
Business
Capacity
Management
Outputs
• Deployment and
Technology
• report
Proactive
changes
••• Capacity
Monitor,
analyze,mode,
tune
run and
and
reporton
on
theplans
Trend, forecast,
prototype,
size
and and
Plan
development
and
SLAs,
SLRs
and
Service
service
improvements
programs
storage
utilization
service
of
storage
performance
components
document
of
future
business
requirements
and
•• Capacity
Database
(CDB)
Catalog
•• Revised
operational
Expected,
scheduled
their
impact
on
storage
services
• Baselines
and
profiles
Establish
baselines
and
profiles
of
use
of the
• Business plans and
changes
schedule
• Capacity
(regular,
storage
services
components
strategy reports
• Effectiveness
Incidents and reviews
problems
•
hoc and exception)
•• ad
Manage
demand
services
IT plans and
strategyfor storage
Service
Reviews
•• Audit
reports
• SLA and SLR
• recommendations
Business requirements
and transaction volumes
• Costing and charging
• recommendations
Operational schedules
• SLA breaches
• Financial plans
• Budgets
The Storage Capacity Design Dilemma © 2007 Storage Networking Industry Association. All Rights Reserved.
1
© Crown Copyright with value added status
7
KT v01.14
Iterative Capacity
Management Activities
The Storage Capacity Design Dilemma © 2007 Storage Networking Industry Association. All Rights Reserved.
EDUCATION
1
© Crown Copyright with value added status
8
KT v01.14
CDB
EDUCATION
The Storage Capacity Design Dilemma © 2007 Storage Networking Industry Association. All Rights Reserved.
1
© Crown Copyright with value added status
9
KT v01.14
Workload Management
EDUCATION
No
Technical
text Metrics
Categorize
Process
Classify Metrics
•
•
•
•
•
•
•
•
•
Utilization
Business
I/Os
Functional
Block
Size
Operational
Transfer Rates
Cost
MTTP
Open
MTTI
Closed
MTTM
MTBF/MTTR
The Storage Capacity Design Dilemma © 2007 Storage Networking Industry Association. All Rights Reserved.
1
© Crown Copyright with value added status
10
KT v01.14
Producing a Capacity Plan
EDUCATION
Capacity
No
text Plan Structure
•
•
•
•
•
•
•
•
•
•
•
Introduction
Scope
Methods
Assumptions
Management Summary
Business Scenarios
Storage Service Summary
Storage Resource summary
Options for Storage Service Improvement
Storage Cost Model
Recommendations
The Storage Capacity Design Dilemma © 2007 Storage Networking Industry Association. All Rights Reserved.
1
© Crown Copyright with value added status
11
KT v01.14
Costs, benefits, problems
EDUCATION
Costs
Benefits
Possible
No
text Problems
•
••
••
••
•
•
•
•
•
Increased
Customer
efficiency
andexceed
cost savings
technical capacity
Hardware,expectations
software and
tools
– Deferred
expenditure
Over
expectation
of tuning benefits
Project
management
– Economic and
provisioning
Unrealistic
unachievable performance targets
Staff
– Planned purchasing
Over
optimistic
vendor
estimates
Accommodation
Reduced risk
Lack of Information
No text
Confident forecasts
Reliance on Linear Prediction
Proven value to application lifecycle
The Storage Capacity Design Dilemma © 2007 Storage Networking Industry Association. All Rights Reserved.
12
KT v01.14
Planning and Implementation
EDUCATION
• Capacity Management
as a planned event!
Reviewmonitoring
what existspoints
– Define
• Plan the process
– Integrate into other
planning processes
– Train
staff
Seek the
business
– confidence
Establish timeline
levels and
– Establish monitoring and
place components
data collection
– integrated
Identify infrastructure
for delivering the
volatility (probability)
– Expand
include:
CapacitytoPlan
– Business Capacity
– Service Capacity
– Resource Capacity
• Synchronize with the
budget cycle
• Implement the process
– Locate the Capacity data
The Storage Capacity Design Dilemma © 2007 Storage Networking Industry Association. All Rights Reserved.
13
KT v01.14
Process Review
EDUCATION
Review
Validate
Critical
Key
No
text
Performance
Success
Goals
Metrics Factors
Indicators
•
•••
•
•
••
••
•
•
•
•
•
•
•
Calculations
and formulas
Accurate
Is
required
business
output
available
forecastsat the required times for
Resource
forecasts
the
appropriate
Component
utilization
Knowledge
of
ITaudience
strategy and plans, and that the
Technology
Acceptable
data
collection
Is it’s activities
plans
are accurate
cost
effective?
Cost-effectiveness
Accuracy
of tuning
predictions
Understanding
of current
and future technologies
Measurement
of
accuracy
of
match
between
Are
potential
and
actual
breaches
predicted
and
within the capacity management team
IT Storagewith
Capacity
and Business Need.
associated
full
notification?
Ability to demonstrate cost effectiveness
Are imposed constraints within acceptable client limits?
Interaction with other Service Management processes
Is reporting regular and on time?
Storage capacity matches business need
Is the capacity plan produced on time?
Are recommendations clear and accurate?
The Storage Capacity Design Dilemma © 2007 Storage Networking Industry Association. All Rights Reserved.
14
KT v01.14
Capacity Process Interfaces
EDUCATION
The Storage Capacity Design Dilemma © 2007 Storage Networking Industry Association. All Rights Reserved.
15
KT v01.14
Roles and Responsibilities
EDUCATION
Capacity
Responsibilities
Storage Team:
Capacity
include
focus
(1):
(2):
areas:
Manager:
••
••
••
•••
••
•
••
Often
Produce
Test
Produce
performance
split
storage
and
intomaintain
performance
capacity
of new
storage
storage
plans
monitoring
capacity
systems
and
plan
Matches
and
demand
by
capacity
planning
increasing
or managing
available
Produce
Recommend
Monitor storage
storage
service
management
levels reports
storage
Size
Recommend
– Tuning
newcapacity
storage
and storage
design
systems
improvements
tuning
Ensuring
thattechnology
existing
storageproblems
capacity
Assess
Manage
– Resolution
new
storage
to
performance
demand
is used optimally
Implement
Recommend
– Use of demand
service
storage
management
level
capacity
reporting of capacity
enhancements
Predict
Studies
demand
as required
for storage services
Account forthat
workload
Determine
storagechanges
service levels are
in storage service
level
targets
maintained
and cost
justified
The Storage Capacity Design Dilemma © 2007 Storage Networking Industry Association. All Rights Reserved.
16
KT v01.14
EDUCATION
Knowledge Statements
The Storage Capacity Design Dilemma © 2007 Storage Networking Industry Association. All Rights Reserved.
17
KT v01.14
Systematic Approach
EDUCATION
Assessment is a systematic approach to enable diverse requirements,
including availability, capacity, cost, compliance and governance to be
met over time. The process achieves alignment at best cost by
summarizing requirements and matching them to capabilities at each
requirement state change. The range of solutions available are limited by
the value of information, with the goal of minimizing budgets, both capital
and expense.
The Storage Capacity Design Dilemma © 2007 Storage Networking Industry Association. All Rights Reserved.
18
KT v01.14
Storage Tier
EDUCATION
A storage tier is a collection of storage capacity that
meets requirements with a consistent set of attributes,
capabilities and characteristics which may include:
•
•
•
•
Availability
Performance
Quality of Service
Cost
Storage Tier ≠ Storage Service
For example: it is possible that disks in the same array might be treated
as different tiers because of RAID Level, data location on a spindle or
other characteristics – the choice is yours…
The Storage Capacity Design Dilemma © 2007 Storage Networking Industry Association. All Rights Reserved.
19
KT v01.14
Storage Service
Business View
The Storage Capacity Design Dilemma © 2007 Storage Networking Industry Association. All Rights Reserved.
EDUCATION
1
© 2000-2007 Knowledge Transfer 20
KT v01.14
Storage Service
Technical View
The Storage Capacity Design Dilemma © 2007 Storage Networking Industry Association. All Rights Reserved.
EDUCATION
1
© 2000-2007 Knowledge Transfer 21
KT v01.14
EDUCATION
Now for something more: technical . . .
The Storage Capacity Design Dilemma © 2007 Storage Networking Industry Association. All Rights Reserved.
22
KT v01.14
Basic Storage Capacity
EDUCATION
Storage Perspective
• Block
• Storage Pool
– Given set of capabilities
– Quality of Service Range
– Pools are described by:
• Total Managed Storage
• Remaining Managed
Storage
•
User Perspective (SRM)
• Allocated
– Static
– Dynamic
– Virtual
• Available
– Mapped
– Unmapped
• Used, Free, Total, Reserved
Primordial Pool
– Total System Size (Raw)
•
•
Concrete Pool
Blocks, Metadata and
Reported Capacity
The Storage Capacity Design Dilemma © 2007 Storage Networking Industry Association. All Rights Reserved.
23
KT v01.14
Basic I/O Metrics
EDUCATION
The Storage Capacity Design Dilemma © 2007 Storage Networking Industry Association. All Rights Reserved.
24
L02T06
KT v01.14
Service Center Metrics
(Queuing Theory)
The Storage Capacity Design Dilemma © 2007 Storage Networking Industry Association. All Rights Reserved.
EDUCATION
25
L02T07
KT v01.14
Little’s Law
EDUCATION
The Storage Capacity Design Dilemma © 2007 Storage Networking Industry Association. All Rights Reserved.
26
KT v01.14
Little’s Law – Random
arrivals and response time
The Storage Capacity Design Dilemma © 2007 Storage Networking Industry Association. All Rights Reserved.
EDUCATION
27
KT v01.14
TLS Inc.
EDUCATION
The Storage Capacity Design Dilemma © 2007 Storage Networking Industry Association. All Rights Reserved.
1
© 2000-2007 Knowledge Transfer 28
KT v01.14
Storage Performance
Requirements
EDUCATION
Performance requirements
Other requirements
•
•
Retention
•
Scalability (is this an SLO?)
•
Format
•
Co-location/Separation
•
•
•
I/Os per second (average/peak)
–
–
–
Bandwidth (average/peak)
Jitter (range/mean+std. Deviation)
Isochronous
Access requirements
–
–
–
MTFB (initial access time)
Locking (concurrent access)
File system affinity
Security requirements
–
–
–
Encryption level
Integrity
Indelibility
Load characterization
–
–
–
Random read vs. write %
Sequential read vs. write %
Block size distribution
–
–
–
–
–
–
–
Initial size (+Max. size)
Size growth % per time period
Bandwidth growth % per time period
Record vs. byte stream vs. ?
Media portability
(for association)
–
–
•
Retention period/Expiration date
Disposition (erase/shred/archive, …)
Level (host/datacenter/geographic)
Co-operation
Price
The Storage Capacity Design Dilemma © 2007 Storage Networking Industry Association. All Rights Reserved.
29
L02T09
KT v01.14
Growth
EDUCATION
• Growth (Capacity: Size)
– Normal: Trend, Horizontal, Vertical (account for each)
– Burst: Aggregation, Consolidation, Integration
– Special Cases: Line Items/Transactions
• Performance (Capacity: Performance, Workload Response)
– General metric is response time (RT)
• Availability
• Manageability (Financial: Cost)
– How many people does it take?
The Storage Capacity Design Dilemma © 2007 Storage Networking Industry Association. All Rights Reserved.
30
KT v01.14
Defining Scalability
EDUCATION
The Storage Capacity Design Dilemma © 2007 Storage Networking Industry Association. All Rights Reserved.
31
KT v01.14
Q&A / Feedback
EDUCATION
•
Please send any questions or comments on this presentation to
SNIA: trackstoragemgmt@snia.org
Many thanks to the following individuals
for their contributions to this tutorial.
SNIA Education Committee
LeRoy Budnik, Knowledge Transfer
Phil Huml, Knowledge Transfer
Bob Rogers, Application Matrix
The Storage Capacity Design Dilemma © 2007 Storage Networking Industry Association. All Rights Reserved.
32