Americas Storage Advanced Technical Skills (ATS)
IBM System Storage DS8700
Workload Optimization with
Easy Tier
Brian Sherman bsherman@ca.ibm.com
Paulus Usong usong@us.ibm.com
Americas Storage ATS
6/10/2010
© 2010 IBM Corporation
Americas Advanced Technical Skills (ATS)
Agenda
 Solid State Storage (SSD) Technology
 IBM Easy Tier Workload Optimization
 Workload Optimization Tools
 Performance with Easy Tier
2
© 2010 IBM Corporation
Americas Advanced Technical Skills (ATS)
Why Solid State Drive (SSD) Technology
 Performance constrained by current
drive limitations
100
90
• Very good $/Gbyte, but slow access performance
• Access performance is slow – 150-200 Ops/Sec
80
– Processor capabilities are out-stripping disk 70
drive and RAID controller performance
60
(rotational speed and IOPS)
50
• As a result, servers and storage systems become
40
more unbalanced between CPU/controller capability
and storage performance
Clients add more drive spindles to improve
performance
•
 Reduced Energy Use
– Power
•
Fraction of power consumed
– Spinning drives typically 10-14 watts
– SSD drives typically 2-8 watts
Percent
– Disk drives are the current technology
CPU Time
Opportunity to reduce
wait time
•Wait %
•Sys %
30
20
10
0
•User %
Clock Time
– Heat
•
Fraction of heat dissipated
3
© 2010 IBM Corporation
Americas Advanced Technical Skills (ATS)
IBM Solid State Drive Strategy
SSD
SSD
SSD
HDD
HDD
Solid State as internal
storage for the Server
SSD as a slower nonvolatile store (NVS) cache
SSD as resident disk
external storage
Edge and DMZ servers
outside corporate firewall
Offers performance
acceleration of all I/O, or
designated I/O
For OLTP, ERP and
other high requirements
for IOPS
Managed using cache
algorithms
Can be used for
business analytics
Fast re-boot
Save up to 1500 Watts per
server rack (84 servers) with
IBM BladeCenter
4
© 2010 IBM Corporation
Americas Advanced Technical Skills (ATS)
What are Solid State Drives
 Semiconductor (NAND flash)
– Electronically erasable persistent medium
– No mechanical read/write interface
– No moving parts
 Plug compatible with common HDD form
factors and interfaces
– 3.5”, 2.5”, 1.8”
– SATA, FC, SAS, PCI-E
 Flash technology improvements allowing
for use in enterprise storage and server
deployments
– Other Storage Class Memory technologies in the
wings
 Greater than 5 year lifetime
New Tier-0 drives for high priority, time-sensitive applications
5
© 2010 IBM Corporation
Americas Advanced Technical Skills (ATS)
Recommended Data Types for SSD Use
 Random access, high performance data sets
– High read-to-write ratio; low cache hit ratio
– High I/O rates; likely to be large data objects
– For example, Oracle, DB2, Exchange and SQL Databases
 SSD is practical for high access density data
– Average access density today is less than 600 I/O per TB
– High access density typically over 1,800 I/O per TB
 Mission-critical applications
– Linked directly to the client’s business success
 Applications that short stroke 15K HDDs
 Used for applications where disks are the bottleneck
 Sequential access data can remain on disks
 All other data stays on HDD
$/IOP makes sense with SSD. $/MB does not make sense with SSD
6
© 2010 IBM Corporation
Americas Advanced Technical Skills (ATS)
Client Challenges with SSD
Inefficient use of an expensive asset is difficult to justify
 Solid State Drive technology remains more expensive than HDD
– HDD and SSD technologies will coexist in hybrid configurations for several years
– During this transition, use SSDs for data which can benefit most from the new technology
 Without sub-volume optimization tools, clients have been over-provisioning
the resource
 Hot data not often static
– Administrators spend too much time regularly monitoring, reporting, and tuning tiers
Result: Many clients feel they can’t afford solid state storage yet
7
© 2010 IBM Corporation
Americas Advanced Technical Skills (ATS)
Workload Monitoring and Smart Data Placement
 Storage hardware based tool
– DS8000
•
•
Performance analytics identify both open and z/OS sub-volume hotspots
Automated data relocation function to dynamically move hot and cold extents across drive
tiers
 Traditional OS based tools
– System Z
•
•
•
•
DFSMS policy based storage management controls allocation to SSDs
Enhanced System Monitoring Facility records
SAS® tooling available for free download
DMCzOS SMF record analysis for candidates + TDMF and zDMF for data movement
– System P
•
•
•
•
Data collection: Iostat, filemon, DB2 Tablespace Snapshot
Migration: Softek Transparent Data Migration Facility (TDMF)
Migration: SVC storage pools
TPC Intelligent Performance Optimization - hotspot function builds scripts for SVC data
movement
– System I
•
•
Data collection: Collection Services, Performance Explorer
Migration: ASP Balancer, DB2 Preferred Media Support, ASP dedicated to SSD
– IBM Novus Intelligent Storage Service Catalogue
•
Storage Enterprise Request Planner (SERP)
8
© 2010 IBM Corporation
Americas Advanced Technical Skills (ATS)
DS8000 Logical Configuration and Virtualization
A LSS/LCU is a collection
of volumes and has
relevance mostly for copy
services and CKD
Volumes are created
from one or more
extents from a single
extent pool
An extent pool is a
collection of ranks all
of which are managed
by the same DS8000
server (0/1)
A rank is created
when an array is
formatted for either
CKD or FB
An array site is a
collection of 8 drives of
the same type spread
across both internal
fabrics(loops) on the
DS8000
9
Ranks are divided into
extents which are
either 1GB (FB) or
1113 cylinders (CKD)
6+P+S 146GB
6+P+S 146GB
An array is formed when
the drives in an array site
are formed into a RAID
group (RAID5/6/10) and
potentially a spare is
assigned
© 2010 IBM Corporation
Americas Advanced Technical Skills (ATS)
IBM System Storage Easy Tier
 Easy Tier functionality
– Workload monitoring
– Workload analysis
– Smart data placement
 Two data relocation functions that enable smart
data placement to optimize SSD deployments with
minimal cost
– Automatically migrate sub-volume data to right tier
– Manually migrate full volumes non-disruptively
 Storage Tier Advisor tool provides guidance for
existing client workloads
– No requirement for SSDs for monitoring and analysis
DS8700
DS8700
Simplicity, flexibility, economy
10
© 2010 IBM Corporation
Americas Advanced Technical Skills (ATS)
Easy Tier – Functional Overview
 Easy Tier supports online dynamic relocation of data
– Data can be relocated to/from SSD and FC/SATA
• Automatic Mode
– Provides automated extent level relocation granularity
– Requires use of a merged extent pool
• Manual Mode
– Provides online dynamic volume relocation capability
– CLI/GUI setup and management
– Storage Tier Advisor Tool (advisor tool) for I/O analysis and projected benefit
 DS8700 Hardware feature
– Easy Tier is a no charge feature available with Release 5.1
– Supported by all server platforms with no additional software
11
© 2010 IBM Corporation
Americas Advanced Technical Skills (ATS)
Easy Tier - Data Relocation Options
Manual Mode (Volume Level) and Automatic Mode (Extent Level)
SSD Rank Pools
FC or SATA Rank Pools
Merged Pool
(SSD/FC or SSD/SATA)
Manual Mode
Volume Based Data Relocation
Automatic Mode
Extent Level Data Relocation
Automatic extent level data relocation enabled in a Merged Extent Pool
12
© 2010 IBM Corporation
Americas Advanced Technical Skills (ATS)
Easy Tier – Automatic Mode
 Extent Migration Plan built based on I/O statistics
– Dynamically relocates a logical volume’s extents
• Hot extents relocated to SSD Arrays
• Cold extents relocated from SSD to FC/SATA Arrays
 Extent level relocation requires mixed technologies in a merged
extent pool
– SSD + Enterprise FC
– SSD + SATA
 DS8000 Extent Size
– 1GB for FB
– 3390 Mod 1 (0.94GB) for CKD
13
© 2010 IBM Corporation
Americas Advanced Technical Skills (ATS)
Easy Tier – Automatic Mode Extent Relocation
Logical Volume
Mixed Technology Extent Pool
Extent
Virtualization
SSD Arrays
Cold Extents
Migrate Down
Hot Extents
Migrate Up
11GB FB Volume
4GB identified as
hot/candidate for SSD
HDD Arrays
14
© 2010 IBM Corporation
Americas Advanced Technical Skills (ATS)
Easy Tier – Automatic Mode Operation
 Monitors stage/destage activity on each extent to determine the data
‘temperature’
– Can be set to monitor all volumes or merged extent pool volumes (default)
 Develops an extent migration plan for optimal data placement every 24
hours of I/O statistics
– Read/write, random/sequential, large/small transfer size all collected as input to
developing the migration plan
– Temperature generated is based on cumulative average latency time
 Migrates extents within an extent pool according to plan over 24 hour
period
– A limited number of extents are chosen for migration every 5 minutes
– Monitors whether rank workload exceeds rank performance capability
– Rank, Device Adapter and cluster level thresholds
15
© 2010 IBM Corporation
Americas Advanced Technical Skills (ATS)
Easy Tier – Automatic Mode Setup
 Extent pool configuration options
– Create new extent pool with SSD + HDD ranks
– Merge existing SSD and existing HDD extent pool
– Add SSD rank to existing HDD extent pool
– Add HDD rank to existing SSD extent pool if no space efficient capacity (virtual capacity
or repository) configured in the extent pool
 Logical volume configuration options
– Create new logical volume in mixed technology extent pool
– Migrate standard logical volumes between homogeneous and mixed technology extent
pool
– Merge existing homogeneous extent pool with existing logical volumes with another
extent pool to create a mixed technology extent pool
16
© 2010 IBM Corporation
Americas Advanced Technical Skills (ATS)
Easy Tier – Automatic Mode User Interfaces
 Design point is for customer to not normally need or use any controls
to manage Automatic Mode
 Easy Tier Automatic Mode
– Enable / disable
•
Stops automatic mode extent migration while disabled
 Easy Tier Automatic Mode monitoring
– Disable / managed volumes / all volumes
•
•
•
Ignores workload activity in plan generation statistics while disabled
Managed volumes is normal setting and monitors workload on volumes in managed pools if the Easy Tier
LIC feature is enabled
All volumes causes all logical volumes to be monitored (with or without LIC feature)
– Can be used with analysis tools prior to ordering and enabling the LIC feature
– Can be used to analyze logical volumes in homogeneous extent pools as candidates for mixed
technology pool
 Storage Tier Advisor Tool (advisor tool)
– Generate reports on extent monitoring and obtain SSD capacity planning
recommendations
17
© 2010 IBM Corporation
Americas Advanced Technical Skills (ATS)
Easy Tier – Manual Mode
 Easy Tier Manual Mode allows a user to perform the following actions:
– Volume Migration
• Allows existing data to be restriped over an extent pool once new capacity is added
• User can change a logical volume’s storage technology by dynamically relocating between
extent pools
– Different RAID format, drive size/type
– Extent Pool Merge
• User can merge two existing extent pools without moving data
– Consolidate extent pools with equivalent disks
– Merge extent pools with to create a mixed technology extent pool for Automatic Mode
18
© 2010 IBM Corporation
Americas Advanced Technical Skills (ATS)
Easy Tier – Manual Mode Volume Migration
Change between extent pools
with and without Easy Tier
automatic mode
Solid State Disk
146 GB / RAID 5
+
Enterprise Disk
300 GB / 15K RPM
RAID 5
Enterprise Disk
600 GB / 15K RPM
RAID 5
Solid State Disk
146 GB / RAID 5
SATA Disk
2 TB / 7.2K RPM
RAID 6
19
Re-stripe
extents
Change disk class
Change RAID type
Change disk RPM
© 2010 IBM Corporation
Americas Advanced Technical Skills (ATS)
Easy Tier – Manual Mode Operation
 Volume migration
– Migration pre-allocates target extents in target extent pool, releases
each source extent when migrated
– Any number of volumes can be requested to migrate
– Processes a limited number of volumes concurrently
– Actively migrates a number extents per CEC concurrently
– Overheads comparable to FlashCopy with background copy
 Extent Pool merge
– Changes source ranks extent pool to target extent pool
– Changes source logical volumes extent pool to target extent pool
– Deletes source extent pool if complete prior actions
20
© 2010 IBM Corporation
Americas Advanced Technical Skills (ATS)
Easy Tier – Manual Mode User Interfaces
 Migrate Volume (Target Extent Pool, Extent Allocation Method)
– Can specify current extent pool for target extent pool
– Can keep the current Extent Allocation Method or change
 Pause/Resume Volume Migration
– Pause stops initiation of any new extent migrations on a volume
– Resume continues migration
 Cancel Volume Migration
– Nullifies volume migration if it has not started
– Stops volume migration if it has started. Can request migrate volume to source
or target extent pool to retry
 Merge Extent Pool (Source Extent Pool, Target Extent Pool)
– Deletes source extent pool if merge is successful
21
© 2010 IBM Corporation
Americas Advanced Technical Skills (ATS)
Easy Tier – Supported Environment in R5.1
 DS8700 R5.1
 Automatic Mode and Volume migration supported on standard logical
volumes
 Space efficient volumes are not Easy Tier managed
– Can reside in an extent pool managed by Automatic Mode
– Volume migration is not supported on space efficient volumes
 Migrate data between extent pools on the same storage image (0-0 / 1-1)
 Copy services considerations
– Easy Tier optimization of data on the primary system is not reflected at the secondary
 Can merge any two extent pools except:
–
–
–
–
–
Both must be same extent type (CKD or Fixed Block)
Both must be within the same storage image
At most, one of two extent pools can have a space efficient repository
At most, one of one of two extent pools can have virtual capacity
Not allowed if one extent pool is homogeneous with SSD disks and additionally has space
efficient repository or virtual capacity configured
 Not supported on encryption capable storage facilities
– No encryption capability on SSD drives
22
© 2010 IBM Corporation
Americas Advanced Technical Skills (ATS)
Easy Tier – Workload Considerations
 Implementation Characteristics
– Extent size is 1 GB on FB, Mod 1 on CKD
– Plan Generation Window is 24 hours
 Some workloads may not benefit significantly from Automatic
Mode
– Hot spots are small in size and uniformly distributed across extents such that
all extents exhibit equal temperatures
– Hot spots vary over time such that they are uniformly distributed given a large
enough monitoring period
– Critical workload to be performance optimized is intermixed with other
workloads that result in a non-optimal extent placement
•
May be able to turn off monitoring in time windows where non-critical workloads are affecting statistics
in an undesirable (e.g. batch windows, off-shift or weekend workloads, month-end processing, etc.)
 Consult Easy Tier performance white paper for additional
information
– http://www03.ibm.com/support/techdocs/atsmastr.nsf/fe582a1e48331b5585256de50062ae1c/b16b8f
8dcb4a21028625763a0081b712/$FILE/DS8000%20Storage%20Virtualization%20Overvie
w%20V2.0.pdf
23
© 2010 IBM Corporation
Americas Storage Advanced Technical Skills (ATS)
IBM System Storage DS8700
Workload Optimization Tools
6/10/2010
© 2010 IBM Corporation
Americas Advanced Technical Skills (ATS)
Tools for SSD
 Easy Tier Tools
– Storage Tier Advisor Tool
– Disk Magic
 SSD planning tool
– IBM z: FLASHDA
– IBM i: SSD analyzer tool
– Disk Magic
– Americas ATS DS8700 SSD Configuration Capacity Calculator
25
© 2010 IBM Corporation
Americas Advanced Technical Skills (ATS)
Storage Tier Advisor Tool
 Data is collected on both servers of the DS8700
– Release 5.1 and later
 Data is pulled onto a Windows PC
 The advisor tool runs as a Windows application
26
© 2010 IBM Corporation
Americas Advanced Technical Skills (ATS)
Disk Magic (Easy Tier)
 Future release
 Model current disk subsystem to a DS8700 using Easy Tier
– Both Open and z Systems
 Model is based on Tucson performance observations with
various benchmarks
27
© 2010 IBM Corporation
Americas Advanced Technical Skills (ATS)
IBM z FLASHDA Tool
 Input is SMF record type 42 subtype 6 and type 70 through 78
 Tool uses SAS
 Analysis will identify datasets and volumes that are good
candidates for placement on SSD
28
© 2010 IBM Corporation
Americas Advanced Technical Skills (ATS)
IBM i SSD Analyzer Tool
 Analyze the workload to determine the benefits of SSD
 Report ranks best SSD candidates by job name
29
© 2010 IBM Corporation
Americas Advanced Technical Skills (ATS)
Disk Magic (non Easy Tier)
 Modeling is at the volume level
 z/OS
– Disk Magic can estimate which volumes are good SSD candidates
and migrate those volumes to SSD
• Or use the result of the FLASHDA analysis
 Open System
– Disk Magic can model SSD on a server basis
30
© 2010 IBM Corporation
Americas Advanced Technical Skills (ATS)
DS8700 SSD Configuration Capacity Calculator
31
© 2010 IBM Corporation
Americas Advanced Technical Skills (ATS)
DS8700 SSD Configuration Capacity Calculator
 Feature is Megapack
 Options
– SSD: 73GB or 146GB
– FB Extents or CKD Extents
– # of SSD Ranks
 Capacity is in decimal
32
© 2010 IBM Corporation
Americas Advanced Technical Skills (ATS)
Storage Tier Advisor Tool
Usage
33
© 2010 IBM Corporation
Americas Advanced Technical Skills (ATS)
DSCLI Commands
 Start Easy Tier monitoring
chsi –etmonitor all|automode|none
– all: monitor all volumes on the DS8700
– automode: monitor only volumes that reside on the hybrid pools
– none: monitoring is turned off
34
© 2010 IBM Corporation
Americas Advanced Technical Skills (ATS)
DSCLI Commands
 Offload the monitor data
offload –etdata c:\et
–
Offloaded monitor data naming convention
• SF75XX101ESS01_heat.data: server 0 data
• SF75XX101ESS11_heat.data: server 1 data
35
© 2010 IBM Corporation
Americas Advanced Technical Skills (ATS)
DSCLI Commands
 Process monitor data
c:\stat\stat -o c:\stat\output
c:\et\SF75XX101ESS01_heat.data
c:\et\SF75XX101ESS11_heat.data
– c:\stat is the folder where the advisor tool resides
– c:\stat\output is the folder where the output will be placed
– Input data must have “_heat” in the name
36
© 2010 IBM Corporation
Americas Advanced Technical Skills (ATS)
Storage Tier Advisor Tool – System Summary
37
© 2010 IBM Corporation
Americas Advanced Technical Skills (ATS)
Advisor Tool Report
Before Migration
38
© 2010 IBM Corporation
Americas Advanced Technical Skills (ATS)
39
© 2010 IBM Corporation
Americas Advanced Technical Skills (ATS)
System Summary Report
 Hot data is 3% of total capacity
 No data allocated/migrated to SSD yet
 Random I/O is 8% of total I/O
– These are random I/Os that need to access the DDM in the rank
• Random read hit I/O are not included in this data
 Predicted performance improvement
– After hot extents are migrated to SSD
– (old_latency – projected_latency) / (old_latency)
– Estimated 2% to 22%
40
© 2010 IBM Corporation
Americas Advanced Technical Skills (ATS)
Latency Time
 I/O service time in the extent pool/rank
 Random read hit is not included
 I/Os included:
– Random read miss
– Sequential read I/Os
• Even though the read hit ratio may be close to 100%
– Random & sequential write I/Os
• Random write will trigger 4 I/Os
– Read & write of the data updated
– Read & write of the parity track
41
© 2010 IBM Corporation
Americas Advanced Technical Skills (ATS)
42
© 2010 IBM Corporation
Americas Advanced Technical Skills (ATS)
43
© 2010 IBM Corporation
Americas Advanced Technical Skills (ATS)
System Summary Report
 Add 1 73GB rank
– Means add 1 rank per server
44
© 2010 IBM Corporation
Americas Advanced Technical Skills (ATS)
Advisor Tool Report
During Migration
45
© 2010 IBM Corporation
Americas Advanced Technical Skills (ATS)
46
© 2010 IBM Corporation
Americas Advanced Technical Skills (ATS)
System Summary Report
 Predicted performance improvement
– After SSD is filled-up with hot extents
– Estimated 0% to 16%
– Estimated that it will still take 7 more hours to migrate the hot extents
47
© 2010 IBM Corporation
Americas Advanced Technical Skills (ATS)
48
© 2010 IBM Corporation
Americas Storage Advanced Technical Skills (ATS)
IBM System Storage DS8700
Performance with Easy Tier
6/10/2010
© 2010 IBM Corporation
Americas Advanced Technical Skills (ATS)
SSD + SATA w/Easy Tier vs. 15KRPM HDDs
192 FC HDD
96 SATA + 16 SSD
Response Time (ms)
Dual frames
Single Frame
15.00
Improves RT in range
of ordinary use
10.00
5.00
0.00
0
10000
20000
30000
40000
50000
60000
Throughput (IO/s)
51
© 2010 IBM Corporation
Americas Advanced Technical Skills (ATS)
SPC-1 Throughput Improvement with Easy Tier
Throughput (IO/s)
60000
50000
40000
> 3 x IOPS
Improvement
30000
20000
10000
0
0:00
2:00
4:00
6:00
8:00
10:00
12:00
14:00
16:00
18:00
Time
52
© 2010 IBM Corporation
Americas Advanced Technical Skills (ATS)
DB2 OLTP Brokerage Transactions w/ Easy Tier
“Typical” I/O Intensity (QD 20)
DB2 OLTP Brokerage Overall Transactions
At Queue Depth 20
7000
DB2 Overall
Transaction/second
6000
5000
Base
4000
3000
EasyTier 16
2000
Easy Tier 32
1000
0
Elaped Time
54
© 2010 IBM Corporation
Americas Advanced Technical Skills (ATS)
Backend Storage Measurements at QD 20
IO Response (ms)
IOPs
60000
10
50000
8
40000
6
30000
4
20000
2
10000
0
0
Base
% Backend IOPs on SSD
Easy Tier Easy Tier
16
32
Backend Read Response (ms)
12
10
8
6
4
2
0
100%
80%
60%
40%
20%
0%
Base Easy TierEasy Tier
16
32
55
Base
Easy Tier Easy Tier
16
32
Base Easy TierEasy Tier
16
32
SSD
HDD
© 2010 IBM Corporation
Americas Advanced Technical Skills (ATS)
DB2 OLTP Brokerage Transactions w/ Easy Tier
“Peak” I/O Intensity (QD 256)
DB2 Brokerage Overall Transactions
At Queue Depth 256
10000
DB2 Overall
Transaction/second
8000
6000
BASE
4000
EasyTier 32
2000
0
Elaped Time
56
© 2010 IBM Corporation
Americas Advanced Technical Skills (ATS)
Backend Storage Measurements at QD 256
IO Response (ms)
IOPs
24
22
20
18
16
14
12
10
8
6
4
2
0
80000
60000
40000
20000
0
Base
Base
Easy Tier 32
% Backend IOPs on SSD
Easy Tier 32
Backend Read Response (ms)
12
10
8
6
4
2
0
100%
80%
60%
40%
20%
0%
Base
Base
Easy Tier 32
SSD
57
Easy Tier 32
HDD
© 2010 IBM Corporation
Americas Advanced Technical Skills (ATS)
DS8000 Easy Tier Summary
 Easy Tier is a DS8000 feature that supports the dynamic relocation of data
using the following methods:
– Automatic Mode
– Manual Mode
 Easy Tier is a no charge feature available with DS8700 and Release 5.1
 Supported on all host attachments with no additional host-based software
 Easy Tier will help customers utilize large capacity drives
 Reduces administrative effort and costs by automating data placement
 New tooling enables clients to see exactly how much their existing
workloads can benefit from how many SSDs
 Helps align storage capacity purchases with performance and cost
requirements
58
© 2010 IBM Corporation