Storage Performance Council (SPC)
www.storageperformance.org
spcadmin@storageperformance.org
Presentations and Speakers

An Introduction to the SPC
Leah Schoeb (Sun Microsystems, Inc.)

SPC-1C Preview
Walter E. Baker (SPC Administrator & Auditor)

SPC-3 Development Status
Mel Boksenbaum (Hitachi Data Systems)

SPC-2 Announcement
Bruce McNutt (IBM Corporation & SPC-2 Chair)

SPC-2 “Wave One” Results
Walter E. Baker (SPC Administrator & Auditor)

Q&A
Sandy Wilson (Fujitsu & SPC Steering Committee Chair)
CMG 2005
Storage Performance Council (SPC)
www.StoragePerformance.org
Page: 2
An Introduction to the SPC
Leah Schoeb
Sun Microsystems, Inc.
CMG 2005
Storage Performance Council (SPC)
www.StoragePerformance.org
Page: 3
Council Mission
The SPC is a non-profit corporation
founded in 1998 to

define, standardize, and promote the first industry
standard storage benchmarks

to disseminate objective, verifiable performance
data

to developers, product managers, and customers
of computer systems
CMG 2005
Storage Performance Council (SPC)
www.StoragePerformance.org
Page: 4
SPC Objectives

Drive performance improvement in the
computer systems industry for storage
Insure that customers can accurately
compare products in a multi-vendor world
 Establish a level “playing
Storage Industry
field” for all manufacturers
Performance Experts



Results
Dissemination
Publicize benchmark
results
Service and
Support
Ensure accuracy and
authenticity of published
results
CMG 2005
Storage Performance Council (SPC)
Test Tools
Benchmark Kits
Performance
Analysis Tools
www.StoragePerformance.org
New Benchmark
Development
SPC
Benchmark
Maintenance
Results Validation
Peer Review
Audit
Page: 5
SPC Organizational Structure
Membership
Full Members
Board of Directors
- Elects Steering Committee
- Approves Standards
- Reviews Compliance
Associate
Members
Benchmark
Subcommittees
Steering
Committee-Management
-Direction
- Administration
CMG 2005
Storage Performance Council (SPC)
Administrator
Standing Committees
www.StoragePerformance.org
Academic
Members
No Cost
Page: 6
SPC Membership
CMG 2005
Storage Performance Council (SPC)
www.StoragePerformance.org
Page: 7
SPC Benchmarks
Application Environment
SPC-1
SPC-x
SPC-2
Database
LDQ
CMG 2005
Storage Performance Council (SPC)
www.StoragePerformance.org
Applications are
clustered based on
characteristics of
load presented to
the I/O Subsystem
SPC-x benchmark
will be defined for
each cluster,
starting with SPC-1
Page: 8
SPC-1C Preview
Walter E. Baker
Gradient Systems, Inc.
SPC Administrator and Auditor
CMG 2005
Storage Performance Council (SPC)
www.StoragePerformance.org
Page: 9
SPC-1 and storage components

There is a distinct need to provide product
performance comparison for storage
configuration components such as disk
drives, HBAs/controllers, small storage
subsystems (single enclosure), etc.

SPC-1 provides that type of product
performance comparison, but is typically
used for large, complex storage
configurations
CMG 2005
Storage Performance Council (SPC)
www.StoragePerformance.org
Page: 10
SPC-1C (Component)

SPC-1C will be the first industry-standard
performance benchmark to provide product
performance comparisons for individual
storage components such as disks,
HBAs/controllers, small storage
subsystems (single enclosure), storage
software, etc.

SPC-1C will be identical to SPC-1 in
workload and reporting requirements.

SPC-1C will have reduced audit and filing
fees.
CMG 2005
Storage Performance Council (SPC)
www.StoragePerformance.org
Page: 11
SPC-1C Producer Audience

Storage component vendors comprise the
primary producer audience for SPC-1C.

SPC-1C will provide performance evaluation
and comparison for storage components such
as:
• Disk drives
• HBAs/controllers
• Small Storage Subsystems (single enclosure)
• Processors used in the above components
• Storage software such as Logical Volume Managers
CMG 2005
Storage Performance Council (SPC)
www.StoragePerformance.org
Page: 12
SPC-1C Consumer Audiences

Storage solution providers (vendors, resellers,
etc.), as well as end users, are the primary
consumer audiences for SPC-1C.

SPC-1C will allow storage solution providers
to evaluate the storage components that
comprise the larger storage solution offered by
the provider.
• Disks used in an array
• HBAs/Controllers
• Small Storage Subsystems
• Logical Volume Managers
CMG 2005
Storage Performance Council (SPC)
www.StoragePerformance.org
Page: 13
SPC-1C Configuration Components

SPC-1C configurations will consist of the
following two components:
• One HBA/Controller (single/dual/quad…).
• One of the following disk drive configurations:
– One, two, or four disks in a “standalone” configuration.
» An external enclosure may be used but only to
provide power and/or connectivity for the disks.
– A small storage subsystem (any form factor) with a
maximum of sixteen disks.
CMG 2005
Storage Performance Council (SPC)
www.StoragePerformance.org
Page: 14
SPC-1C Availability Test

An optional SPC-1C Availability Test will
replace the required SPC-1 Persistence Test.

Applicable in SPC-1C configurations that
provide uninterrupted operation and “hot”
rebuild/recovery in the case of a single
physical drive failure.
CMG 2005
Storage Performance Council (SPC)
www.StoragePerformance.org
Page: 15
SPC-1C Availability Test

The SPC-1C Availability Test will report:
• If processing continues in the case of a single
physical drive failure.
• The level of performance when operating in
“degraded mode” after the failure.
• The time required to recover from the failure.
• The level of performance during the recovery.
CMG 2005
Storage Performance Council (SPC)
www.StoragePerformance.org
Page: 16
SPC-3 Development Status
Mel Boksenbaum
Hitachi Data Systems
CMG 2005
Storage Performance Council (SPC)
www.StoragePerformance.org
Page: 17
Description of effort
A protocol independent benchmark that
measures storage performance as seen by
applications

SPC-1 and SPC-2 characterize lower level
components in the storage hierarchy
• Storage from the bottom up

SPC-3 will take the end-user view of the
virtualized storage environment.
• Storage from the top down
CMG 2005
Storage Performance Council (SPC)
www.StoragePerformance.org
Page: 18
Environments and workloads
File Services

Expansion/Business Use

Traversal/Backup
Database

OLTP

Decision Support
Workflow
CMG 2005
Storage Performance Council (SPC)

Refinement/Digital Film Editing

Traversal/Playback
www.StoragePerformance.org
Page: 19
Critical Questions
Draft Workload Definition available

www.storageperformance.org
Is a measurement at the application level useful?
What factors make it interesting?

Scale? Volume, users, bandwidth

Constraints? Price, latency, protocol

Topology? NAS, SAN, DAS
Your input is invited and eagerly awaited!
spc3-input@storageperformance.org
CMG 2005
Storage Performance Council (SPC)
www.StoragePerformance.org
Page: 20
SPC-2 Announcement
Bruce McNutt
IBM Corporation
Chair, SPC-2 Technical Subcommittee
CMG 2005
Storage Performance Council (SPC)
www.StoragePerformance.org
Page: 21
Announcing SPC-2
(Dec 6, 2005)

New test of storage system performance for
environments that require large scale data
movement.

Built using a series of application-oriented
sequential workloads.

Audited results will be posted today on the
SPC web site for 6 tested storage products.

Available now to SPC members/associate
members.
CMG 2005
Storage Performance Council (SPC)
www.StoragePerformance.org
Page: 22
SPC-1: Predominately random access
Represents environments such as OLTP, database, and
mail server:
CMG 2005
Storage Performance Council (SPC)
Read
Write
All
(pct)
(pct)
(pct)
Random
29
32
61
Sequential
11
28
39
All
40
60
100
www.StoragePerformance.org
Page: 23
SPC-2: Large Block, Sequential I/O
Sequential tests: valuable to many customers

Customers with dedicated servers.
• Data warehouse & Data mining
• Scientific Processing
• Media

Customers with a mix of applications including
sequential.
• Banking & Financial, Insurance, Manufacturing,
Medical, Government, many others…
• To such customers, the sequential component of
performance is often the most visible.
CMG 2005
Storage Performance Council (SPC)
www.StoragePerformance.org
Page: 24
SPC-2: A collection of sequential tests
Why a collection?
•
Types of sequential processing can be very
different.
•
It is easy to include a variety of tests.
•
Since the hardware imposes a ceiling on all data
rate measurements, a variety of measurements will
tend to reinforce each other.
CMG 2005
Storage Performance Council (SPC)
www.StoragePerformance.org
Page: 25
Sequential processing tested in SPC-2
Large file processing:
simple sequential processing of one or more large files.
Large database queries:
scans or joins of large relational tables.
Video on demand:
individualized playback from a digital film library.
CMG 2005
Storage Performance Council (SPC)
www.StoragePerformance.org
Page: 26
SPC-2 measurements (Example)
Large File Processing - Data Rate
Large Database Query - Data Rate
5,000
5,000
4,500
Data Rate, MB/sec
Data Rate, MB/sec
4,000
3,500
3,000
2,500
2,000
1,500
1,000
500
0
Write 256KiB Read/Write Read 256KiB
256KiB
1 Stream
12 Streams
Write
1024KiB
24 Streams
Read/Write
1024KiB
48 Streams
4,500
4,000
3,500
3,000
2,500
2,000
1,500
1,000
500
0
64KiB w/ 1
IO/Stream
Read
1024KiB
1 Stream
96 Streams
64KiB w/ 4
IOs/Stream
12 Streams
1024KiB w/ 1
IO/Stream
24 Streams
48 Streams
1024KiB w/ 4
IOs/Stream
96 Streams
6,000
4,000
2,000
MI
4400 Streams
0
0:00
0:05
0:10
0:15
0:20
0:25
0:30
0:35
0:40
0:45
0:50
0:55
1:00
1:05
1:10
1:15
1:20
1:25
1:30
1:35
1:40
1:45
1:50
1:55
2:00
2:05
2:10
2:15
2:20
Data Rate, MB/sec
Data Rate -- Video On Demand
Time within Test Run, min
CMG 2005
Storage Performance Council (SPC)
www.StoragePerformance.org
Page: 27
Watch This Space

Additional results are expected to
accumulate rapidly

Watch for them on the SPC Web site:
• http://www.StoragePerformance.org

Aiming for release to non-SPC members
in 1H2006
CMG 2005
Storage Performance Council (SPC)
www.StoragePerformance.org
Page: 28
SPC-2 “Wave One” Results
Walter E. Baker
Gradient Systems, Inc.
SPC Administrator and Auditor
CMG 2005
Storage Performance Council (SPC)
www.StoragePerformance.org
Page: 29
SPC-2 “Wave One” Results

SPC-2 “Wave One” results submitted
by four member companies:
• Fujitsu Limited
• Hewlett-Packard Company
• IBM Corporation
• Sun Microsystems, Inc.
CMG 2005
Storage Performance Council (SPC)
www.StoragePerformance.org
Page: 30
SPC-2 “Wave One” Results

Six storage products:
• Fujitsu Storage Systems ETERNUS 6000 Model 900
• HP StorageWorks 8000 Enterprise Virtual Array
• IBM TotalStorage SAN Volume Controller 3.1
• IBM TotalStorage DS8300
• Sun StorEdge 6130 Array
• Sun StorEdge 3510 FC Array
CMG 2005
Storage Performance Council (SPC)
www.StoragePerformance.org
Page: 31
SPC-2 Reported Data
SPC-2 Reported Data
Priced Storage Configuration Name:
SPC-2 MBPS™
SPC-2
Price-Performance
Total Price
ASU Capacity
Data
Protection Level
The above SPC-2 MBPS™ value represents the aggregate data rate of all three SPC-2 workloads:
Large File Processing (LFP), Large Database Query (LDQ), and Video On Demand (VOD)
SPC-2 Large File Processing (LFP) Reported Data
Number of
Data Rate
Data Rate
Streams
per Stream
Price-Performance
LFP Composite
Write Only:
1024 KiB Transfer
256 KiB Transfer
Read-Write:
1024 KiB Transfer
256 KiB Transfer
Read Only:
1024 KiB Transfer
256 KiB Transfer
The above SPC-2 Data Rate value for LFP Composite represents the aggregate performance of all three LFP Test
Phases: (Write Only, Read-Write, and Read Only).
SPC-2 Large Database Query (LDQ) Reported Data
Number of
Data Rate
Data Rate
Streams
per Stream
Price-Performance
LDQ Composite
1024 KiB Transfer Size
4 I/Os Outstanding
1 I/O Outstanding
64 KiB Transfer Size
4 I/Os Outstanding
1 I/O Outstanding
The above SPC-2 Data Rate value for LDQ Composite represents the aggregate performance of the two LDQ Test
Phases: (1024 KiB and 64 KiB Transfer Sizes).
SPC-2 Video On Demand (VOD) Reported Data
Number of
Data Rate
Data Rate
Streams
per Stream
CMG 2005
Storage Performance Council (SPC)
www.StoragePerformance.org
Price-Performance
Page: 32
SPC-2 Reported Data

Overall Composite:
• Large File Processing (LFP)
• Large Database Query (LDQ)
• Video on Demand Delivery (VOD)
SPC-2 Reported Data
Priced Storage Configuration Name:
SPC-2 MBPS™
SPC-2
Price-Performance
Total Price
ASU Capacity
Data
Protection Level
The above SPC-2 MBPS™ value represents the aggregate data rate of all three SPC-2 workloads:
Large File Processing (LFP), Large Database Query (LDQ), and Video On Demand (VOD)
CMG 2005
Storage Performance Council (SPC)
www.StoragePerformance.org
Page: 33
SPC-2 Reported Data
SPC-2 Reported Data
Priced Storage Configuration Name:
SPC-2 MBPS™
SPC-2
Price-Performance
Total Price
ASU Capacity
Data
Protection Level
The above SPC-2 MBPS™ value represents the aggregate data rate of all three SPC-2 workloads:
Large File Processing (LFP), Large Database Query (LDQ), and Video On Demand (VOD)
• SPC-2 MBPS™:
– 263.01 / 343.72 / 1,025.62 / 2,129.47 / 3,217.53 / 3,517.75
• SPC-2 Price-Performance ($ per SPC-2 MBPS™):
– $137.52 / $201.41 / $384.31 / $415.37 / $563.93 / $679.23
• Range of disk devices (spindles) configured:
– 14 / 36 / 112 / 280 / 512 / 672
CMG 2005
Storage Performance Council (SPC)
www.StoragePerformance.org
Page: 34
SPC-2 Reported Data

Large File Processing (LFP)
SPC-2 Large File Processing (LFP) Reported Data
Number of
Data Rate
Data Rate
Streams
per Stream
Price-Performance
LFP Composite
Write Only:
1024 KiB Transfer
256 KiB Transfer
Read-Write:
1024 KiB Transfer
256 KiB Transfer
Read Only:
1024 KiB Transfer
256 KiB Transfer
The above SPC-2 Data Rate value for LFP Composite represents the aggregate performance of all three LFP Test
Phases: (Write Only, Read-Write, and Read Only).
• SPC-2 LFP Composite Data Rate (MB/s):
– 221.66 / 248.33 / 833.19 / 1,875.53 / 2,573.57 / 2,707.28
• SPC-2 Price-Performance ($ per MB/s):
– $163.19 / $278.77 / $471.61 / $473.06 / $732.76 / $849.18
CMG 2005
Storage Performance Council (SPC)
www.StoragePerformance.org
Page: 35
SPC-2 Reported Data

Large Database Query (LDQ)
SPC-2 Large Database Query (LDQ) Reported Data
Number of
Data Rate
Data Rate
Streams
per Stream
Price-Performance
LDQ Composite
1024 KiB Transfer Size
4 I/Os Outstanding
1 I/O Outstanding
64 KiB Transfer Size
4 I/Os Outstanding
1 I/O Outstanding
The above SPC-2 Data Rate value for LDQ Composite represents the aggregate performance of the two LDQ Test
Phases: (1024 KiB and 64 KiB Transfer Sizes).
• SPC-2 LDQ Composite Data Rate (MB/s):
– 323.60 / 389.60 / 1,118.66 / 2,273.09 / 3,697.33 / 4,389.76
• SPC-2 Price-Performance ($/MB/s):
– $111.78 / $177.69 / $352.34 / $394.92 / $452.32 / $591.08
CMG 2005
Storage Performance Council (SPC)
www.StoragePerformance.org
Page: 36
SPC-2 Reported Data

Video on Demand Delivery (VOD)
SPC-2 Video On Demand (VOD) Reported Data
Number of
Data Rate
Data Rate
Streams
per Stream
Price-Performance
• Number of Streams:
– 310 / 500 / 1,500 / 2,848 / 4,300 / 4,400
• SPC-2 VOD Data Rate (MB/s):
– 243.78 / 393.22 / 1,124.99 / 2,239.77 / 3,381.70 / 3,460.23
• SPC-2 Price-Performance ($/MBs):
– $148.37 / $176.06 / $350.36 / $394.92 / $573.31 / $646.25
CMG 2005
Storage Performance Council (SPC)
www.StoragePerformance.org
Page: 37
SPC Q&A
Sandy Wilson
Fujitsu Computer Systems Corporation
Chair, SPC Steering Committee
CMG 2005
Storage Performance Council (SPC)
www.StoragePerformance.org
Page: 38
SPC Q&A

Why are there not more SPC member
companies releasing SPC-2 results for this
announcement?

What type of testing was done, prior this
initial release of the benchmark
specification?

Why is the benchmark only available to
SPC members and not to the general
public?
CMG 2005
Storage Performance Council (SPC)
www.StoragePerformance.org
Page: 39