PRISM: Zooming in
Persistent RAM Storage Behavior
Juyoung Jung and Dr. Sangyeun Cho
Dept. of Computer Science
University of Pittsburgh
Juyoung Jung
University of Pittsburgh
Contents





Introduction
Background
PRISM
Preliminary Results
Summary
Juyoung Jung
University of Pittsburgh
Technical Challenge
HDD
- Slow and
- Power hungry
Fundamental solution?
Alternative storage medium
Juyoung Jung
University of Pittsburgh
Technical Challenge
NAND flash based SSD
+ much faster
+ better energy
+ smaller …
- erase-before-write
- write cycles
- unbalanced rw
- scalability …
Juyoung Jung
University of Pittsburgh
Emerging Persistent RAMs Technologies
Latency
Read
Program
energy
Endurance
Density
Write
Erase
NAND flash 25us
200us
1.5ms
10nJ
104~106
4~5F2
PCM
20ns
100ns
N/A
100pJ
108~109
5F2
STT-RAM
10ns
10ns
N/A
0.02pJ
1015
4F2
RRAM
10ns
20ns
N/A
2pJ
106
6F2
FeRAM
75ns
50ns
N/A
2pJ
1013
6F2
PRAMs Win !
Juyoung Jung
University of Pittsburgh
Write endurance
Read/write
imbalance
Latency
Energy
consumption
Technological
maturity
Juyoung Jung
University of Pittsburgh
Our Contribution

There is little work to evaluate Persistent RAM
based Storage Device (PSD)
 Research environment not well prepared

Present an efficient tool to study the impact of
PRAM storage device built with totally different
physical properties
Juyoung Jung
University of Pittsburgh
PRISM

PeRsIstent RAM Storage Monitor

Study Persistent RAM (PRAM) storage behavior
 Potentials of new byte addressable storage device
 PRAMs’ challenges as storage media

Guide PSD (PRAM Storage Device) design
 Measure detailed storage activities (SW/HW)
Juyoung Jung
University of Pittsburgh
PRISM
HDD or SSD
Block devices
PSD
Non-Block devices
Juyoung Jung
University of Pittsburgh
PRISM
test.exe
Low-level storage behavior
read(file1)
address mapping
wear leveling
write(buf,file1)
bit masking
exploit parallelism
resource conflicts
etc.
Juyoung Jung
University of Pittsburgh
PRISM Implementation Approach
User Applications
Virtual File System
SW
Instrumentation
Technique
Tracers
Individual FS (EXT3, HFS, TMPFS, etc.)
Emulation
File system
Buffer/Page Cache
I/O scheduler
Block I/O Layer
HDD/SSD device driver
Tracers
Interconnect
Slots, chips
HW
Simulation
PSD
Juyoung Jung
University of Pittsburgh
PRISM Components
User Apps
Frontend Tracer
Backend PSDSim
Juyoung Jung
University of Pittsburgh
Frontend Tracer
write
Application
File I/O requests
write
write
Linux VFS
In-memory TMPFS file system
Write data
Invoking
Tracer
WR tracer
Module
access time
byte-granule
request info
……
Metadata
Tracer
Module
User-defined
Tracer
Modules
Proc-based Logger
Raw data gathered
Juyoung Jung
University of Pittsburgh
PRISM Components
User Apps
Frontend Tracer
Backend PSDSim
Juyoung Jung
University of Pittsburgh
Backend Event-driven PSDSim
Raw trace data
the used PRAM technology
storage capacity
Storage configuration
# of packages, dies, planes
fixed/variable page size
wear-leveling scheme …
Persistent RAM storage simulator (PSDSim)
PRAM
Reference
Statistics
Analyzer
PRAM
Storage
Controller
Wear leveling
Emulator
PRAM
Energy Model
Various performance results
Juyoung Jung
University of Pittsburgh
Preliminary Results of Case Study


Objective: enhancing PRAM write endurance
Experimental Environment
Components
CPU
L2 Cache
Main Memory
Operating System
File System
Workload
Specification
Intel Core Duo 1.66 GHz (32-bit)
2 MB
1 GB SDRAM
Linux 2.6.24
Tmpfs 10 GB capacity
TPC-C 2-hours measurement
Juyoung Jung
University of Pittsburgh
Possible Questions from Designers

How our workloads generate writes to storage?
1. Temporal behavior of write request pattern
2. The most dominant data size written
3. Virtual address space used by OS
4. File metadata update pattern

What about hardware resource access pattern?
1. Resource utilized efficiently
2. Do we need consider wear-leveling?
Juyoung Jung
University of Pittsburgh
TPC-C File update pattern
Temporal behavior of writes
Populating 9 relational tables
writes are very bursty
proper queue size
(Wall-clock time)
Juyoung Jung
University of Pittsburgh
File Data Written Size
4KB written size dominant
Not cause whole block update
Juyoung Jung
University of Pittsburgh
Virtual Page Reuse Pattern
HIGHMEM
OS VMM allocates pages
from low memory
Juyoung Jung
University of Pittsburgh
Example Hardware Organization
256MB package 0
PKG 0
Die 0
Die 1
1GB PSD
PKG 1
16MB plane 7
8 planes in
128MB Die 1
PKG 2
Plane 7
Plane1
Plane1
Plane1
Plane 0
PKG 3
Juyoung Jung
Page 0
Page 1
Page 2
Page 3
Page N-1
Page N
University of Pittsburgh
Hardware Resource Utilization
4000
Access Count
3500
3000
2500
Die1
2000
1500
1000
Die0
500
0
pkg0
pkg1
Juyoung Jung
pkg2
pkg3
University of Pittsburgh
Storage-wide plane resource usage
1258
600
plane0
plane1
plane2
plane3
plane4
plane5
plane6
plane7
Access Count
500
Q. plane-level usage fair ?
400
300
Serious unbalanced
resource utilization
200
100
0
Die0
Die1
Package 0
Die0
Die1
Package 1
Juyoung Jung
Die0
Die1
Package 2
Die0
Die1
Package 3
University of Pittsburgh
Wear-leveling Effect
Before Wear-leveling
After RR Wear-leveling
Balanced
resource usage
Juyoung Jung
University of Pittsburgh
PRISM Overhead
PRISM’s I/O performance impact
 We ran IOzone benchmark with 200MB (HDD-friendly)
sequential file write operations
4
to PRISM
normalized
Speed
Latency
Normalized

3.5
3.5
3
3
2.5
2.5
2
2
1.5
3.5x slowdown
1.5
1
0.51
0
0.5
Raw tmpfs
PRISM
HDD
0
Raw tmpfs
Juyoung Jung
PRISM
University of Pittsburgh
PRISM Overhead
PRISM’s I/O performance impact
 We ran IOzone benchmark with 200MB (HDD-friendly)
sequential file write operations
144
Speed
normalizedtotoPRISM
Speednormalized
HDD

3.5
12
11.5x faster
3
10
2.5
8
2
6
1.5
4
1
2
0.5
00
Raw tmpfs
PRISM
PRISM
Juyoung Jung
HDD HDD
University of Pittsburgh
PRISM Summary

Versatile tool to study PRAM storage system
 Study interactions between storage level interface
(programs/OS) and PRAM device accesses
 Run a realistic workload fast
 Extendible with user-defined tracer easily
 Explore internal hardware organizations in detail
Juyoung Jung
University of Pittsburgh
Thank you !
Juyoung Jung
University of Pittsburgh