Anti-Caching in Main Memory
Database Systems
Justin DeBrabant
Brown University
debrabant@cs.brown.edu
A bit of history…
1974 – System R
• query optimization
• recovery
• transaction serialization
–lots of locks
Application
Buffer Pool
Primary Storage
Change is Good
1.E+10
USD ($)
1.E+08
1.E+06
1.E+04
1.E+02
1.E+00
1970 1973 1976 1979 1982 1985 1988 1991 1994 1997 2000 2003 2006 2009 2012
source: http://www.archivebuilders.com/whitepapers/22045p.pdf
Great, that’s what the buffer
pool is for...right?
Buffer Pool
31%
26%
31%
Locking
Recovery
12%
Real Work
OLTP Through the Looking Glass,
and What We Found There
SIGMOD ‘08, pp. 981-992, 2008.
What to do with all this memory?
Application
Consistent?
Updates?
Writes?
Parallel Main Memory
Transaction Processing System
H-Store: A High-Performance, Distributed
Main Memory Transaction Processing System
VLDB 2008.
YCSB, Update-Heavy, data < memory
Anti-Caching in H-Store
• Memory becomes primary storage for “hot”
data
• “Cold” data is evicted to disk in blocks, fetched
when requested by a transaction
• Still no locks/latches
Application
Primary Storage
Anti-Cache
YCSB, Update-Heavy, data > memory
15x
The New Traditional Wisdom
+
AntiCaching
>
+
Buffer
Pool
Future Work
• Alternative eviction strategies
• Larger-than-memory queries
• New hardware
– flash, persistent memory
The Team
Justin
Andy
DeBrabant Pavlo
(Brown)
(Brown)
Stephen
Mike
Stan
Tu
Stonebraker Zdonik
(MIT)
(MIT)
(Brown)
Anti-Caching: A New Approach to
Database Management System Architecture
In Preparation.
Questions?
debrabant@cs.brown.edu
hstore.cs.brown.edu