CSE880
Professor Pramanik
Shayne Rashedan & Chongguang Bi
1

●
●
●
●
●
●
●
Background
Motivation and Challenges
Examples
Architecture
Schema Level Partitioning
Live Database Migration
Evaluation
2

3

● Software as Service , Apps are located in Cloud, software experiences are delivered through the internet
● Platform as Service , Internet based platforms for developers who want to create services but don’t want to build their own cloud
● Infrastructure as Service , Cloud users rent storage, computation and maintenance from cloud providers (pay per use utilities)
● Data as Service, Infrastructure for web-scale data mining and knowledge discovery
4

● Large numbers of application databases (or tenants) with diverse schemas, unpredictable load and transactions.
● Scaling out using clusters of commodity servers and sharing resources among tenants (multitenancy)
● Pay-Per-Use: paying a small subscription fee on per usage basis every time the software is being used (e.g. hourly)
● Elastic Scaling : When the load increases, add more servers to the database tier to distribute the load, and vice versa.
● Fault-tolerant , self-managing and highly available to support mission critical applications
5

ElasTranS: Elastic, Scalable, Self-Managing
6

:
● Key-value stores
Bigtable, PNUTS and Dynamo, serving hundreds of thousands of concurrent requests
● Schema Level Partitioning
Microsoft SQL Azure ,Relational Cloud and a prototype from Yahoo!, standard
RDBMSs (such as SQL Server or MySQL)
●
Decoupled shared storage
Project Deuteronomy and Hyder, persistent data layer is separate from the logic that executes operations on that data,on the other hand in Amazon EC2, each node is data node and the transaction processor
7

:
● Live data migration
VM pages, iterative phase-by-phase copying pages, migrates live processes and network connections using OS and networking primitives
Zephyr, for shared nothing database architectures such as Cloud SQL
Server or Relational Cloud. The persistent data is stored in disks locally attached to every node. Hence, the persistent data must also be migrated
● Advanced transactions at Scale ,
Megastore, Percolator, ecStore and G-Store
Sinfonia, two phase commit (2PC) protocol to guarantee atomicity of operations in a distributed set of nodes
8

Transaction
Manager
Server
Owning
Transaction
Manager
Append-only Multiversion Data
Partitions of Data
Row Groups
9

User / Client
Routing
Routing
Routing
Routing
OTM
Routing
OTM
Routing
OTM
OTM
Data Storage
Data Storage
Data Storage
Data Storage
TM Master
10

11

12

13

14

15

When amounts of data and transactions are large, assign some jobs to more nodes….
Assign all data and transactions related to Michigan State University Football Team to another node?
How??
16

MSU
UM
MSU
UM
Spartan Stadium
Michigan Stadium
MSU
MSU
UM
UM
Partition Keys
Player 1
Player 2
Player 3
Player 4
Information 1
Information 2
Information 3
Information 4
MSU
MSU
MSU
MSU
UM
UM
UM
UM
Player 1
Player 1
Player 2
Player 2
Player 3
Player 3
Player 4
Player 4
Game 1
Game 2
Game 3
Game 4
Game 5
Game 6
Game 7
Game 8
...
...
Score
...
...
...
...
...
17

Problem 1
A Read
Manager
Distribute
Transactions?
Data
Read
B
Problem 2 Read Lock?
Write Lock?
Read
Manager Data
A
Write
B
18

Problem 1
Just Read Snapshot!
A Read
OTM Data
Read
B
Problem 2
Just Read Snapshot!
Read
A
B
Write
OTM Data
While “A”s are reading,
B can update database.
19

Source Ownership
Steady State 1.Begin Migration 2.Iterative Copying
Destination Ownership
3.Atomic Handover Steady State
Clients
TM Master
20

● Minimum node unavailability achieved by iteratively copying database cache and state of active transactions
●
Elastic Load Balancing, rule-based model, if OTMs load is above the threshold, new OTMs are added, if load is below a min-threshold, partitions are consolidated into a smaller number of nodes.
● Partition assignment algorithm optimization problem with Greedy
Heuristic based on load statistics of partitions
● Atomic ownership handover , ensure efficient and ACID execution of transactions, transactions must execute during migration, synchronizing commit logs of two nodes
● Aborted transactions , minimize loss of data, restart aborted transactions in destination
● Failure Handling, how to recover migration state after failure
21

Data
Data
OTM
User
User
User
TM Master
OTM
Data
OTM
OTM
Data
22

Evaluation
Benchmarks:
Yahoo! Cloud Serving Benchmark
Key-value stores
3 tables, 10 VARCHAR(100) column per table
TPC-C Benchmark
9 tables describing wholesale supplier
5 Transactions: NewOrder, Payment, OrderStatus,
Delivery, StockLevel
23

Evaluation
Comparing with Albatross:
S&M: Stop and Migrate stop serving, flush, transfer ownership
F&M: Flush and Migrate flush, stop serving, transfer ownership
24

Data Data
Data
Migration
OTM
User
User
User
TM Master
OTM
Data
Data
User
OTM
OTM
Data
25

Albatross:
Low Unavailability
Low Latency
26

Albatross:
Almost no additional cost for read operations
Migration costs much time
27

Albatross:
Low Latency
Long Migration Time
28

Not Enough Scale
Scale is Large Enough
Load is Low
Migrate to Low Scale
29