Anshul Gandhi (Carnegie Mellon University)
Varun Gupta (CMU), Mor Harchol-Balter (CMU)
Michael Kozuch (Intel, Pittsburgh)
Requests
Server Farm


Server farms are important for today’s IT
infrastructure (Amazon, Google, IBM, HP, …)
However, server farms cost a lot of money to
power ($4 billion in 2006)
Requests
Server Farm

How many servers, given request rate ?
 Don’t want to waste power
1.
Server farm model
2.
Provisioning for fixed arrival rate
3.
Provisioning for unpredictable, time-varying
arrival rate
4.
Future work
BUSY
BUSY
BUSY
IDLE
IDLE servers consume
a lot of power
~ 60 % of BUSY
IDLE
OFF
OFF
5
BUSY
BUSY
Turn IDLE servers
OFF to save power
BUSY
OFF
HOWEVER
OFF
OFF
OFF
6
To turn on an OFF server ..
OFF
SETUP
BUSY
Time delay (setup time)
• 1 min – 5 mins
and
Power penalty
• peak power during setup time
7
To turn on an OFF server ..
OFF
SETUP
BUSY
Should we ever turn servers OFF ?
8

ON

Server states:
BUSY
PBUSY 240 W
IDLE
PIDLE 150 W
OFF
POFF
0W
SETUP PSETUP 240 W
Setup times:
TOFF→ON
TON→OFF
200 s
0s
Intel Xeon E5320
• 2 X 1.86 GHz quad-core
• 4GB memory
9
Requests
FCFS



Server Farm
Poisson arrival process: λ(t) requests/sec
Exponentially distributed job sizes: E[S] secs
Load: ρ(t) = λ(t) ∙ E[S]
Minimum # servers to handle incoming load
10

Interested in response time and power
conumption

Perf/W = 1/(Mean RT X Mean Power)

Maximize Perf/W
11
1.
Server farm model
2.
Provisioning for fixed arrival rate
3.
Provisioning for unpredictable, time-varying
arrival rate
4.
Future work
Poisson arrivals
Server Farm
Max. Perf/W


Existing solutions: prediction based, reactive
controllers.
Is there a simple, yet, near-optimal solution ?
13
Keep n servers always ON (M/M/n)


Servers are BUSY or IDLE
n*    
14
15
Turn servers OFF when IDLE


Servers are BUSY, OFF or in SETUP
n*  
Auto-scales
if n is high
16
17
TON→OFF < γ E[S]/√ρ
18


Best of {NEVEROFF, INSTANTOFF} is optimal
for single-server
Multi-server ?
For ρ > 10, we are
within 20% of OPT
19
1.
Server farm model
2.
Provisioning for fixed arrival rate
3.
Provisioning for unpredictable, time-varying
arrival rate
4.
Future work

Data center demand
has daily variations

INSTANTOFF can auto-scale
21
NEVEROFF requires continual updates based
on predicted load


Predictions are not always accurate
Can we find a simple traffic-oblivious policy?


Auto-scaling in nature
22


Like INSTANTOFF, except we wait for twait
seconds before turning IDLE servers OFF
Routing ?
MRB routing is crucial !
23

Rule of thumb: twait ∙ PIDLE = TOFF→ON ∙ PON
24
Worse at higher
frequencies
25

1998 World Cup Soccer trace (ITA)
26
1.
Server farm model
2.
Provisioning for fixed arrival rate
3.
Provisioning for unpredictable, time-varying
arrival rate
4.
Future work
Experimental evaluation of proposed schemes


Preliminary experiments on 15-server testbed using
CPU-bound workload and sinusoidal arrival pattern
Experimental results agree with analysis
Web workloads:


▪

What does the experimental setup look like ?
Try out various arrival traces and workloads
28

Anshul Gandhi, Varun Gupta, Mor Harchol-Balter, Michael Kozuch
Optimality analysis of energy-performance trade-off for server
farm management, PERFORMANCE 2010

Anshul Gandhi, Mor Harchol-Balter, Ivo Adan
Server farms with setup costs, PERFORMANCE 2010

Anshul Gandhi, Varun Gupta, Mor Harchol-Balter, Michael Kozuch
Energy-efficient dynamic capacity provisioning in server farms,
CMU technical report CMU-CS-10-108
29