Online Algorithms
Advanced Seminar A
Supervisor: Matya Katz
Ran Taig, Achiya Elyasaf
December, 2009
Overview-part A

Introduction
• What is Online Algorithm?
• Evaluating the Online Algorithm
• The Online Paging Problem & Algorithm
• Deterministic Online Algorithms For Paging
• CA ≥ k Theorem

Adversary Models
• Randomize Online Paging Algorithm
• Oblivious Adversary
• Adaptive Adversary
2
Introduction
What is Online Algorithm?

So far, all algorithms received their entire
inputs at one time

Online algorithms (OA), receive and process
the input in partial amounts.

A sequence of requests are received and the
OA must service each request before it
receives the next one.
3
Introduction
What is Online Algorithm? (Cont.)
In servicing each request, a several
alternatives with an associated cost are
possible.
 The alternative chosen may influence the
costs of alternatives on future requests.


Examples:
• data-structuring
• resource-allocation in operating systems
• finance
• distributed computing
4
Introduction
Evaluating the Online Algorithm

It is often meaningless to have an absolute
performance measure for an algorithm.
•
The algorithm can be forced to incur an unbounded cost
by appropriately choosing the input sequence

It is difficult, if not impossible, to perform a
comparison of competing strategies.


Q: How to evaluate the algorithm?
A: We compare the total cost of the OA on a
sequence of requests, to the total cost of an offline
algorithm

We refer to such an analysis as a competitive
analysis
5
Introduction
The Paging Problem
Some definitions:
• Memory item - a page of virtual memory
• Cache memory – a fast memory of size of k memory items
• Main memory - a slower memory that can potentially hold
•
•
•
•
an infinite number of items
A paging algorithm - decides which k items to retain in the
cache at each point in time
We have a sequence of requests, each of which specifies a
memory item.
A hit – the requested item is in the cache. There is no cost
A miss – the item must be fetched from the main memory.
There is a ‘unit’ cost and one of the k items in the cache
must be evicted
Paging – The replacement of one page with another in the
cache is called paging or page fault
6
Introduction
The Paging Algorithm

The cost measure is the number of misses on a
sequence of requests

This cost depends on the algorithm that decides
which k items to retain in the cache at each
point in time

When a page fault happens, the paging
algorithm invoke an eviction rule for deciding
which item currently in the cache should be
evicted to make room for the new item

Intuitively, items that will be requested again in
the near future should not be evicted
7
Introduction
Offline Algorithm

The offline algorithm (aka the MIN
algorithm): on a miss, evict that item whose
next request occurs furthest in the future

The worst-case number of misses on a
request sequence of length N is N/k.
8
Introduction
Deterministic Online Algorithms For Paging

Least Recently Used (LRU): evict the item in
the cache whose most recent request
occurred furthest in the past.

First-in, First-out (FIFO): evict the item that
has been in the cache for the longest period

Least Frequently Used (LFU): evict the item
in the cache that has been requested least
often
9
Introduction
Deterministic Online Algorithms For Paging (Cont.)

Let p1, p2,...,pn be a request sequence
presented to an online paging algorithm A

Let fA(p1, p2,...,pn) denote the number of
times A misses on p1, p2,...,pn

Let f0(p1, p2,...,pn) denote the minimum
number of misses on p1, p2,...,pn
10
Introduction
Deterministic Online Algorithms For Paging (Cont.)

A deterministic online paging algorithm A is
said to be C-Competitive if there exists a
constant b such that on every sequence of
requests p1, p2,...,pn:
fA(p1, p2,...,pn) - C∙f0(p1, p2,...,pn) ≤ b
where the constant b must be independent
of N but may depend on k

Competitiveness measures the performance
of an OA in terms of the worst-case ratio of
its cost to that of the optimal offline
algorithm
11
Introduction
Deterministic Online Algorithms For Paging (Cont.)

The competitiveness coefficient of A, denoted CA, is
the infimum of C such that A is C-competitive

Since the worst case of offline algorithm is N/k, no
deterministic online paging algorithm has
competitiveness coefficient smaller than k

LRU, FIFO are known to be k-competitive

No deterministic online paging algorithm has
competitiveness coefficient smaller than k (proof will
follow soon…)

Thereby, LRU and FIFO are optimal deterministic OA
12
Introduction
Paging Algorithm - Formally

A paging algorithm consists of an automaton
with a finite set S of states

The automaton response is defined by
function F : S  k  ITEMS  ITEMS  S  k  ITEMS
curr. state

curr. cache
new item
curr. state
curr. cache
The cache after the request is serviced must
include the requested item
13
Introduction
CA ≥ k Theorem
Let A be a deterministic online algorithm for paging then CA ≥ k
Proof:
Initialization  Both A & the offline algorithm are managing different caches for
the same request sequence

They both has the same k-items in the cache

First request is to an item not in either cache, and the
algorithms incur a miss

Let S be the set of k+1 items consisting of the initially k items
together with the new item

From then on, every request is for the unique item in S not in
A's cache
• Thus A misses on every request
14
Introduction
CA ≥ k Theorem (cont.)

A round is a maximal sequence of requests in
which at most k distinct items are requested; each
of these items may be requested any number of
times and in any order

A round ends when, after k distinct items have
been requested, a new item p is requested, and p
then becomes the first request of the next round

Since the round contains at least k requests and A
misses on every one of them, it misses at least k
times during the round
15
Introduction
CA ≥ k Theorem (cont.)

There is an offline algorithm that misses only
once during a round, in fact on the first
request of the round

We denote p as first request of the following
round

When the offline algorithm misses on the
first request, it evicts p and thereby ensures
that there are no further misses in that
round (as expected from a MIN algorithm) 16
Introduction
CA ≥ k Theorem (cont.)

The offline algorithm knows A’s initial cache,
the entire request sequence in advance and
the identity of p for every round

At the end of each round, both the online
algorithm and the offline algorithm have the
same set of items in their caches

This construction can be repeated many
times, proving that there are arbitrarily long
sequences on which A has k times as many
misses as the offline algorithm.
17
Introduction
Conclusions
The proof uses only the fact that the OA
doesn’t know future requests
 Thus the lower bound applies to any
deterministic OA without any regard for its use
of computational resources such as time or
space
 This is a negative result for the online
algorithms


The offline algorithm may be view as an
adversary who is not only managing a cache,
but is also generating the request sequence
18
Overview-part A

Introduction
• What is Online Algorithm?
• Evaluating the Online Algorithm
• The Online Paging Problem & Algorithm
• Deterministic Online Algorithms For Paging
• CA ≥ k Theorem

Adversary Models
• Randomize Online Paging Algorithm
• Oblivious Adversary
• Adaptive Adversary
19
Adversary Models
Randomize Online Paging Algorithm

The Adversary Models, where in collusion
with a reference algorithm that is the
yardstick against which the competitiveness
of the given online algorithm is being
measured

The adversary's goal is to increase the cost
to the given online algorithm, while keeping
it down for the reference algorithm
20
Adversary Models
Randomize Online Paging Algorithm

2 definitions:
• R- a randomize online paging algorithm (=ROA)
• fR(p1, p2,...,pn) – a random variable, denotes the
number of times that R misses on the sequence

Evaluating the ROA
• We still study the behavior of R when the sequence
•
of requests is generated by an adversary
However, there is no longer a unique notion of an
"adversary" for a randomized online algorithm
21
Adversary Models
Oblivious Adversary

The weakest adversary knows R in advance, but has
no knowledge of the random choices made by R

This adversary calculates the “worst case” request
sequence for R, regardless of the actual execution of
R

The fixed cost of this sequence is not a random
variable and is denoted by
f0(p1, p2,...,pn)

We call such an adversary an oblivious adversary,
reflecting that the adversary is oblivious to the
random choices made by R
22
Adversary Models
Oblivious Adversary

R is C-competitive against the oblivious
adversary if for every sequence of requests
p1, p2,...,pn:  f R  p1, p2 ..., pn   C  f 0  p1 , p2 ..., pn   b
for a constant b independent of N.

The oblivious competitiveness coefficient of
R, denoted by CRobl , is the infimum of C such
that R is C-competitive
23
Adversary Models
Adaptive Adversary

The Adaptive Adversary chooses pi+1 after
observing the responses of the ROA to
p1, p2,...,pi

To define the cost of the optimal algorithm it
might help to think of the adaptive
adversary and the optimal algorithm as
working in collusion
24
Adversary Models
Adaptive Adversary

First scenario:
• The adversary generates the optimal strategy
adaptively by learning p1, p2,...,pi
• We refer to this as the adaptive offline
adversary
• The request sequence is a random sequence.
Thus, f0(p1, p2,...,pn) and fR(p1, p2,...,pn) are
random variables
25
Adversary Models
Adaptive Adversary

Second scenario:
• The adversary works as before, but also required to
concurrently manage a cache online
• Meaning, the adversary generates pi+1 based on the
responses of R to p1,p2,...,pi, and immediately
exhibits its own response to pi+1
• Again both f0(p1, p2,...,pn) and fR(p1, p2,...,pn) are
random variables
• We refer to such an adversary as an adaptive online
adversary
26
Adversary Models
Adaptive Adversary

We say that R is C-competitive against the
adaptive offline adversary if for a constant b
independent of N: E f R  p1 , p2 ..., pn   C  E f 0  p1, p2 ..., pn   b

The adaptive offline competitiveness
aof
coefficient of R, C R , is the infimum of C
such that R is C-competitive

Likewise, C R is the adaptive online
competitiveness coefficient of R
aon
27
Adversary Models
Adaptive Adversary

Clearly, we can define the following proportion –
CRobl  CRaon  CRaof

Let C obl be the lowest oblivious competitive
coefficient of any randomized paging algorithm

Similarly we define

Finally we define C det to be the lowest
competitive coefficient of any deterministic
paging algorithm

Then we have
C aon C aof
C obl  C aon  C aof  C det
28
Overview-part B

Introduction
•
•
•

Proof of a Theorem on CR :
•
•
•
•

Preparations.
Behavior of the offline algorithm.
Behavior of a deterministic algorithm.
Results.
The marker algorithm
•
•

Paging against an oblivious – Definitions.
The players on this section – What is the goal?
Yao’s minimax theorem.
Description.
Proof of it’s competitiveness coefficient.
Summary
29
Paging against an oblivious adversary

Remember that in the proof of the result :
CA ≥ k we based our analysis on the fact that
the request sequence was determined at
each step according to the behavior (the
cache contents) of the deterministic online
algorithm

When talking about randomized algorithm
it’s intuitive that this ability of the offline
algorithm won’t be so helpful
30
Our goal now

We now want to prove that against an oblivious
adversary a randomized online algorithm can do
much better in terms of competitiveness against the
result (K) we saw when we talked about
deterministic online algorithm.

We will see that under the above assumptions .
CRobl  Hk where H k is the k’th harmonic number
which is clearly smaller then K.
31
So… who against who?

Our players now are an oblivious adversary
which specifies all the request sequence in
advance and never changing it later on

Then, an optimal offline algorithm is
activated on this sequence and announcing
it’s result

Only then – the sequence is given to the
random online algorithm we want to test –
each request at a time.
32
A reminder : Yao’s MiniMax principle

Yao's minimax principle states that given
any arbitrarily chosen input distribution P:
the expected cost of the optimal
deterministic algorithm under this
distribution is a lower bound on the
expected cost of the optimal randomized
(Las Vegas) algorithm.
33
How to use this principle for our
purposes?

The principle gives us the ability to deal only
with deterministic algorithms in order to give
a lower bound for the randomized algorithm.

All we need to do is to choose a probability
distribution for the inputs and then give a
lower bound for the best (in terms of
expected cost) deterministic online algorithm
under this distribution.
34
Overview-part B

Introduction
•
•
•

Proof of a Theorem on CR :
•
•
•
•

Preparations.
Behavior of the offline algorithm.
Behavior of a deterministic algorithm.
Results.
The marker algorithm
•
•

Paging against an oblivious – Definitions.
The players on this section – What is the goal?
Yao’s minimax theorem.
Description.
Proof of it’s competitiveness coefficient.
Summary
35
Some preparations:
The probability distribution on the inputs can
be taught as the probability to choose the
i’th memory item on the sequence – this
probability can depend in the older requests
on the chosen sequence;
 Given a deterministic online paging
algorithm A we define it’s competitiveness
under a distribution ρ to be the infimum of C
such that:
E f A  p1, p2 ..., pn   C  E f 0  p1, p2 ..., pn   b

36
Cont.

Formally – the MiniMax principle gives us:
inf
R
obl
C
R
 sup (inf (CA ))
P
P
A

Meaning of the left member:
The competitiveness co efficient of the best random
algorithm (Vs. an oblivious adversary).

Meaning of the Right member:
The competitiveness coefficient of the best
deterministic online Alg. Under a “worst-case” input
distribution probability.
37
Theorem: Let R be a randomized
algorithm for paging then C  H
obl
R
k

Since we are looking for a lower bound we can
look at a very simple case

We will look on a world where there are only
K+1 memory items: I = {I1,……….,Ik+1}

Since the cache can consist at most K items at
once only one of the k + 1 items is out at each
step

So, we can say the algorithm decides which of
the items will be out at each step.
38
Choosing a “worst-case” distribution

-
First we should give P – an input probability
distribution :
The sequences to be chosen will be of length N – we assume N>>K.
Choose the first request p1 uniformly from all the
items in I.
2. Choose the i’th request pi uniformly from the set :
I\{pi-1}. i>1
1.
Intuitive explanation:
•
We never request an item that we are sure about
it’s status – this is why this is a worst-case
distribution.
39
Demonstration
I = {I1,I2,I3,I4,I5,I6,I7,I8,I9,I10,I11}
I4
1
2
3
4
5
6
7
8
9
10
current choice set = {I1,I2,I3,I5,I6,I7,I8,I9,I10,I11}
I4
I5
1
2
3
4
5
6
7
8
9
10
40
Now we need to find a lower bound for
A – an optimal det’ online algorithm

We will use again the notion of rounds that
Achiya has mentioned:
• The first round starts with the first request and
ends when, in the first time all the k+1 items in I
has been requested at least once.
• The next round starts with the next request and
ends just before the request to the (k+1)th
distinct item since the start of the current round.
• All next rounds acts the same.
41
How the offline algorithm behaves on
each round?

The offline algorithm will use MIN on each
round – the result will be that the item left
out will be the one requested only at the last
request of the round

This means that the offline algorithm will
miss only once during a round – at the last
request of a round

when it adds the missed item to the cache it
puts out the last requested item of the next
round
42
Demonstration
I = {I1,I2,I3,I4}
Requests=
{I1,I2,I1,I4,I3,I2,I4,I1,I3,I1,I3,I2,I4,I2,I4,I3}
I1 I2 I4
I3 I2 I4
I3 I2 II1
I2 II1 I4
1
1
1
1
2
3
2
3
2
3
2
3
43
How often does the offline algorithm
miss?

We saw that the answer to that question depends the
length of each round ( since it misses once in a
round).

We can ask an equivalent question:
what is the expected number of random choices we
need to do until we choose all items in a group of
order K+1?

Let’s look at it as a random walk on a complete graph
with k+1 vertices – the expected number of steps to
achieve full cover of that graph was approved to be :
K*Hk
44
How a deterministic online algorithm
behaves on each round?

Since the request sequence is random we
can’t expect any regularity in the sequence.

Since the policy of the known deterministic
algorithms is based on some kind of
regularity we can’t expect any of these
policies to be better then the others.

So – the probability for a miss is totally
random.
45
Cont.

the probability that any request falls on the item that
A leaves out at same time-step is exactly 1/k

explanation: remember there is no item that will be
requested twice consecutively so the next request is
for one of the other k items that A might have choose
to leave out.

Given the expected length of a round we conclude
that the expected number of misses by A during a
round will be:
(K*Hk)*(1/k) = Hk
46
Let’s finish the proof

We found that the any of the deterministic online
algorithms misses (expected) Hk times the number
of (expected) misses by an optimal offline
algorithm

This result is under the worst case input distribution
we defined

So we found Hk to be the competitiveness
coefficient of A and by the MinMax principle we
conclude this is also the competitiveness coefficient
of an optimal randomized algorithm for our
problem
47
Overview-part B

Introduction
•
•
•

Proof of a Theorem on CR :
•
•
•
•

Preparations.
Behavior of the offline algorithm.
Behavior of a deterministic algorithm.
Results.
The marker algorithm
•
•

Paging against an oblivious – Definitions.
The players on this section – What is the goal?
Yao’s minimax theorem.
Description.
Proof of it’s competitiveness coefficient.
Summary
48
The Marker Algorithm

After we proved that there are randomized
online algorithms with a competitiveness
coefficient of Hk we will now see an example
for a randomized online algorithm with a
very close achievement.

The algorithm require K bits (marker bits) –
one for each cache location

At The beginning all these bits are set to 0.
49
The Algorithm
1.
Given a memory request:
1.1 if the requested Item is currently in cache –
mark it’s location (set mark bit to 1).
1.2 else
bring the Item into memory and evict an item
by the following policy:
1.2.1 choose uniformly at random an
unmarked memory location.
1.2.2 mark this location, replace the item in it
with the currently requested item.
2. If all memory locations are marked - unmark all
locations when the next request is given.
50
Theorem : The Marker Algorithm is
2*Hk - competitive




We will again use the notion of rounds – about the
same definition as earlier :
A round starts with Pi that is not in the cache and
ends with Pj where :
Pi,Pi+1,……….Pj,Pj+1
where j+1 is the smallest integer s.t the above list
consists k+1 distinct items.
As we will demonstrate each round ends with all the
locations marked.
Each round begins with an item that is not currently
in the cache.
51
Demonstration
I = {I1,I2,I3,I4}
Requests=
{I1,I2,I1,I4,I3,I2,I4,….I1,I3,I1,I4,I2,I4,I2,I3}
I1
1
I1 I2
2
3
1
2
3
I1 I2 I4
I1 I2 I4
1
1
2
3
2
3
I1 I3 I4
I1 I3 I2
I4 II3 I2
I4 II3 I2
1
1
1
1
2
3
2
3
2
3
2
3
I1 I3 I4
I2 I3 I4
I2 II3 I4
I4 II3 I2
1
1
1
1
2
3
2
3
2
3
2
3
52
Some definitions

We will divide all the items requested during a round
to 2 groups:

Stale item is an item that is currently unmarked but
was marked during the last round – meaning: he was
requested during the last round but not during this
round

clean item is an item that is not stale and/or marked
– meaning it wasn’t requested during the last and
current rounds

Let l be the number of clean items requested during
each round.
53
Some more definitions :

S0 is the set of k items currently in the offline
algorithm cache

Sm is the set of k items currently in the Marker
algorithm cache

DI = |S0/SM| at the beginning of a round

DF = |S0/SM| at the end of a round

M0 = the number of misses incurred by the
offline algorithm cache during a round.
54
Observetions:
Note that each of the distinct items is either clean or
stale at the beginning of a round.
 M0≥ l – DI
the number of misses by the offline algorithm is at
least as the number of the items that wasn’t
requested during the last round (l) and are not in it’s
cache at the beginning of the round.


on the other end , all The k items in Sm at the end of
a round are items that were requested during the last
round so we can conclude that the offline algorithm
missed at least DF misses in the last round since
these DF items are not in Sm!
55
How many time the offline algorithm
miss during all rounds?

By the last observation we have:
M 0  max{ l  DI , DF} 
l  DI  DF
2

Note that DF of the k’th round equals DI of the (k+1)’th
round so they are telescoping when summing the above
lower bound on all rounds.

We can use amortization and we charge each round with
l/2 misses for the offline algorithm.

Our mistake range is ±2k – size of the 2 cache memories.
56
How many time the Marker algorithm
miss during all rounds?

First of all by definition , all the clean items
are missed during each round

For the k-l requests to stale items we should
consider the probability that the item
requested is not in the cache and this
depends directly on the number of clean
items requested before him in the round

See the board for calculation of this
probability and the expected number of
misses.
57
So, let’s finish the proof:

We saw the Marker algorithm misses at most twice the
best result possible (achieved by the offline algorithm:
l * Hk
 2 Hk
l /2

Note that these of you interested can find a more
sophisticated algorithm that actually achieves Hk
competitiveness coefficient.
58
Overview-part B

Introduction
•
•
•

Proof of a Theorem on CR :
•
•
•
•

Preparations.
Behavior of the offline algorithm.
Behavior of a deterministic algorithm.
Results.
The marker algorithm
•
•

Paging against an oblivious – Definitions.
The players on this section – What is the goal?
Yao’s minimax theorem.
Description.
Proof of it’s competitiveness coefficient.
Summary
59
Some more interesting insights/summary

One should ask himself if it was a fair game
since we gave the randomized algorithm
some more comfortable conditions
comparing to these of the deterministic
algorithm

Well the all advantage of the randomized
algorithm lies on the fact that it’s actions are
unexpected

This way we prevent the ability to predict
the behavior of the algorithm and affect it.
60
Cont.

This is why it’s intuitive to think on an oblivious
adversary when talking on randomized online
algorithms which hides the status of memory and the
eviction policy at each step

At the same time, it’s intuitive that on application
where it is important to prevent outsiders from
affecting the algorithm we won’t use a deterministic
algorithm which is totally expectable

Application for these kind of algorithms with changing
demands can be found in finance, robot navigation,
short paths in graph, task systems etc.
61
Further results

For these of you interested:
Further results shows much less impressive
achievements of the randomized algorithms
against the smarter adversaries presented by
Achiya.

competitive coefficient against the online
adaptive algorithm : k

competitive coefficient against the offine
adaptive algorithm : k*Hk
62
THE END
63