Randomized Algorithms
CS648
Lecture 16
Randomized Incremental Construction
(Backward analysis)
1
PROBLEM 1
FIND-MIN PROBLEM
2
Find-Min algorithm
A
1
2
Find-Min(A[1..𝒏])
{ 𝒎𝒊𝒏  A[1];
For 𝒊 = 𝟐 to 𝒏 do
{ if (A[𝒊] < 𝒎𝒊𝒏)
𝒎𝒊𝒏  A[𝒊] ;
}
return 𝒎𝒊𝒏;
}
𝒏−𝟏
…
𝑿: no. of times 𝒎𝒊𝒏 is updated.
𝑿𝒊 =
𝟏 if 𝒎𝒊𝒏 is updated in 𝒊th iteration
𝟎 otherwise
𝑿=𝟏+
𝒊>𝟏 𝑿𝒊
𝐄 𝐗 =𝟏+
=𝟏+
Probability that
“A[𝒊] is smaller than {A[𝟏],…, A[𝒊 − 𝟏]}”
𝒊>𝟏 𝐄[𝑿𝒊 ]
𝒊>𝟏 𝐏(𝑿𝒊
= 𝟏)
??
3
Forward analysis for
First 𝒊 − 𝟏 elements
𝐏(𝑿𝒊 = 𝟏)
A
1
2
Find-Min(A[1..𝒏])
{ 𝒎𝒊𝒏  A[1];
For 𝒊 = 𝟐 to 𝒏 do
{ if (A[𝒊] < 𝒎𝒊𝒏)
𝒎𝒊𝒏  A[𝒊] ;
}
return 𝒎𝒊𝒏;
}
…
𝒊
𝒏−𝟏
Notations:
• 𝑺𝒊−𝟏 : set of all subsets of A of size 𝒊 − 𝟏.
• For any 𝒂 ∈ 𝑺𝒊−𝟏 ,
𝓔𝒂 : first 𝒊 − 𝟏 elements of A are some
permutation of 𝒂.
Using Partition Theorem,
𝐏(𝑿𝒊 = 𝟏) = 𝒂∈𝑺𝒊−𝟏 𝐏(𝑿𝒊 = 𝟏 𝓔𝒂
∙ 𝐏(𝓔𝒂 )
4
Forward analysis for
𝐏(𝑿𝒊 = 𝟏)
𝒂 : a subset of 𝒊 − 𝟏 elements.
𝐏(𝑿𝒊 = 𝟏 𝓔𝒂 =
“Given that first 𝒊 − 𝟏 elements of A are some permutation of 𝒂, what is prob.
that 𝑿𝒊 = 𝟏 ?”
For this event to happen, A[𝒊] must be smaller than every element of 𝒂.
 𝐏(𝑿𝒊 = 𝟏 𝓔𝒂 depends upon 𝒂.
For example, if the smallest element of 𝒂 has rank 𝒌 in A, then
𝐏(𝑿𝒊 = 𝟏 𝓔𝒂 = ?? 𝒌 − 𝟏
𝒏−𝒊+𝟏
Dependency on 𝒂 makes it hard to calculate
𝒂∈𝑺𝒊−𝟏
𝐏(𝑿𝒊 = 𝟏 𝓔𝒂
∙ 𝐏(𝓔𝒂 )
5
Backward analysis for
First 𝒊 elements
𝐏(𝑿𝒊 = 𝟏)
A
1
2
Find-Min(A[1..𝒏])
{ 𝒎𝒊𝒏  A[1];
For 𝒊 = 𝟐 to 𝒏 do
{ if (A[𝒊] < 𝒎𝒊𝒏)
𝒎𝒊𝒏  A[𝒊] ;
}
return 𝒎𝒊𝒏;
}
…
𝒊
𝒏−𝟏
Notations:
• 𝑺𝒊 : set of all subsets of A of size 𝒊.
• For any 𝒂 ∈ 𝑺𝒊 ,
𝓔𝒂 : first 𝒊 elements of A are some
permutation of 𝒂.
Using Partition Theorem,
𝐏(𝑿𝒊 = 𝟏) = 𝒂∈𝑺𝒊 𝐏(𝑿𝒊 = 𝟏 𝓔𝒂 ∙ 𝐏(𝓔𝒂 )
6
Backward analysis for
𝐏(𝑿𝒊 = 𝟏)
𝒂 : a subset of 𝒊 − 𝟏 elements.
𝐏(𝑿𝒊 = 𝟏 𝓔𝒂 =
“Given that first 𝒊 elements of A are some permutation of 𝒂, what is prob.
that 𝑿𝒊 = 𝟏 ?”
For this event to happen, the smallest element of 𝒂 must appear at A[𝒊].
 𝐏(𝑿𝒊 = 𝟏 𝓔𝒂 = 𝐏( “the smallest element? ?of 𝒂 appear at A[𝒊]” )
Fact: A is permuted randomly uniformly
 Every element of 𝒂 is equally likely to appear at place A[𝒊].
𝟏
 𝐏(𝑿𝒊 = 𝟏 𝓔𝒂 = ??
𝒊
Same for each 𝒂
7
Backward analysis for
First 𝒊 elements
𝐏(𝑿𝒊 = 𝟏)
A
1
2
Find-Min(A[1..𝒏])
{ 𝒎𝒊𝒏  A[1];
For 𝒊 = 𝟐 to 𝒏 do
{ if (A[𝒊] < 𝒎𝒊𝒏)
𝒎𝒊𝒏  A[𝒊] ;
}
return 𝒎𝒊𝒏;
}
…
𝒊
𝒏−𝟏
Notations:
• 𝑺𝒊 : set of all subsets of A of size 𝒊.
• For any 𝒂 ∈ 𝑺𝒊 ,
𝓔𝒂 : first 𝒊 elements of A are some
permutation of 𝒂.
Using Partition Theorem,
𝐏(𝑿𝒊 = 𝟏) = 𝒂∈𝑺𝒊 𝐏(𝑿𝒊 =𝟏𝟏 𝓔𝒂 ∙ 𝐏(𝓔𝒂 )
=
𝟏
𝒊
𝟏
𝒊
𝒊
𝒂∈𝑺𝒊
𝐏(𝓔𝒂 )
= ⋅𝟏
8
PROBLEM 2
CLOSEST PAIR OF POINTS
9
Closest Pair of Points
Problem Definition:
Given a set 𝑷 of 𝒏 > 𝟏 points in plane, compute the pair of points with
minimum Euclidean distance.
Randomized algorithm:
• O(𝒏) : Randomized Incremental Construction based algorithm
10
𝒊th iteration
Grid structure for first 𝒊 − 𝟏 points
𝜹𝒊−𝟏
𝜹𝒊−𝟏
11
Analysis of 𝒊th iteration
Closest-pair-algorithm(𝑷)
Let <𝒑𝟏 ,𝒑𝟐 ,…,𝒑𝒏 > be a uniformly random permutation of 𝑷;
𝜹𝟐  distance(𝒑𝟏 ,𝒑𝟐 );
𝑮  Build_Grid(𝜹𝟐 , {𝒑𝟏 , 𝒑𝟐 });
For 𝒊 = 𝟑 to 𝒏 do
{
Step 1: locate the cell of the grid 𝑮 containing 𝒑𝒊 ;
Step 2: find the point 𝒑 ∈ {𝒑𝟏 , … , 𝒑𝒊−𝟏 } closest to 𝒑𝒊 ;
let 𝜹 = distance(𝒑, 𝒑𝒊 );
Step 3: If 𝜹 > 𝜹𝒊−𝟏
𝜹𝒊  𝜹𝒊−𝟏 ; Insert(𝒑𝒊 ,𝑮);
Else
𝜹𝒊  𝜹;
𝑮 Build_Grid(𝜹, {𝒑𝟏 , … , 𝒑𝒊 });
}
return 𝜹𝒏 ;
O(1)
O(1)
O(1)
𝒄𝒊 for constant 𝒄
12
running time of 𝒊th iteration
𝑿𝒊 : running time of 𝒊th iteration
O(1) + 𝒄𝒊 ∙ 𝐏(𝜹𝒊 < 𝜹𝒊−𝟏 )
E[𝑿𝒊 ] = ??
Question: What is 𝐏(𝜹𝒊 < 𝜹𝒊−𝟏 ) ?
13
Forward analysis for
𝐏(𝜹𝒊 < 𝜹𝒊−𝟏 )
Closest-pair-algorithm(𝑷)
Let <𝒑𝟏 ,𝒑𝟐 ,…,𝒑𝒏 > be a uniformly random permutation of 𝑷;
𝜹𝟐  distance(𝒑𝟏 ,𝒑𝟐 );
𝑮  Build_Grid(𝜹𝟐 , {𝒑𝟏 , 𝒑𝟐 });
For 𝒊 = 𝟑 to 𝒏 do
{
Step 1: locate the cell of the grid 𝑮 containing 𝒑𝒊 ;
Step 2: find the point 𝒑 ∈ {𝒑𝟏 , … , 𝒑𝒊−𝟏 } closest to 𝒑𝒊 ;
let 𝜹 = distance(𝒑, 𝒑𝒊 );
Step 3: If 𝜹 > 𝜹𝒊−𝟏
𝜹𝒊  𝜹𝒊−𝟏 ; Insert(𝒑𝒊 ,𝑮);
Else
𝜹𝒊  𝜹;
𝑮 Build_Grid(𝜹, {𝒑𝟏 , … , 𝒑𝒊 });
}
return 𝜹𝒏 ;
14
Forward analysis for
𝐏(𝜹𝒊 < 𝜹𝒊−𝟏 )
𝒂 : a subset of 𝒊 − 𝟏 points from 𝑷.
𝓔𝒂 :
𝜹𝒊−𝟏
first 𝒊 − 𝟏 points of 𝑷 are some permutation of 𝒂
𝐏(𝜹𝒊 < 𝜹𝒊−𝟏 𝓔𝒂 = ??Depends upon 𝒂
𝜹𝒊−𝟏
Calculating 𝐏(𝜹𝒊 < 𝜹𝒊−𝟏 ) :
Let 𝑺𝒊−𝟏 be the set of all subsets of 𝑷 of size 𝒊 − 𝟏.
𝐏(𝜹𝒊 < 𝜹𝒊−𝟏 )=
𝒂∈𝑺𝒊 𝐏(𝜹𝒊
< 𝜹𝒊−𝟏 𝓔𝒂 ∙ 𝐏(𝓔𝒂 )
Grid structure for first 𝒊 − 𝟏 points
15
Backward analysis for
𝐏(𝜹𝒊 < 𝜹𝒊−𝟏 )
Closest-pair-algorithm(𝑷)
Let <𝒑𝟏 ,𝒑𝟐 ,…,𝒑𝒏 > be a uniformly random permutation of 𝑷;
𝜹𝟐  distance(𝒑𝟏 ,𝒑𝟐 );
𝑮  Build_Grid(𝜹𝟐 , {𝒑𝟏 , 𝒑𝟐 });
For 𝒊 = 𝟑 to 𝒏 do
{
Step 1: locate the cell of the grid 𝑮 containing 𝒑𝒊 ;
Step 2: find the point 𝒑 ∈ {𝒑𝟏 , … , 𝒑𝒊−𝟏 } closest to 𝒑𝒊 ;
let 𝜹 = distance(𝒑, 𝒑𝒊 );
Step 3: If 𝜹 > 𝜹𝒊−𝟏
𝜹𝒊  𝜹𝒊−𝟏 ; Insert(𝒑𝒊 ,𝑮);
Else
𝜹𝒊  𝜹;
𝑮 Build_Grid(𝜹, {𝒑𝟏 , … , 𝒑𝒊 });
}
return 𝜹𝒏 ;
16
Backward analysis for
𝐏(𝜹𝒊 < 𝜹𝒊−𝟏 )
𝒂 : a subset of 𝒊 points from 𝑷.
𝓔𝒂 : first 𝒊 points of 𝑷 are some permutation of 𝒂
𝜹𝒊
𝟐
𝐏(𝜹𝒊 < 𝜹𝒊−𝟏 𝓔𝒂 = ??
𝒊
Calculating 𝐏(𝜹𝒊 < 𝜹𝒊−𝟏 ) :
Let 𝑺𝒊 be the set of all subsets of 𝑷 of size 𝒊.
𝜹𝒊
𝐏(𝜹𝒊 < 𝜹𝒊−𝟏 )=
𝒂∈𝑺𝒊 𝐏(𝜹𝒊
=
𝒂∈𝑺𝒊
𝟐
Grid structure for first 𝒊 points
=𝒊∙
𝒂∈𝑺𝒊
𝟐
𝒊
< 𝜹𝒊−𝟏 𝓔𝒂 ∙ 𝐏(𝓔𝒂 )
∙ 𝐏(𝓔𝒂 )
𝐏(𝓔𝒂 )
=
𝟐
𝒊
17
running time of 𝒊th iteration
𝑿𝒊 : running time of 𝒊th iteration
E[𝑿𝒊 ] = O(1) + 𝒄𝒊 ∙ 𝐏(𝜹𝒊 < 𝜹𝒊−𝟏 )
= O(1) + 𝒄𝒊 ∙
𝟐
𝒊
= O(1)
Expected running time of the algorithm :
E[𝑿𝒊 ]
𝒊≤𝒏
=
O(1)
= O(𝒏)
𝒊≤𝒏
Theorem:
There exists a linear time Las Vegas algorithm to compute closest pair of
points in plane.
18
RANDOMIZED INCREMENTAL
CONSTRUCTION
19
Randomized Incremental Construction
• Permute the elements of input randomly uniformly.
• Build the structure incrementally.
• Keep some data structure to perform 𝒊th iteration efficiently.
• Use Backward analysis to analyze the expected running time.
20
Randomized Incremental Construction
• Convex Hull of a set of points
• Trapezoidal decomposition of a set of segments.
• Convex polytope of a set of half-planes
• Smallest sphere enclosing a set of points.
• Linear programming in finite dimensions.
21
PROBLEM 3
CONVEX HULL OF POINTS
22
Convex hull of Points
Problem definition:
Given 𝒏 points in a plane, compute a convex polygon of smallest area that
encloses all the points.
23
Convex hull of Points
Deterministic algorithm:
• O(𝒏 𝐥𝐨𝐠 𝒏) time algorithm.
• Many algorithms exist:
Grahams Scan, Jarvis’s march, divide and conquer,…
Randomized algorithm:
• O(𝒏 𝐥𝐨𝐠 𝒏) time algorithm.
• Based on Randomized Incremental Construction.
• Generalizable to higher dimensions.
24
Randomized algorithm for convex hull
Convex-hull-algorithm(𝑷)
{
Let <𝒑𝟏 ,𝒑𝟐 ,…,𝒑𝒏 > be a uniformly random permutation of 𝑷;
𝑯 triangle(𝒑𝟏 ,𝒑𝟐 , 𝒑𝟑 );
For 𝒊 = 𝟒 to 𝒏 do
insert 𝒑𝒊 and update 𝑯
return 𝑯;
}
25
A simple exercise from geometry
Exercise: Given a line L and two points p and q, determine whether the
points lie on the same/different sides of L.
L
p
q
q
𝒚 = 𝒎𝒙 + 𝒄
26
Conflict graph : a powerful data structure
𝒄𝟏
𝒏 − 𝒊 points
𝒄𝟐
cones
𝒄𝟏
𝒄𝟖
𝒄𝟑
𝒄𝟐
𝒄𝟑
𝒄𝟒
𝒄𝟕
𝒄𝟒
𝒄𝟓
𝒄𝟔
𝒄𝟕
𝒄𝟓
𝒄𝟖
𝒄𝟔
27
Before entering the for loop
28
Before entering the for loop
𝒏 − 𝟑 points
cones
𝒄𝟏
𝒄𝟏
𝒄𝟐
𝒄𝟐
𝒄𝟑
𝒄𝟑
29
INSERTING 𝒊TH POINT
30
𝒊th iteration
𝒄𝟏
𝒄𝟖
𝒄𝟐
𝒄𝟑
𝒄𝟒
𝒄𝟕
𝒄𝟓
𝒄𝟔
31
𝒊th iteration
𝒄𝟏
𝒏 − 𝒊 points
𝒄𝟐
cones
𝒄𝟏
𝒄𝟖
𝒄𝟑
𝒄𝟐
𝒄𝟑
𝒄𝟒
𝒄𝟕
𝒄𝟒
𝒄𝟓
𝒄𝟔
𝒄𝟕
𝒄𝟓
𝒄𝟖
𝒄𝟔
32
𝒊th iteration
𝒄𝟏
𝒏 − 𝒊 points
𝒄𝟐
cones
𝒄𝟏
𝒄𝟖
𝒄𝟑
𝒄𝟐
𝒄𝟑
𝒄𝟒
𝒄𝟕
𝒄𝟒
𝒄𝟓
𝒄𝟔
𝒄𝟕
𝒄𝟓
𝒄𝟖
𝒄𝟔
33
𝒊th iteration
𝒄𝟏
𝒏 − 𝒊 points
𝒄𝟐
cones
𝒄𝟏
𝒄𝟖
𝒄𝟑
𝒄𝟐
𝒄𝟑
𝒄𝟒
𝒄𝟕
𝒄𝟒
𝒄𝟓
𝒄𝟔
𝒄𝟕
𝒄𝟓
𝒄𝟖
𝒄𝟔
34
𝒊th iteration
𝒄𝟏
𝒏 − 𝒊 points
𝒄𝟐
cones
𝒄𝟏
𝒄𝟖
𝒄𝟑
𝒄𝟐
𝒄𝟑
𝒄𝟒
𝒄𝟕
𝒄𝟒
𝒄𝟓
𝒄𝟔
𝒄𝟕
𝒄𝟓
𝒄𝟖
𝒄𝟔
35
𝒊th iteration
𝒄𝟏
𝒏 − 𝒊 points
𝒄𝟐
cones
𝒄𝟏
𝒄𝟖
𝒄𝟑
𝒄𝟐
𝒄𝟑
𝒄𝟒
𝒄𝟕
𝒄𝟒
𝒄𝟓
𝒄𝟔
𝒄𝟕
𝒄𝟓
𝒄𝟖
𝒄𝟔
36
𝒊th iteration
𝒄𝟏
𝒏 − 𝒊 points
𝒄𝟐
cones
𝒄𝟏
𝒄𝟖
𝒄𝟑
𝒄𝟐
𝒄𝟑
𝒄𝟒
𝒄𝟕
𝒄𝟒
𝒄𝟓
𝒄𝟔
𝒄𝟕
𝒄𝟓
𝒄𝟖
𝒄𝟔
37
𝒊th iteration
𝒄𝟏
𝒏 − 𝒊 points
𝒄𝟐
cones
𝒄𝟏
𝒄𝟖
𝒄𝟑
𝒄𝟐
𝒄𝟑
𝒄𝟒
𝒄𝟕
𝒄𝟒
𝒄𝟓
𝒄𝟔
𝒄𝟕
𝒄𝟓
𝒄𝟖
𝒄𝟔
38
𝒊th iteration
𝒄𝟏
𝒏 − 𝒊 points
𝒄𝟐
cones
𝒄𝟏
𝒄𝟖
𝒄𝟑
𝒄𝟐
𝒄𝟑
𝒄𝟒
𝒄𝟕
𝒄𝟒
𝒄𝟓
𝒄𝟔
𝒄𝟕
𝒄𝟓
𝒄𝟖
𝒄𝟔
39
𝒊th iteration
𝒏 − 𝒊 points
𝒄𝟏
cones
𝒄𝟏
𝒄𝟖
𝒄′
𝒄′
𝒄"
𝒄𝟔
𝒄𝟕
𝒄"
𝒄𝟕
𝒄𝟖
𝒄𝟔
40
Running time of 𝒊th iteration
Running time of 𝒊th iteration is of the order of
• Number of edges destroyed
Total time for 𝒏 iterations = O(𝒏)
• Number of new edges created
• Number of points in the two adjacent cones that get created
𝑿𝒊
Question: What is the max. number of new edges created in an iteration ?
Answer: 2
 Number of edges created during the algorithm = O(𝒏)
Since every edge destroyed was once created, so
Total number of edges destroyed < Total number of edges decreated
41
Backward analysis of 𝒊th iteration
𝒂 : a subset of 𝒊 points from 𝑷.
𝓔𝒂 : first 𝒊 points of 𝑷 are some
permutation of 𝒂
𝐄[𝑿𝒊 𝓔𝒂 = ??
42
Backward analysis of 𝒊th iteration
𝒄𝟏
𝒄𝟖
𝒂 : a subset of 𝒊 points from 𝑷.
𝓔𝒂 : first 𝒊 points of 𝑷 are some
permutation of 𝒂
𝒄𝟐
𝒑𝟏
𝒑𝟖
𝒄𝟑
𝒑𝟐
𝒑𝟕
𝐄[𝑿𝒊 𝓔𝒂 = ??
𝒑𝟑
𝒄𝟒
𝒄𝟕
𝒑𝟒
𝒑𝟔
𝒑𝟓
𝒄𝟓
𝒄𝟔
43
Backward analysis of 𝒊th iteration
𝒄𝟏
𝒄𝟖
𝒂 : a subset of 𝒊 points from 𝑷.
𝓔𝒂 : first 𝒊 points of 𝑷 are some
permutation of 𝒂
𝒄𝟐
𝒑𝟏
𝒑𝟖
𝒄𝟑
𝒑𝟐
𝒑𝟕
𝐄[𝑿𝒊 𝓔𝒂 = ??
𝒑𝟑
𝒄𝟒
𝒄𝟕
𝒑𝟒
𝒑𝟔
𝒑𝟓
𝒄𝟓
𝒄𝟔
44
Backward analysis of 𝒊th iteration
𝒄𝟏
𝒄𝟖
𝒂 : a subset of 𝒊 points from 𝑷.
𝓔𝒂 : first 𝒊 points of 𝑷 are some
permutation of 𝒂
𝒄𝟐
𝒑𝟏
𝒑𝟖
𝒄𝟑
𝒑𝟐
𝒑𝟕
𝒑𝟑
Calculating 𝐄[𝑿𝒊 ] :
𝒄𝟒
𝒄𝟕
𝒑𝟒
𝒑𝟔
𝒑𝟓
𝒄𝟔
𝐄[𝑿𝒊 𝓔𝒂 = ??𝟐(𝒏 − 𝒊)
𝒊
𝒄𝟓
Let 𝑺𝒊 be the set of all subsets of
𝑷 of size 𝒊.
𝐄[𝑿𝒊 ] =
=
=
𝒂∈𝑺𝒊 𝐄[𝑿𝒊
𝒂∈𝑺𝒊
𝟐(𝒏−𝒊)
𝒊
𝓔𝒂 ∙ 𝐏(𝓔𝒂 )
𝟐(𝒏−𝒊)
∙
𝒊
𝐏(𝓔𝒂 )
45
Running time of the algorithm
Expected running time of ith iteration
= 𝐄[𝑿𝒊 ] + O(1)
= O(
𝒏−𝒊
)
𝒊
Expected running time of the algorithm = O(𝒏 𝐥𝐨𝐠 𝒏 )
Theorem:
There is an O(𝒏 𝐥𝐨𝐠 𝒏 ) time Las Vegas algorithm for computing convex hull.
46
47