ENGG 2040C: Probability Models and Applications
Spring 2014
The probabilistic method
Andrej Bogdanov
Sometimes probability helps
us solve problems that do not
involve randomness at all.
Friends and “non-friends”
Among every six people, there must be three
that are all friends or are all “non-friends”.
Proof
:I must have at least 3 friends,
or at least 3 non-friends
If I have 3 friends:
Either they are all non-friends with one another,
Or a pair of them are friends, so the three of us
are all friends.
Ramsey’s theorem
Among every 4n people, there must be n that
are all friends or all “non-friends”.
4n is a very large number!
E.g. when n = 40, 4n = 109 951 162 777
Do we really need that many people for that?
Erdös’s theorem
There can be 2n/2 - 1 people among which no n
of them are all friends or all non-friends.
(for n ≥ 3)
Proof: Let N = 2n/2 - 1
Probability model: Every pair is friends with
probability ½, independently of the others.
Let X be the number of groups of n people that are
all friends or all non-friends.
Erdös’s theorem
X = ∑ groups g of n people Xg,
where Xg is an indicator r.v. for the event
“People in g are all friends or all non-friends.”
E[Xg] = P(Xg = 1) = 2∙2-C(n, 2)
E[X] = ∑ groups g E[Xg] = C(N, n) ∙ 2 ∙ 2-C(n, 2)
≤ Nn ∙ 2 ∙ 2-C(n, 2)
= 2n log N + 1 – n(n – 1)/2
< 1.
Erdös’s theorem
Because E[X] < 1, it must be that P(X = 0) > 0.
So the event X = 0 can happen.
But X = 0 means there is no group of n people in
which all are friends or all are non-friends.
We used probability not to model any reality,
but as a tool to solve a mathematical problem
about friendships.
Quicksort
1. Choose a pivot stick p.
2. Go over all other sticks from left to right:
Move all sticks shorter than p to the left of p
and all sticks longer than p to the right of p.
3. Recursively sort sticks to the left of p.
4. Recursively sort sticks to the right of p.
Quicksort
How to choose the pivot?
Quicksort
Every pivot works, but some take us there faster
than others
We will measure the number N of pairwise
comparisons done by the algorithm
N depends on the input and on the choice of pivot
We want to choose the pivot so N is small regardless
of the input
Quicksort
Example:
What if we always chose the leftmost pivot?
requires all possible C(n, 2)
comparisons for n sticks
For any fixed pivot-choosing strategy there is an
input that entails all possible C(n, 2) comparisons
The randomized quicksort algorithm
1. Choose a pivot stick p uniformly at random.
2. Go over all other sticks from left to right:
Move all sticks shorter than p to the left of p
and all sticks longer than p to the right of p.
3. Recursively sort sticks to the left of p.
Analysis of randomized quicksort
The number N of pairwise comparisons is now a
random variable that depends on the pivot choices
N = N12 + N13 + ... + N(n-1)n
where Nij is an indicator r.v. for the event “the ith and
jth shortest sticks were compared at some point”
Nij = 0 happens if sticks i and j were split before
either was chosen as pivot
Analysis of randomized quicksort
j–i–1
P(Nij = 0) =
j–i+1
P(Nij = 1) =
2
j–i+1
i
E[N] = E[N12] + E[N13] + ... + E[N(n-1)n]
= (n – 1) × 2/2 + (n – 2) × 2/3 + ... + 1 × 2/n
≤ n × (2/2 + 2/3 + ... + 2/n)
≤ 2n ln n.
j
What is next?
If you like probability you will find lots of it...
Computer networks [CSCI 4430, CSCI 5470]
Algorithm design [CSCI 3160, CSCI 5450]
Machine learning [CSCI 3320]
Reliable communication [IERG 3010, IERG 5154]
Secure communication and computation [CSCI 5440]
Data privacy [CSCI 5520]
The nature of efficient computation [CSCI 5170]