Permutations and Combinations
Note that a permutation is an arrangement of objects. For example there are 6 permutations of the set {a, b, c}. They are
{{a, b, c}, {a, c, b}, {b, a, c}, {b, c, a}, {c, b, a}, {c, a, b}}. To count the number of permutations of this set notice that
there are three ways to choose the first element, leaving two ways to choose the second element and then one way to
choose the last element. ie. 6 = 3 × 2 × 1 = 3!. In general there are n! permutations of n distinct objects.
The above argument for permutations of n objects also works if we only want an arrangement of k of them. Suppose we
have four letters to choose from, {a, b, c, d}, and we want to count how many permutations of two letters we can make
from this set. Well here they all are:
{a, b}, {a, c}, {a, d}, {b, c}, {b, d}, {c, d}
{b, a}, {c, a}, {d, a}, {c, b}, {d, b}, {d, c}
4!
There are 12 of them, that is 12 = 4 × 3 = 2!
because there are 4 ways of choosing the first element then 3 ways of
n!
choosing the second. In general there are (n−k)! k-permutations of n distinct objects.
If instead of counting permutations, we counted combinations so that the order of the elements is unimportant the above
arrangement is suggestive and shows us how to calculate the number of combinations of 4 distinct objects taken 2 at a
time. It is the total number of permutations (12) divided by two, since the sets in columns are themselves just
permutations. The general formula for this is the binomial coefficient
n!
n
=
.
k
k!(n − k)!
We read this symbol as “n choose k”. One can think of all the permutations of n objects taken k at a time displayed as
above in a rectangular table. The width of the table would be n choose k and the depth of the table would be k-factorial.
The ‘choice coefficient’ may be generalized to consider say n distinct objects chosen k1 and k2 at a time. This gives the
number of combinations
n!
(n − k1 )!
n!
n
n − k1
×
=
=
k1
k2
k1 !(n − k1 )! k2 !(n − k1 − k2 )!
k1 !k2 !(n − k1 − k2 )!
combinations of n distinct objects chosen in groups of k1 , k2 and n − k1 −k2 . The
above coefficient is called the
n
multinomial coefficient as it is a generalization of the binomial coefficient
.
k
Example: 15 ()
Ten gloves in a closet. How many unique pairs?
Example: 16 ()
100 students in class. A panel to be constructed with 2 Students designated ’cool’, 3 students
designated ’warm’. How many distinct possibilities for the panel?
22
Permutations and Combinations
n
Example: 17 ( Simple
)
k
You have 10 pairs of socks. Each pair is a different colour, but each sock in a pair is the same colour. You wash all of them
together and you put on the first two that you happen to find. What is the probability you will be wearing a matching pair?
Sol:
23
Binomial Choice
Example: 18 ( A Previous problem Generalized)
Consider a collection of 4-bit words where each bit is either 0 or
1. All bits are independent and individual bits are 1 with probability p. The ’check sum’ of a word is the sum of the bits.
1. How many elements does the sample space of 4-bit words contain?
2. Are the probabilities of all 4-bit words the same?
3. Find a formula for P[ check sum =k] for all appropriate values of k.
4. Verify from your formula that P[S] = 1
5. Find the equivalent formula for 10-bit words.
Sol:
24
Sampling with and without Replacement
In random experiments with sequential events, the sampling itself can make a difference to the probabilities of successive
events. Generally, if a sample is replaced after every measurement, each event ’sees’ the same sample space and the
individual outcomes are independent. If samples are not replaced after each measurement, subsequent measurements see
different sample spaces and successive outcomes are no longer independent. Here is an example. An experiment done with
and without replacement.
Example: 19 (Sampling)
Consider an urn containing 5 balls numbered 1 ... 5. Suppose you draw 2 balls at random, replacing the first before
drawing the second, recording the outcome as an ordered pair.
1. What is the sample space?
2. How many distinct outcomes are there?
3. Assuming all distinct outcomes are equally likely, what is the probability that the second ball is a 5 given that the first
is a 1?
4. Are the first and second ball draw independent?
Sol:
The sample space here
(1, 1) (1, 2) (1, 3)
(2, 1) (2, 2) (2, 3)
(3, 1) (3, 2) (3, 3)
(4, 1) (4, 2) (4, 3)
(5, 1) (5, 2) (5, 3)
is the set of ordered pairs S = {(i, j) | i, j = 1...5}. Pictorially:
(1, 4) (1, 5)
(2, 4) (2, 5)
(3, 4) (3, 5)
(4, 4) (4, 5)
(5, 4) (5, 5)
Now let us consider the same problem without replacement.
25
Sampling Without
The sample space here
––− (1, 2) (1, 3)
(2, 1) ––− (2, 3)
(3, 1) (3, 2) ––−
(4, 1) (4, 2) (4, 3)
(5, 1) (5, 2) (5, 3)
is the set of ordered pairs S = {(i, j) | i, j = 1...5, i 6= j}. Pictorially:
(1, 4) (1, 5)
(2, 4) (2, 5)
(3, 4) (3, 5)
––− (4, 5)
(5, 4) ––−
What happens as to the difference between these experiments as the number of balls in the urn get large?
26
Expectation
Whenever the sample space of an experiment is just a set of numbers, the concept of Expectation becomes useful. For
example, suppose a sample space is S = {1, 2, 3, . . . n} so the outcome of any particular experiment is a whole number
from one to n. Call the probability that an outcome is the number k, P[k]. Then the ’Expected value of the outcome’ is
defined as:
n
X
E [outcome] =
k P[k]
k=1
Example: 20 (Expectation)
A coin, when tossed, has a probability p of turning up heads. A Random experiment is
conducted in which the coin is tossed six times and the number of heads counted. Let X be the number of heads counted.
The event A is the event that there are an even number of heads. The event B is the event that the number of heads is
less than three.
1. Describe the sample space SX
2. Find P[A].
3. Find P[B].
4. Find P[A ∩ B]
5. Find P[A | B]
6. What is E [X ].
Sol:
27
Discrete Random Variables
A Random Variable is just a name for an event that is numerical. Such events occur in three ways.
1. If the outcome of a random experiment is a number, the Random
variable, X say, is just that number.
2. If the outcome of a random experiment is not a number, create a
random variable by associating a number with each outcome.
3. Random variables may also be defined as functions of other random
variables. These are often called ‘derived random variables’.
A random variable may be discrete, so that the sample space is a finite or countable list, for example SX = {x1 , x2 , . . .}. A
random variable can also be continuous, so that the sample space is a union of intervals. Mixed type random variables are
also possible. In this section we shall consider only discrete random variables.
Convention Usually random variables are labeled by upper-case letters. So a discrete random variable might be labeled X
and its sample space would be called SX and the possible outcomes would be lower case. So for example:
SX = {x1 , x2 , . . . , xn }
would be a sample space for a discrete random variable X that had n possible outcomes.
28
The PMF
The Probability Mass Function is the probability measure associated with the random variable X . So PX (x) = P[X = x].
For example the PMF for the random variable X from the die experiment is:
(
1/6 x ∈ {1, 2, 3, 4, 5, 6}
PX [x] =
0
otherwise
and is sketched in the fig.(10).
Figure: PMF for the die problem.
Example: 21 (Derived)
For the above probability mass function, find the sample spaces and the probability mass
functions for the following derived random variables:
2
1. Y = X
( .
0 X < 5,
2. Y =
1 otherwise.
Sol:
29
Bernoulli RV
The Bernoulli PMF is the most basic probability mass function. There are
just two elements in the sample space.


x =0
p
PX [x] = 1 − p x = 1


0
otherwise
We have encountered this distribution before. Think of the coin tossing
experiment in which we represent heads by ones and tails by zeroes.
Any discrete random variable may be transformed into the Bernoulli random variable using a convenient set operation · · · ?
Example: 22 (illustrate)
30
Binomial RV
The Binomial distribution is obtained from the Bernoulli distribution by sampling the Bernoulli distribution n times and counting the
number of ones. If K is the number of ones, K is a binomial random
variable and:
!


n

p k (1 − p)n−k k = 0, 1, . . . , n
PK [k] =
k


0
otherwise
For example if we were to test n circuits, where each circuit is
accepted with probability p and rejected with probability 1 − p,
if we let K be the number of accepted circuits, then K would
have the above PMF. Figure (11) shows the PMF for the binomial
distribution with n = 4 and p = 0.5.
Figure: The Binomial PMF.
Many discrete random variable may be transformed into A Binomial random variable by asking something like: "In a
sample of n outcomes, how many have property A".
Example: 23 (illustrate)
31