CHAPTER 1
Probability
The idea around probability, chance and possibility is quite old and is commonly used in
everyday speech. For example, what is the possibility that it will rain today, or what is the
chance that your rugby team will win the match on Saturday? In everyday life these chances
and possibilities are based on peoples intuition based guesses.
Probability is a concept founded on solid mathematical theory, and can be a powerful aide in
important decision making. In modern society, daily decisions are made in almost every field
where there is some element of risk, such as in those involving large quantities of money, or
those involving human lives. In these situations accurate probability calculations are of
paramount importance (Swanepoel and Allison, 2011).
Probability theory plays an important role in Statistical Inference. In some situations only
sample data is available, in such cases probability theory in used to evaluate the conclusions
made about the population. Probability theory involves experiments in which the outcomes
occur randomly.
1.2
Sample spaces
Experiment: An experiment is an act where the outcome cannot be foreseen (predicted),
meaning that the result are not always the same and the results are not known in advance
before the experiment is conducted.
Sample space: The sample space of an experiment is a collection of all possible outcomes and
is denoted by (
).
Event: An event (denotes by capital letters such as A, B, C, etc.) is a collection of some of the
outcomes.
Example 1
i)
A fair coin is tossed:
ii)
A dice is rolled:
Let A be the event that a number larger than 3 is rolled: A = {4,5,6},
iii)
Two fair coins are tossed:
Let A be the event that at least one head is tossed: A = {HH,HT,TH}
Let B be the event that at least one tail is tossed: B = {TT,HT,TH}
1
.
See Example A and B (Rice, p. 2 - 3).
We make use of Venn-diagrams to illustrate certain concepts of probability theory.
A
Union: The union of two event A and B is the event that either A or B or both will take place,
denoted by
.
Intersection: The intersection of two events A and B is the event that both A and B will take
place, denoted by
.
Complement: The complement of an event A is the event that A will not take place denoted
by Ac or . Therefore all the outcomes in
that are not in A.
Empty set: An empty set occur when the event is impossible, denoted by .
Disjoint: If two events A and B are disjoint (mutually exclusive) it means that if event A
occur event B cannot occur and vice versa, therefore
.
2
Example 1 (iii) continue:
Let C be the event
: C = {HH,HT,TH,TT}
Let D be the event
: D = {HT,TH}
Ac = {TT}, Bc = {HH}
Let E be the event
:E=?
Let F be the event
:F=?
See Example A continuing (Rice, p. 3)
Laws of set theory:
i)
Commutative law:
ii)
Associative law:
iii)
Distributive law:
iv)
and
v)
,
vi)
and
and
and
and
De Morgan’s law:
and
vii)
1.3
Probability measure
We are now interested in calculating the probability that a certain event, for example, event A
will occur, this is denoted by
. The following axioms are always satisfied:
i)
ii)
If
iii)
If
then
and
In general, if
are disjoint, then
.
are mutually disjoint then
3
⋃
∑
Important properties of probability measure:
Property A:
, it follows from axioms (i) and (iii),
Ac
A
Property B:
, it follows from Property A and axiom (i)
Property C:
If
then
B
A
can be expressed as:
therefore
form axiom (iii)
.
Property D:
,
)
A
B
Example 2
Two fair coins are tossed and the outcomes are recorded:
. Let A be
the event of observing two heads:
and B the event of observing one tail:
.
Calculate
.
Example 3
Consider two events, A and B, where:
4
Calculate
.
[
[
]
]
(
[
)
]
See Example A (Rice, p. 6).
1.4
Computing probabilities
Consider the following sample space:
same probability of occurring then,
. If each of the outcomes have the
.
Suppose now that A is the following event:
n
er o ele en s n
n
er o ele en s n
{
}. Then
.
See Example A (Rice, p.6) and Example B (Rice, p. 7).
In the previous examples it is clear that the number of outcomes must be known to calculate
the probability. In many cases the number of outcomes is small and counting them is easy.
Sometimes the number of outcome is large and it is difficult to count them. In such cases we
make use of systematic methods to count the number of outcomes.
Multiplicative principle: If one experiment has m outcomes and another experiment has n
outcomes, then there are a total of mn outcomes for the two experiments.
Example 4:
Suppose you can choose a car in any of five models, and in any of four colours. Then there
are 5x4 = 20 different choices you can make.
See Examples A and B (Rice, p. 8).
Extended Multiplicative principle: If there are p experiments, and the first one has
possible outcomes, the second one has
poss le o co es, …, and he pth one has
possible outcomes, then there is a total of
possible outcomes for the p
experiments.
5
Example 5:
A truck driver can take three different routes in order to get from city A to city B, then one of
four routes from city B to city C, and finally one of three routes from city C to city D. How
many possible routes are there if the driver must go from city A to B to C and then to D?
different routes from A to D.
See Example C (Rice, p. 8).
1.4.2 Permutations and Combinations
Permutations: Ordered set of elements also called orderings.
The total number of orderings of
elements is
(called the permutation of
The following question arises: In how many ways can
with elements? There are two situations:
elements).
elements be chosen from a set C
i)
Sampling with replacement: Each time an element is chosen it is chosen from the
entire set C (an element can be chosen more than once). All the elements have the
same probability of being chosen at any point in time. This is done in
many ways (multiplication principle).
ii)
Sampling without replacement: Each time an element is chosen it is removed from set
C (an element cannot be chosen more than once). Each of the remaining elements has
the same probability of being chosen. This is done in
many ways.
Example 6
Suppose you have three books A, B and C, but you have room form only two on your
bookshelf. In how many ways can you select and arrange the two books?
This is an example of sampling without replacement, therefore the number of ways to arrange
the two books is
[
].
Example 7
“word” s an ordered collec on o le ers. In Engl sh, how any o r le er words are
possible? Supposing each of the 26 letters in the alphabet are chosen (with replacement) to be
used, there will be
many possible four letter words that
could be formed.
See Examples A – F (Rice, p. 10-11).
Combinations: An unordered set of elements. We are no longer interested in ordered samples
just in the samples regardless of the order.
6
How many different samples are possible if objects are taken from set with objects
without replacement and disregarding order? The total number of unordered samples is
( )
n
Cr.
NB: ( )
Example 8
A printed circuit board may be purchased from five suppliers. In how many ways can three
suppliers be chosen from five?
The order does not matter, therefore:
See Examples G – I (Rice, p. 12-13).
Proposition C (Rice, p. 14):
The number of ways in which elements can be grouped in
and ∑
, is
(
classes with
in the ith class,
)
Example 9
In how many different ways can three copies of one novel and one copy each of four other
novels be arranged on a shelf?
(
)
Example 10
The number of permutations of the 11 letters of the word MATHEMATICS (which consists
o 2 M’s, 2 ’s, 2 T’s and 1 o each o he o her le ers) s
(
)
See Examples J – L (Rice, p. 15).
7
1.5
Conditional probability
Sometimes we have some information about a certain event, and we would like to know how
this information will influence the probabilities of other events of interest. This concept is
called “cond onal pro a l y”.
Let A and B be two events with the
. The conditional probability of A given B
(
), the probability of event A given that event B has occurred is:
By rewriting the above formula we can find
(Multiplication law).
Multiplication law: Let A and B be events and assume
A and B will occur is:
. The probability that both
See Rice (p.16) and Examples A and B (Rice, p. 17-18).
Example 11
Two dice are rolled. Let A be the event that the sum of the values facing up of the dice is
equal to seven, and let B be the event the first die has the value six facing up. Suppose that
we rolled the dice one at a time and that the first die came up as a 6 (event B occurred). What
is the probability of event A, given that event B has already taken place?
Law of total probability: Let
be events such that ⋃
for
with
for all . Then for any event A
∑
For example: Let
B1
B2
A
B3
8
and
∑
See Examples C and D (Rice, p. 19).
Example 12
A statistics student is randomly chosen at the university. This student is either a first, second
or third year student with probability 0.5, 0.3 and 0.2 respectively. The probability that the
student will pass statistics in the semester is respectively 0.4, 0.6 and 0.8, for the three years.
Calculate the probability that the student will pass.
Let A be the event that the student pass and B1 the event that the student is a first year, B2 that
the student is a second year, and B3 that the student is a third year. Then
Bayes’r le: Let
with
be events such that ⋃
for all . Then
and
( | )
(disjoint) for
∑
Example 13 (Continuation of Example 12)
Calculate the probability that a randomly chosen Statistics student is a second year student,
given that he has failed the semester.
[
]
NOTE:
See Examples E and F (Rice, p. 20 - 22).
Example Rice, p. 23
Let BC denote the event Breast Cancer and + the event of a positive mammogram.
(
)
( |
9
)
Example 14
Three different machines M1, M2 and M3 were used for producing a large batch of similar
manufactured items. Suppose that 20% of the items were produced by M 1, 30% by M2 and
50% by M3. Suppose further that 1% of the items produced by M1 are defective, that 2% of
the items produced by M2 are defective and that 3% of the items produced by M3 are
defective.
i)
Suppose one item is selected at random from the entire batch and it is found to be
defective. Determine the probability that this item was produced by M2.
Defective item is denoted by event D.
∑
ii)
Suppose one item is selected at random from the entire batch and it is found to be non
defective. Determine the probability that it was produced by M2
∑
Example 15
Suppose that a box contains one fair coin and one with a head on each side. Suppose also that
one coin is selected at random from the box and that when it is tossed, a head is obtained.
Determine the probability that the coin is the fair coin.
Let A denote the event of a fair coin, B the event of the coin with a head on each side and C
the event that a head is obtained.
1.6
Independence
Events A and B are said to be independent if
and
.
or if
Two events A and B are independent if and only if the probability of event B is not
influenced or changed by the occurrence of event A, or vice versa.
10
Example 16
S ppose a researcher no es a person’s gender and whe her or no a person s colo r-blind to
red and green. Does the probability that a person is colour-blind change depending on
whether the person is male or not?
Let A be the event that a person is male and B the event that a person is colour blind.
Since colour-blindness is a male sex-linked characteristic, the probability that a man is
colour-blind will be greater than the probability that a person chosen from the general
population will be colour-blind. The probability of event B depends on whether or not event
A has occurred. A and B are therefore dependent.
Example 17
Consider tossing a single die two times. Let A be the event that a 2 is observed on the first
toss and B is the event that a 2 is observed on the second toss.
The probability of A:
The probability of B:
B g en
occ red
B g en
d d no occ r
Since the probability of B is not changed by the occurrence of event A, we say A and B are
independent.
NOTE:
If A and B are independent events, then A and Bc are also independent.
Three events A, B and C are independent if and only if:
i)
ii)
iii)
iv)
See Examples A – G (Rice, p. 23 - 26).
Example 18
In a survey of 1000 adults respondents were asked about the expense of college education.
The respondents were classified according to whether they currently had a child in college
and whether they thought the loan burden for college students is too high.
11
Too high (A)
Child in college 0.35
(D)
No child in 0.25
college (E)
Right
(B)
0.08
amount Too little (C)
0.20
0.11
0.01
Are events D and A independent?
Therefore A and D are dependent.
OR
Therefore A and D are dependent.
12