Applied Statistics I
Liang Zhang
Department of Mathematics, University of Utah
June 16, 2008
Liang Zhang (UofU)
Applied Statistics I
June 16, 2008
1 / 37
Conditional Probability
Liang Zhang (UofU)
Applied Statistics I
June 16, 2008
2 / 37
Conditional Probability
Liang Zhang (UofU)
Applied Statistics I
June 16, 2008
2 / 37
Conditional Probability
The Law of Total Probability
Let A1 , A2 , . . . , Ak be mutually exclusive and exhaustive events. Then for
any other event B,
P(B) = P(B | A1 ) · P(A1 ) + P(B | A2 ) · P(A2 ) + · · · + P(B | Ak ) · P(Ak )
=
k
X
P(B | Ai ) · P(Ai )
i=1
where exhaustive means A1 ∪ A2 ∪ · · · Ak = S.
Liang Zhang (UofU)
Applied Statistics I
June 16, 2008
2 / 37
Conditional Probability
The Law of Total Probability
Let A1 , A2 , . . . , Ak be mutually exclusive and exhaustive events. Then for
any other event B,
P(B) = P(B | A1 ) · P(A1 ) + P(B | A2 ) · P(A2 ) + · · · + P(B | Ak ) · P(Ak )
=
k
X
P(B | Ai ) · P(Ai )
i=1
where exhaustive means A1 ∪ A2 ∪ · · · Ak = S.
Liang Zhang (UofU)
Applied Statistics I
June 16, 2008
2 / 37
Conditional Probability
Liang Zhang (UofU)
Applied Statistics I
June 16, 2008
3 / 37
Conditional Probability
Bayes’ Theorem
Let A1 , A2 , . . . , Ak be a collection of k mutually exclusive and exhaustive
events with prior probabilities P(Ai )(i = 1, 2, . . . , k). Then for any other
event B with P(B) > 0, the posterior probability of Aj given that B has
occurred is
P(Aj | B) =
Liang Zhang (UofU)
P(B | Aj ) · P(Aj )
P(Aj ∩ B)
= Pk
P(B)
i=1 P(B | Ai ) · P(Ai )
Applied Statistics I
j = 1, 2, . . . k
June 16, 2008
3 / 37
Conditional Probability
Liang Zhang (UofU)
Applied Statistics I
June 16, 2008
4 / 37
Conditional Probability
Application of Bayes’ Theorem
Example 2.30 Incidence of a rare disease
Only 1 in 1000 adults is afflicted with a rare disease for which a diagnostic
test has been developed. The test is such that when an individual actually
has the disease, a positive result will occur 99% of the time, whereas an
individual without the disease will show a positive test result only 2% of
the time. If a randomly selected individual is tested and the result is
positive, what is the probability that the individual has the disease?
Liang Zhang (UofU)
Applied Statistics I
June 16, 2008
4 / 37
Conditional Probability
Liang Zhang (UofU)
Applied Statistics I
June 16, 2008
5 / 37
Conditional Probability
Example 2.29 A chain of video stores sells three different brands of DVD
players. Of its DVD player sales, 50% are brand 1, 30% are brand 2, and
20% are brand 3. Each manufacturer offers a 1-year warranty on parts and
labor. It is known that 25% of brand 1’s DVD players require warranty on
parts and labor, whereas the corresponding percentages for brands 2 and 3
are 20% and 10%, respectively.
1. What is the probability that a randomly selected purchaser has bought
a brand 1 DVD player that will need repair while under warranty?
Liang Zhang (UofU)
Applied Statistics I
June 16, 2008
5 / 37
Conditional Probability
Example 2.29 A chain of video stores sells three different brands of DVD
players. Of its DVD player sales, 50% are brand 1, 30% are brand 2, and
20% are brand 3. Each manufacturer offers a 1-year warranty on parts and
labor. It is known that 25% of brand 1’s DVD players require warranty on
parts and labor, whereas the corresponding percentages for brands 2 and 3
are 20% and 10%, respectively.
1. What is the probability that a randomly selected purchaser has bought
a brand 1 DVD player that will need repair while under warranty?
2. What is the probability that a randomly selected purchaser has a DVD
player that will need repair while under warranty?
Liang Zhang (UofU)
Applied Statistics I
June 16, 2008
5 / 37
Conditional Probability
Example 2.29 A chain of video stores sells three different brands of DVD
players. Of its DVD player sales, 50% are brand 1, 30% are brand 2, and
20% are brand 3. Each manufacturer offers a 1-year warranty on parts and
labor. It is known that 25% of brand 1’s DVD players require warranty on
parts and labor, whereas the corresponding percentages for brands 2 and 3
are 20% and 10%, respectively.
1. What is the probability that a randomly selected purchaser has bought
a brand 1 DVD player that will need repair while under warranty?
2. What is the probability that a randomly selected purchaser has a DVD
player that will need repair while under warranty?
3. If a customer returns to the store with a DVD player that needs
warranty work, what is the probability that it is a brand 1 DVD player? A
brand 2 DVD player? A brand 3 DVD player?
Liang Zhang (UofU)
Applied Statistics I
June 16, 2008
5 / 37
Independence
Liang Zhang (UofU)
Applied Statistics I
June 16, 2008
6 / 37
Independence
Example: A fair die is tossed and we want to guess the outcome. The
outcomes will be 1, 2, 3, 4, 5, 6 with equal probability 16 each. If we are
interested in getting the following results: A = {1, 3, 5}, B = {1, 2, 3} and
C = {3, 4, 5, 6}, then we can calculate the probability for each event:
P(A) = P(B) =
Liang Zhang (UofU)
3
1
4
2
= , and P(C ) = = .
6
2
6
3
Applied Statistics I
June 16, 2008
6 / 37
Independence
Example: A fair die is tossed and we want to guess the outcome. The
outcomes will be 1, 2, 3, 4, 5, 6 with equal probability 16 each. If we are
interested in getting the following results: A = {1, 3, 5}, B = {1, 2, 3} and
C = {3, 4, 5, 6}, then we can calculate the probability for each event:
P(A) = P(B) =
3
1
4
2
= , and P(C ) = = .
6
2
6
3
If someone tell you that after one toss, event C happened, i.e. the
outcome is one of {3, 4, 5, 6}, then what is the probability for event A to
happen and what for B?
P(A | C ) =
P(A ∩ C )
=
P(C )
Liang Zhang (UofU)
1
3
2
3
1
P(B ∩ C )
= ; P(B | C ) =
=
2
P(C )
Applied Statistics I
1
6
2
3
1
= .
4
June 16, 2008
6 / 37
Independence
Example: A fair die is tossed and we want to guess the outcome. The
outcomes will be 1, 2, 3, 4, 5, 6 with equal probability 16 each. If we are
interested in getting the following results: A = {1, 3, 5}, B = {1, 2, 3} and
C = {3, 4, 5, 6}, then we can calculate the probability for each event:
P(A) = P(B) =
3
1
4
2
= , and P(C ) = = .
6
2
6
3
If someone tell you that after one toss, event C happened, i.e. the
outcome is one of {3, 4, 5, 6}, then what is the probability for event A to
happen and what for B?
P(A | C ) =
P(A ∩ C )
=
P(C )
1
3
2
3
1
P(B ∩ C )
= ; P(B | C ) =
=
2
P(C )
1
6
2
3
1
= .
4
P(A | C ) = P(A) while P(B | C ) 6= P(B)
Liang Zhang (UofU)
Applied Statistics I
June 16, 2008
6 / 37
Independence
Liang Zhang (UofU)
Applied Statistics I
June 16, 2008
7 / 37
Independence
Definition
Two events A and B are independent if P(A | B) = P(A), and are
dependent otherwise.
Liang Zhang (UofU)
Applied Statistics I
June 16, 2008
7 / 37
Independence
Definition
Two events A and B are independent if P(A | B) = P(A), and are
dependent otherwise.
Remark:
Liang Zhang (UofU)
Applied Statistics I
June 16, 2008
7 / 37
Independence
Definition
Two events A and B are independent if P(A | B) = P(A), and are
dependent otherwise.
Remark:
1. P(A | B) = P(A) ⇒ P(B | A) = P(B). This is natural since the
definition for independent should be symmetric.
Liang Zhang (UofU)
Applied Statistics I
June 16, 2008
7 / 37
Independence
Definition
Two events A and B are independent if P(A | B) = P(A), and are
dependent otherwise.
Remark:
1. P(A | B) = P(A) ⇒ P(B | A) = P(B). This is natural since the
definition for independent should be symmetric.
P(B | A) =
Liang Zhang (UofU)
P(A ∩ B)
P(A | B) · P(B)
=
P(A)
P(A)
Applied Statistics I
June 16, 2008
7 / 37
Independence
Liang Zhang (UofU)
Applied Statistics I
June 16, 2008
8 / 37
Independence
Remark:
Liang Zhang (UofU)
Applied Statistics I
June 16, 2008
8 / 37
Independence
Remark:
2. If events A and B are mutually disjoint, then they can not be
independent. Intuitively, if we know event A happens, we then know that
B does not happen, since A ∩ B = ∅. Mathmatically,
P(A | B) =
P(A ∩ B)
P(∅)
=
= 0 6= P(A),
P(B)
P(B)
unless P(A) = 0 which is trivial.
Liang Zhang (UofU)
Applied Statistics I
June 16, 2008
8 / 37
Independence
Remark:
2. If events A and B are mutually disjoint, then they can not be
independent. Intuitively, if we know event A happens, we then know that
B does not happen, since A ∩ B = ∅. Mathmatically,
P(A | B) =
P(A ∩ B)
P(∅)
=
= 0 6= P(A),
P(B)
P(B)
unless P(A) = 0 which is trivial.
e.g. for the die tossing example, if A = {1, 3, 5} and B = {2, 4, 6}, then
P(A ∩ B) = P(∅) = 0, therefore P(A | B) = 0. However, P(A) = 0.5.
Liang Zhang (UofU)
Applied Statistics I
June 16, 2008
8 / 37
Independence
Liang Zhang (UofU)
Applied Statistics I
June 16, 2008
9 / 37
Independence
The Multiplication Rule for Independent Events
Liang Zhang (UofU)
Applied Statistics I
June 16, 2008
9 / 37
Independence
The Multiplication Rule for Independent Events
The general multiplication rule tells us P(A ∩ B) = P(A | B) · P(B).
Liang Zhang (UofU)
Applied Statistics I
June 16, 2008
9 / 37
Independence
The Multiplication Rule for Independent Events
The general multiplication rule tells us P(A ∩ B) = P(A | B) · P(B).
However, if A and B are independent, then the above equation would be
P(A ∩ B) = P(A) · P(B) since P(A | B) = P(A).
Liang Zhang (UofU)
Applied Statistics I
June 16, 2008
9 / 37
Independence
The Multiplication Rule for Independent Events
The general multiplication rule tells us P(A ∩ B) = P(A | B) · P(B).
However, if A and B are independent, then the above equation would be
P(A ∩ B) = P(A) · P(B) since P(A | B) = P(A).
Furthermore, we have the following
Proposition
Events A and B are independent if and only if
P(A ∩ B) = P(A) · P(B)
Liang Zhang (UofU)
Applied Statistics I
June 16, 2008
9 / 37
Independence
The Multiplication Rule for Independent Events
The general multiplication rule tells us P(A ∩ B) = P(A | B) · P(B).
However, if A and B are independent, then the above equation would be
P(A ∩ B) = P(A) · P(B) since P(A | B) = P(A).
Furthermore, we have the following
Proposition
Events A and B are independent if and only if
P(A ∩ B) = P(A) · P(B)
In words, events A and B are independent iff (if and only if) the
probability that the both occur (A ∩ B) is the product of the two
individual probabilities.
Liang Zhang (UofU)
Applied Statistics I
June 16, 2008
9 / 37
Independence
Liang Zhang (UofU)
Applied Statistics I
June 16, 2008
10 / 37
Independence
In real life, we often use this multiplication rule without noticing it.
Liang Zhang (UofU)
Applied Statistics I
June 16, 2008
10 / 37
Independence
In real life, we often use this multiplication rule without noticing it.
The probability for getting {HH} when you toss a fair coin twice is 41 ,
which is obtained by 12 · 12 ;
Liang Zhang (UofU)
Applied Statistics I
June 16, 2008
10 / 37
Independence
In real life, we often use this multiplication rule without noticing it.
The probability for getting {HH} when you toss a fair coin twice is 41 ,
which is obtained by 12 · 12 ;
The probability for getting {6,5,4,3,2,1} when you toss a fair die six times
is ( 16 )6 , which is simply obtained by 16 · 61 · 16 · 16 · 61 · 16 ;
Liang Zhang (UofU)
Applied Statistics I
June 16, 2008
10 / 37
Independence
In real life, we often use this multiplication rule without noticing it.
The probability for getting {HH} when you toss a fair coin twice is 41 ,
which is obtained by 12 · 12 ;
The probability for getting {6,5,4,3,2,1} when you toss a fair die six times
is ( 16 )6 , which is simply obtained by 16 · 61 · 16 · 16 · 61 · 16 ;
The probability for getting {♠♠♠} when you draw three cards from a
1
deck of well-shuffled cards with replacement is 64
, which is simply
1 1 1
obtained by 4 · 4 · 4 .
Liang Zhang (UofU)
Applied Statistics I
June 16, 2008
10 / 37
Independence
In real life, we often use this multiplication rule without noticing it.
The probability for getting {HH} when you toss a fair coin twice is 41 ,
which is obtained by 12 · 12 ;
The probability for getting {6,5,4,3,2,1} when you toss a fair die six times
is ( 16 )6 , which is simply obtained by 16 · 61 · 16 · 16 · 61 · 16 ;
The probability for getting {♠♠♠} when you draw three cards from a
1
deck of well-shuffled cards with replacement is 64
, which is simply
1 1 1
obtained by 4 · 4 · 4 .
However, if you draw the cards without replacement, then the
multiplication rule for independent events fails since the event {the first
card is ♠} is no longer independent of the event {the second card is ♠}.
In fact,
P({the second card is ♠ | the first card is ♠}) =
Liang Zhang (UofU)
Applied Statistics I
12
.
51
June 16, 2008
10 / 37
Independence
Liang Zhang (UofU)
Applied Statistics I
June 16, 2008
11 / 37
Independence
Example: Exercise 89
Suppose identical tags are placed on both the left ear and the right ear of
a fox. The fox is then let loose for a period of time. Consider the two
events C1 ={left ear tag is lost} and C2 = {right ear tag is lost}. Let
π = P(C1 ) = P(C2 ), and assume C1 and C2 are independent events.
Derive an expression (involving π) for the probability that exactly one tag
is lost given that at most one is lost.
Liang Zhang (UofU)
Applied Statistics I
June 16, 2008
11 / 37
Independence
Example: Exercise 89
Suppose identical tags are placed on both the left ear and the right ear of
a fox. The fox is then let loose for a period of time. Consider the two
events C1 ={left ear tag is lost} and C2 = {right ear tag is lost}. Let
π = P(C1 ) = P(C2 ), and assume C1 and C2 are independent events.
Derive an expression (involving π) for the probability that exactly one tag
is lost given that at most one is lost.
Liang Zhang (UofU)
Applied Statistics I
June 16, 2008
11 / 37
Independence
Liang Zhang (UofU)
Applied Statistics I
June 16, 2008
12 / 37
Independence
Remark:
Liang Zhang (UofU)
Applied Statistics I
June 16, 2008
12 / 37
Independence
Remark:
1. If events A and B are independent, then so are events A0 and B, events
A and B 0 as well as events A0 and B 0 .
P(B) − P(A ∩ B)
P(A ∩ B)
P(A0 ∩ B)
=
=1−
P(B)
P(B)
P(B)
0
= 1 − P(A | B) = 1 − P(A) = P(A )
P(A0 | B) =
Liang Zhang (UofU)
Applied Statistics I
June 16, 2008
12 / 37
Independence
Remark:
1. If events A and B are independent, then so are events A0 and B, events
A and B 0 as well as events A0 and B 0 .
P(B) − P(A ∩ B)
P(A ∩ B)
P(A0 ∩ B)
=
=1−
P(B)
P(B)
P(B)
0
= 1 − P(A | B) = 1 − P(A) = P(A )
P(A0 | B) =
2. We can use the condition P(A ∩ B) = P(A) · P(B) to define the
independence of the two events A and B.
Liang Zhang (UofU)
Applied Statistics I
June 16, 2008
12 / 37
Independence
Liang Zhang (UofU)
Applied Statistics I
June 16, 2008
13 / 37
Independence
Independence of More Than Two Events
Definition
Events A1 , A2 , . . . , An are mutually independent if for every k
(k = 2, 3, . . . , n) and every subset of indices i1 , i2 , . . . , ik ,
P(Ai1 ∩ Ai2 ∩ · · · ∩ Aik ) = P(Aii ) · P(Ai2 ) · ··· · P(Aik ).
Liang Zhang (UofU)
Applied Statistics I
June 16, 2008
13 / 37
Independence
Independence of More Than Two Events
Definition
Events A1 , A2 , . . . , An are mutually independent if for every k
(k = 2, 3, . . . , n) and every subset of indices i1 , i2 , . . . , ik ,
P(Ai1 ∩ Ai2 ∩ · · · ∩ Aik ) = P(Aii ) · P(Ai2 ) · ··· · P(Aik ).
In words, n events are mutually independent if the probability of the
intersection of any subset of the n events is equal to the product of the
individual probabilities.
Liang Zhang (UofU)
Applied Statistics I
June 16, 2008
13 / 37
Independence
Liang Zhang (UofU)
Applied Statistics I
June 16, 2008
14 / 37
Independence
An very interesting example: Exercise 113
A box contains the following four slips of paper, each having exactly the
same dimensions: (1) win prize 1; (2) win prize 2; (3) win prize 3; and (4)
win prize 1, 2 and 3. One slip will be randomly selected. Let A1 = {win
prize 1}, A2 = {win prize 2}, and A3 = {win prize 3}. Are these three
events mutually independent?
Liang Zhang (UofU)
Applied Statistics I
June 16, 2008
14 / 37
Independence
Liang Zhang (UofU)
Applied Statistics I
June 16, 2008
15 / 37
Independence
Example:
Consider a system of seven identical components connected as following.
For the system to work properly, the current must be able to go through
the system from the left end to the right end. If components work
independently of one another and P(component works)=0.9, then what is
the probability for the system to work?
Liang Zhang (UofU)
Applied Statistics I
June 16, 2008
15 / 37
Independence
Example:
Consider a system of seven identical components connected as following.
For the system to work properly, the current must be able to go through
the system from the left end to the right end. If components work
independently of one another and P(component works)=0.9, then what is
the probability for the system to work?
Let A = {the system works} and Ai = {component i works}. Then
A = (A1 ∪ A2 ) ∩ ((A3 ∩ A4 ) ∪ (A5 ∩ A6 )) ∩ A7 .
Liang Zhang (UofU)
Applied Statistics I
June 16, 2008
15 / 37
Random Variables
Liang Zhang (UofU)
Applied Statistics I
June 16, 2008
16 / 37
Random Variables
Definition
For a given sample sample space S of some experiment, a random
variable (rv) is any rule that associates a number with each outcome in
S. In mathematical language, a random variable is a function whose
domain is the sample space and whose range is the set of real numbers.
Liang Zhang (UofU)
Applied Statistics I
June 16, 2008
16 / 37
Random Variables
Definition
For a given sample sample space S of some experiment, a random
variable (rv) is any rule that associates a number with each outcome in
S. In mathematical language, a random variable is a function whose
domain is the sample space and whose range is the set of real numbers.
We use uppercase letters, such as X and Y to, denote random variables
and use lowercase letters, such as x and y , to denote some particular value
of the corresponding random variable. For example, X (s) = x means that
value x is associated with the oucome s by the rv X .
Liang Zhang (UofU)
Applied Statistics I
June 16, 2008
16 / 37
Random Variables
Definition
For a given sample sample space S of some experiment, a random
variable (rv) is any rule that associates a number with each outcome in
S. In mathematical language, a random variable is a function whose
domain is the sample space and whose range is the set of real numbers.
We use uppercase letters, such as X and Y to, denote random variables
and use lowercase letters, such as x and y , to denote some particular value
of the corresponding random variable. For example, X (s) = x means that
value x is associated with the oucome s by the rv X .
Liang Zhang (UofU)
Applied Statistics I
June 16, 2008
16 / 37
Random Variables
Liang Zhang (UofU)
Applied Statistics I
June 16, 2008
17 / 37
Random Variables
Examples:
Liang Zhang (UofU)
Applied Statistics I
June 16, 2008
17 / 37
Random Variables
Examples:
1. Assume we toss a coin. Then S = {H, T}. We can define a rv X by
X (H) = 1 and X (T) = 0
Liang Zhang (UofU)
Applied Statistics I
June 16, 2008
17 / 37
Random Variables
Examples:
1. Assume we toss a coin. Then S = {H, T}. We can define a rv X by
X (H) = 1 and X (T) = 0
2. A techincian is going to check the quality of 10 prodcuts. For each
product the outcome is either successful (S) or defective (D). Then we can
define a rv Y by
(
1, successful
Y =
0, defective
Liang Zhang (UofU)
Applied Statistics I
June 16, 2008
17 / 37
Random Variables
Examples:
1. Assume we toss a coin. Then S = {H, T}. We can define a rv X by
X (H) = 1 and X (T) = 0
2. A techincian is going to check the quality of 10 prodcuts. For each
product the outcome is either successful (S) or defective (D). Then we can
define a rv Y by
(
1, successful
Y =
0, defective
Definition
Any random variable whose only possible values are 0 abd 1 is called a
Bernoulli random variable.
Liang Zhang (UofU)
Applied Statistics I
June 16, 2008
17 / 37
Random Variables
Liang Zhang (UofU)
Applied Statistics I
June 16, 2008
18 / 37
Random Variables
More examples:
3. (Example 3.3) We are investigating two gas stations. Each has six gas
pumps. Consider the experiment in which the number of pumps in use at
a particular time of day is determined for each of the stations.
Define rv’s X , Y and U by
X = the total number of pumps in use at the two stations
Y = the difference between the number of pumps in use at station 1
and the number in use at station 2
U = the maximum of the numbers of pumps in use at the two stations
Liang Zhang (UofU)
Applied Statistics I
June 16, 2008
18 / 37
Random Variables
More examples:
3. (Example 3.3) We are investigating two gas stations. Each has six gas
pumps. Consider the experiment in which the number of pumps in use at
a particular time of day is determined for each of the stations.
Define rv’s X , Y and U by
X = the total number of pumps in use at the two stations
Y = the difference between the number of pumps in use at station 1
and the number in use at station 2
U = the maximum of the numbers of pumps in use at the two stations
If this experiment is performed and s = (3, 4) results, then
X ((3, 4)) = 3 + 4 = 7, so we say that the observed value of X was x = 7.
Similarly, the observed value of Y would be y = 3 − 4 = −1, and the
observed value of U would be u = max(3, 4) = 4.
Liang Zhang (UofU)
Applied Statistics I
June 16, 2008
18 / 37
Random Variables
Liang Zhang (UofU)
Applied Statistics I
June 16, 2008
19 / 37
Random Variables
More examples:
4. Assume we toss a coin until we get a Head. Then the sample space
would be S = {H, TH, TTH, TTTH, . . . } If we define a rv X by X
X = the number we totally tossed
Then X ({H}) = 1, X ({TH}) = 2, X ({TTH}) = 3, . . . , and so on.
Liang Zhang (UofU)
Applied Statistics I
June 16, 2008
19 / 37
Random Variables
More examples:
4. Assume we toss a coin until we get a Head. Then the sample space
would be S = {H, TH, TTH, TTTH, . . . } If we define a rv X by X
X = the number we totally tossed
Then X ({H}) = 1, X ({TH}) = 2, X ({TTH}) = 3, . . . , and so on.
In this case, the random variable X can be any positive integer, which in
all is infinite.
Liang Zhang (UofU)
Applied Statistics I
June 16, 2008
19 / 37
Random Variables
More examples:
4. Assume we toss a coin until we get a Head. Then the sample space
would be S = {H, TH, TTH, TTTH, . . . } If we define a rv X by X
X = the number we totally tossed
Then X ({H}) = 1, X ({TH}) = 2, X ({TTH}) = 3, . . . , and so on.
In this case, the random variable X can be any positive integer, which in
all is infinite.
5. Assume we are going to measure the length of 100 desks. Define the rv
Y by
Y = the length of a particular desk
Y can also assume infinitly possible values.
Liang Zhang (UofU)
Applied Statistics I
June 16, 2008
19 / 37