Probability Theory
and Statistics
Kasper K. Berthelsen, Dept. For Mathematical Sciences
kkb@math.aau.dk
Literature:
Walpole, Myers, Myers & Ye:
Probability and Statistics for Engineers and Scientists,
Prentice Hall, 8th ed.
Slides and lecture overview:
http://people.math.aau.dk/~kkb/Undervisning/ET610/
Lecture format:
2x45 min lecturing followed by exercises in group rooms
1
Lecture1
STATISTICS
What is it good for?
90
80
70
60
50
A
40
B
C
30
20
Quality control:
10
0
1
2
3
4
Forecasting:
• Expectations for the
future?
• How will the stock
markets behave??
2
Analysis of sales:
• How much do we sell,
and when?
• Should we change or
sales strategy?
• What is my rate of
defective products?
• How can I best manage
my production?
• What is the best way to
sample?
Lecture1
Probability theory
Sample space and events
Consider an experiment
Sample space S:
S
VENN DIAGRAM
Event A:
Example:
S={1,2,…,6} rolling a dice
S={plat,krone} flipping a coin
S
Example:
A
A={1,6} when rolling a dice
Complementary
event A´:
3
S
A
A´
Example:
A´={2,3,4,5} rolling a dice
lecture 1
Probability theory
Events
Example:
Rolling a dice
S={1,2,3,4,5,6}
A={2,4,6}
B={1,2,3}
S
A
5
4 6 2
1 3
Disjoint events: CD = Ø
C={1,3,5} and D={2,4,6} are disjoint
4
Intersection:
AB={2}
B
Union:
AB={1,2,3,4,6}
S
C
D
lecture 1
Probability theory
Counting sample points
Ways of placing your bets: Guess the results of 13 matches
Possible outcomes:
1X2
Home win
3 possibilities
Draw
3 possibilities
Away win
•
•
•
Answer: 3·3·3· ··· ·3 = 3
13
The multiplication rule
3 possibilities
5
lecture 1
Probability theory
Counting sample points
Ordering n different objects
Number of permutations ???
There are
• n ways of selecting the first object
• n -1 ways of selecting second object
•
•
•
• 1 way of selecting the last object
”n factorial”
n · (n -1) · ··· · 1 = n ! ways
The multiplication rule
3!= 6
6
lecture 1
Probability theory
Counting sample points
Multiplication rule:
If k independent operations can be performed in
n1, n2, … , nk ways, respectively, then the k operations can be
performed in
n1 · n2 · ··· · nk ways
T
Tree diagram:
T
Flipping a coin three times
H
T
(Head/Tail)
T
H
H
23 = 8 possible outcomes
T
H
T
H
7
H
T
H
lecture 1
Probability theory
Counting sample points
Number of possible ways of selecting r objects from a
set of n destinct elements:
Ordered
Unordered
8
Without
replacement
With
replacement
n!
n Pr 
(n  r )!
n
r
n
n!

 
 r  r !(n  r )!
-
lecture 1
Probability theory
Counting sample points
Example:
Ann, Barry, Chris, and Dan should from a committee
consisting of two persons, i.e. unordered without replacement.
Number of possible combinations:
 4
4!
6
 
 2  2!2!
Writing it out : AB AC AD BC BD CD
9
lecture 1
Probability theory
Counting sample points
Example:
Select 2 out of 4 different balls ordered and without
replacement
Number of possible combinations:
Notice: Order matters!
10
4!
 12
4 P2 
(4  2)!
lecture 1
Probability theory
Probability
Let A be an event, then we denote
P(A) the probability for A
It always hold that 0 < P(A) < 1
S
A
P(Ø) = 0
Consider an experiment which has N equally
likely outcomes, and let exactly n of these
events correspond to the event A. Then
n
# successful outcomes
P( A) 
=
# possible outcomes
N
11
P(S) = 1
Example:
Rolling a dice
P(even number)

3 1

6 2
lecture 1
Probability theory
Probability
Example: Quality control
A batch of 20 units contains 8 defective units.
Select 6 units (unordered and without replacement).
Event A: no defective units in our random sample.
 20 
Number of possible samples: N    (# possible)
6
 12 
Number of samples without defective units: n   
6
 12 
(# successful)
 
6  12!6!14!
77

P(A) 


 0.024
6!6!20!
3230
 20 
lecture 1


12
6


Probability theory
Probability
Example: continued
Event B: exactly 2 defective units in our sample
Number of samples with exactly 2 defective units:
 12   8 
n    
 4   2
 12   8 
  
4   2
12!8!6!14!

P(B) 

 0.3576
4!8!2!6!20!
 20 


6


13
(# successful)
lecture 1
Probability theory
Rules for probabilities
A
B
Intersection:
AB
Union:
AB
P(A B) = P(A) + P(B) - P(A  B)
P(B) = P(B  A) + P(B  A´ )
If A and B are disjoint:
In particular:
14
P(A  B) = P(A) + P(B)
P( A ) + P(A´ ) = 1
lecture 1
Probability theory
Conditional probability
Conditional probability for A given B:
P(A  B)
P(A|B) 
P(B)
Bayes’ Rule:
where P ( B ) > 0
A
B
P(AB) = P(A|B)P(B) = P(B|A)P(A)
Rewriting Bayes’ rule:
P(B|A)P(A)
P(B|A)P(A)
P(A|B) 

P(B)
P(B|A)P(A)+P(B|A´)P(A´)
15
lecture 1
Probability theory
Conditional probability
Example page 59:
The distribution of
employed/unemployed
amongst men and women in
a small town.
Employed
Unemployed
Total
Man
460
40
500
Woman
140
260
400
Total
600
300
900
P(man & employd ) 460 / 900 460 23
P(man | employed ) 



 76.7%
P(employd )
600 / 900 600 30
P(man |unemployed ) 
16
P(man & unemployed ) 40 / 900
40
2


  13.3%
P(unemployed )
300 / 900 300 15
lecture 1
Probability theory
Bayes’ rule
Example: Lung disease & Smoking
According to ”The American Lung Association” 7% of the population
suffers from a lung disease, and 90% of these are smokers. Amongst
people without any lung disease 25.3% are smokers.
Events:
A: person has lung disease
B: person is a smoker
Probabilities:
P(A)
= 0.07
P(B|A) = 0.90
P(B|A´ ) = 0.253
What is the probability that at smoker suffers from a lung disease?
P( A | B) 
17
P( B | A) P( A)
0.9  0.07

 0.211
P( B | A) P( A)  P( B | A´)P( A´) 0.9  0.07  0.253  0.93
lecture 1
Probability theory
Bayes’ rule – extended version
A1 , … , Ak are a
partitioning of S
A1
Law of total probability:
k
P( B)   P( B | Ai ) P( Ai )
A4
A3
B
A5
S
A6
A2
i 1
Bayes’ formel udvidet:
P( B | Ar ) P( Ar )
P( Ar | B)  k
 P( B | Ai ) P( Ai )
18
i 1
lecture 1
Probability theory
Independence
Definition:
Two events A and B are said to be independent if and only if
P(B|A) = P(B)
or
P(A|B) = P(A)
Alternative Definition:
Two events A and B are said to be independent if and only if
P(A∩B) = P(A)P(B)
Notice: Disjoint event (mutually exclusive event) are
dependent!
19
lecture 1
Probability theory
Conditional probability
Example:
Employed
Unemployed
Total
Man
460
40
500
Woman
140
260
400
Total
600
300
900
460 / 900
P(man |employed ) 
 76.7%
600 / 900
P(man )  500 / 900  55.6%
Conclusion: the two events “man” and “employed” are
dependent.
20
lecture 1
Probability theory
Rules for conditional probabilities
Probability of events A and B happening simultaneously
P( A  B)  P( A | B) P( B)
Probability of events A, B and C happening simultaneously
P( A  B  C )  P( A | B  C ) P( B | C ) P(C )
Proof:
P( A  B  C )  P( A | B  C ) P( B  C )  P( A | B  C ) P( B | C ) P(C )
General rule:
P( A1  A2    Ak )  P( A1 | A2    Ak ) 
P( A2 | A3    Ak ) 
21
 P( Ak 1 | Ak )  P( Ak )
lecture 1