STATS 412 COURSE NOTES – CHAPTER 2
PROBABILITY
Section 2.1 Basic Ideas
In this section we will establish a foundation of common definitions and ideas so that we have a shared
vocabulary for the course.
Definition: An experiment is a process that results in an outcome that cannot be predicted in advance
with certainty (but it is known that it will be one of several outcomes).
Definition: The sample space, denoted S, of an experiment is the set of all possible outcomes for the
experiment.
Examples: Find the sample space for each of the following experiments.
• Toss a coin, record which side is up: S = {
}
• Toss two coins, record which side is up on each coin: S = {
}
• Toss two coins, record the number of heads: S = {
}
• Examine three fuses in sequence and determine whether each is defective:
Let N = not defective and D = defective. The sample space is then
S={
}
Definition: An event, denoted E, is a subset (collection) of outcomes from the sample space.
Example: Roll a ten-sided die. Consider the following events:
• Roll an even number: E = {
}, N(E)=
• Roll a value that is at most 3: E = {
}, N(E)=
• Roll an even number that is at most 3: E = { }, N(E)=
• Roll a number larger than 10: E = { }
, N(E)=
The event is a simple event if it contains
The event is a compound event if it contains
The event is a null event if it contains
outcome.
outcome.
outcome.
Combining Events
Definition: The complement of an event A, denoted π΄π΄ππ , consists of all outcomes in S that are not in A.
Example: Let S = {0, 1, 2, 3, 4, 5} and A = {0, 2, 4}. What is Ac? Ac = {
}
Definition: The intersection of two events A and B, denoted π΄π΄ ∩ π΅π΅ is the event consisting of all
outcomes in S that are both in A and in B.
Example: Let A = {1, 2, 3} and B = {2, 4, 6}. What is A ∩ B? A ∩ B = {
}
Definition: The union of two events A and B, denoted π΄π΄ ∪ π΅π΅ is the event consisting of all outcomes in S
that are in at least one of the events in A or in B.
Example: Let A = {1, 2, 3} and B = {2, 4, 6}. What is A ∪ B? A ∪ B = {
DeMorgan’s Laws: (π΄π΄ ∩ π΅π΅)ππ = π΄π΄ππ ∪ π΅π΅ ππ , and (π΄π΄ ∪ π΅π΅)ππ = π΄π΄ππ ∩ π΅π΅ ππ .
}
Chapter 2, page 1
Mutually Exclusive Events
Events that can never occur together are considered to be mutually exclusive (or disjoint). For example,
a student at Michigan cannot be classified as both an in-state student and an out-of-state student.
Definition: The events A and B are said mutually exclusive (or disjoint) if they have no outcomes in
common: π¨π¨ ∩ π©π© = ∅.
More generally, events π΄π΄1 , π΄π΄2 , … , π΄π΄ππ are said to be mutually exclusive if no two of them have any
outcomes in common: for any indices ππ, ππ, ππ ≠ ππ ⇒ π΄π΄ππ ∩ π΄π΄ππ = ∅.
Example: Roll a six-sided die. Let the event A be that an even number appears. Let the event B be that
an odd number appears. What are the outcomes in A? In B? What can you say about the events A and
B? A = {
}
B={
}
Because π΄π΄ ∩ π΅π΅ = ∅, the events A and B are
Because π΄π΄ ∪ π΅π΅ = ππ, the events A and B are called
Because π΄π΄ = π΅π΅ ππ and π΅π΅ = π΄π΄ππ , events A and B are called
More generally, if events π΄π΄1 , π΄π΄2 , … , π΄π΄ππ are mutually exclusive and π΄π΄1 ∪ π΄π΄2 ∪ … ∪ π΄π΄ππ = ππ, then the
events π΄π΄1 , π΄π΄2 , … , π΄π΄ππ are called a
of the sample space S.
Example: (Exercise 5, page 60) Four candidates are to be interviewed for a job. Two of them, numbered
1 and 2, are qualified, and the other two, numbered 3 and 4, are not. The candidates are interviewed at
random, and the first qualified candidate interviewed will be hired. The outcomes are the sequences of
candidates that are interviewed. So one outcome is 2, and another is 431.
a. List all the possible outcomes.
The outcomes are the sequences of candidates that end with either #1 or #2. These are {1, 2, 31, 32,
41, 42, 341, 342, 431, 432}.
b. Let A be the event that only one candidate is interviewed. List the outcomes in A.
A = {1, 2}
c. Let B be the event that three candidates are interviewed. List the outcomes in B.
B = {341, 342, 431, 432}
d. Let C be the event that candidate 3 is interviewed. List the outcomes in C.
C = {31, 32, 341, 342, 431, 432}
e. Let D be the event that candidate 2 is not interviewed. List the outcomes in D.
D = {1, 31, 41, 341, 431}
f. Let E be the event that candidate 4 is interviewed. Are A and E mutually exclusive? How about B and
E, C and E, D and E?
E = {41, 42, 341, 342, 431, 432}
A and E are mutually exclusive, because they have no outcomes in common.
B and E are not mutually exclusive, because they both contain the outcomes 341, 342, 431, and 432.
Note that in this situation B ∩ E = B.
C and E are not mutually exclusive, because they both contain the outcomes 341, 342, 431, and 432.
D and E are not mutually exclusive, because they both contain the outcomes 41, 341, and 431.
Chapter 2, page 2
What is Probability?
When we talk about probability, we usually say
“the probability of _______________________________ is ___________________________ .”
For example, “probability of getting a “Heads” in one flip of a fair coin is 1/2.”
What does the 1/2 mean?
Definition: The probability of an event A, written P(A), is the proportion of times that event A would
occur in the long run if the experiment were to be repeated over and over again under the same
experimental conditions:
ππ(π΄π΄)
ππ(π΄π΄) = lim
ππ→∞ ππ
Note: In the real world we estimate probability by assuming only a “sufficiently large” number of
repetitions, and “reasonably similar” experimental conditions.
Axioms of Probability
Axioms serve as the fundamental pieces on which theory is based. They are statements accepted
without proof.
Consider an experiment with outcomes in a sample space S and events A, B, etc.
A probability model P( ) is a function that satisfies the following axioms:
1. ππ(ππ) = 1
2. For any event π΄π΄, ππ(π΄π΄) ≥ 0.
3. If π΄π΄1 , π΄π΄2 , … are mutually exclusive events, then ππ(π΄π΄1 ∪ π΄π΄2 ∪ β― ) = ππ(π΄π΄1 ) + ππ(π΄π΄2 ) + β―
Sample Spaces with Equally Likely Outcomes
If we have a sample space S with N equally likely and distinct outcomes πΈπΈ1 , πΈπΈ2 , … , πΈπΈππ , then
ππ(πΈπΈ1 ) = ππ(πΈπΈ2 ) = β― = ππ(πΈπΈππ ) = ππ
1 = ππ(ππ) = ππ(πΈπΈ1 ∪ πΈπΈ2 ∪ … ∪ πΈπΈππ ) = { ππ ≠ ππ ⇒ πΈπΈππ ∩ πΈπΈππ = ∅} = ππ(πΈπΈ1 ) + ππ(πΈπΈ2 ) + β― + ππ(πΈπΈππ ) = ππππ
ππ(πΈπΈππ ) =
And if A is an event containing k outcomes, then
1
ππ
for every ππ = 1, 2, … , ππ.
ππ(π΄π΄) = πποΏ½πΈπΈππ1 ∪ πΈπΈππ2 ∪ … ∪ πΈπΈππππ οΏ½ = οΏ½ ππ ≠ ππ ⇒ πΈπΈππππ ∩ πΈπΈππππ = ∅οΏ½ = πποΏ½πΈπΈππ1 οΏ½ + πποΏ½πΈπΈππ2 οΏ½ + β― + πποΏ½πΈπΈππππ οΏ½
ππ(π΄π΄) =
ππ
ππ
Example: Roll a fair six-sided die once. Let πΈπΈππ = {roll the number i}, where i = 1, 2, 3, 4, 5, 6.
Let A be the event that we roll a number that is a factor of 8.
Note: A population from which an item is sampled at random can be thought of as a sample space with
equally likely outcomes.
Chapter 2, page 3
Properties of Probability
1. Complement Rule: for any event A, ππ(π΄π΄πΆπΆ ) = 1 − ππ(π΄π΄). Why?
2. Let ∅ denote the null event. Then P(∅) = 0. Why?
3. For any two events A and B, ππ(π΅π΅ ∩ π΄π΄πΆπΆ ) = ππ(π΅π΅) − ππ(π΅π΅ ∩ π΄π΄). Why?
4. General Addition Rule: for two events A and B, ππ(π΄π΄ ∪ π΅π΅) = ππ(π΄π΄) + ππ(π΅π΅) − ππ(π΄π΄ ∩ π΅π΅). Why?
Note: when A and B are mutually exclusive, ππ(π΄π΄ ∪ π΅π΅) = ππ(π΄π΄) + ππ(π΅π΅)
Example: Consider randomly selecting a student in Stats 412. Let D denote the event that the selected
student is majoring in Data Science and LSA denote the event that the student is in the College of
Literature, Science, and the Arts (LSA). Suppose we know that P(D) = 0.16, P(LSA) = 0.22, and the
probability that a randomly selected student is both majoring in Data Science and is in LSA is 0.10.
a) What is the probability that a randomly selected student is majoring in Data Science or is in LSA?
b) What is the probability that a randomly selected student is neither majoring in Data Science nor is in
LSA?
c) What is the probability that a randomly selected student is majoring in Data Science but is not in
LSA?
Chapter 2, page 4
Section 2.3 Conditional Probability and Independence
In many experiments, knowing that one event has occurred can modify our assignment of probability to
other events.
Example: (modified Exercise 15, page 86) A population of 600 semiconductor wafers contains wafers
from three lots. The wafers are categorized by lot and by whether they conform to a thickness
specification. The following table presents the number of wafers in each category. A wafer is chosen at
random from the population.
Lot Conforming Nonconforming Total
A
88
12
100
B
165
35
200
C
260
40
300
Total
513
87
600
Consider the following three probabilities:
a. What is the probability that a wafer is from Lot B and is conforming?
b. What is the probability that a conforming wafer is from Lot B?
c. What is the probability that a wafer from Lot B is conforming?
Definition: The conditional probability of event A given that event B has occurred is
ππ(π΄π΄|π΅π΅) =
ππ(π΄π΄ ∩ π΅π΅)
ππ(π΅π΅)
Multiplication Rule
The formula for conditional probability provides another method for computing ππ(π΄π΄ ∩ π΅π΅):
ππ(π΄π΄ ∩ π΅π΅) = ππ(π΅π΅)ππ(π΄π΄|π΅π΅)
165
200
Note: In the wafer example: ππ(Lot B | conforming) = 513 = 0.3216 ≠ 0.3333 = 600 = ππ(Lot B)
Question: What does it mean when ππ(π΄π΄|π΅π΅) = ππ(π΄π΄)?
Chapter 2, page 5
Definition: Two events A and B are independent if ππ(π΄π΄|π΅π΅) = ππ(π΄π΄), and are dependent otherwise.
The general multiplication rule says ππ(π΄π΄ ∩ π΅π΅) = ππ(π΅π΅)ππ(π΄π΄|π΅π΅).
If A and B are independent, then ππ(π΄π΄ ∩ π΅π΅) = ππ(π΅π΅)ππ(π΄π΄).
To show independence of two events, we need to show any one of the following equalities (they all are
equivalent) holds:
Definition: Events π΄π΄1 , π΄π΄2 , … , π΄π΄ππ are (mutually) independent if the probability of each remains the same
no matter which of the others occur.
That is, events π΄π΄1 , π΄π΄2 , … , π΄π΄ππ are mutually independent if for each π΄π΄ππ and each collection of ππ events
π΄π΄ππ1 , π΄π΄ππ2 , … , π΄π΄ππππ , 1 ≤ ππ ≤ ππ − 1, distinct from π΄π΄ππ , ππ(π΄π΄ππ |π΄π΄ππ1 ∩ π΄π΄ππ2 ∩ β― ∩ π΄π΄ππππ ) = ππ(π΄π΄ππ ).
For mutually independent events it is also true that for any collection of ππ events
π΄π΄ππ1 , π΄π΄ππ2 , … , π΄π΄ππππ , 2 ≤ ππ ≤ ππ, ππ(π΄π΄ππ1 ∩ π΄π΄ππ2 ∩ β― ∩ π΄π΄ππππ ) = ππ(π΄π΄ππ1 )ππ(π΄π΄ππ2 ) β― ππ(π΄π΄ππππ )
For example, three events π΄π΄, π΅π΅, πΆπΆ are mutually independent if
ππ(π΄π΄) = ππ(π΄π΄|π΅π΅) = ππ(π΄π΄|πΆπΆ) = ππ(π΄π΄|π΅π΅ ∩ πΆπΆ)
ππππ
ππ(π΄π΄ ∩ π΅π΅) = ππ(π΄π΄)ππ(π΅π΅)
ππ(π΅π΅) = ππ(π΅π΅|π΄π΄) = ππ(π΅π΅|πΆπΆ) = ππ(π΅π΅|π΄π΄ ∩ πΆπΆ)
ππ(π΄π΄ ∩ πΆπΆ) = ππ(π΄π΄)ππ(πΆπΆ)
ππ(πΆπΆ) = ππ(πΆπΆ|π΄π΄) = ππ(πΆπΆ|π΅π΅) = ππ(πΆπΆ|π΄π΄ ∩ π΅π΅)
ππ(π΅π΅ ∩ πΆπΆ) = ππ(π΅π΅)ππ(πΆπΆ)
ππ(π΄π΄ ∩ π΅π΅ ∩ πΆπΆ) = ππ(π΄π΄)ππ(π΅π΅)ππ(πΆπΆ)
Example: Let π΄π΄ππ be the event that the ππ π‘π‘β circuit relay works, and let ππ(π΄π΄1 ) = ππ(π΄π΄2 ) = ππ(π΄π΄3 ) = ππ.
Assume that the events π΄π΄1 , π΄π΄2 , π΄π΄3 are mutually independent. What is the probability that the entire
circuit works?
Note: For mutual independence, we must check that every subset of events is independent.
If we find one inequality indicating dependence, we can state that the events are not independent.
Example: Consider rolling two fair, six-sided dice. The sample space consists of 36 equally likely
outcomes, each with probability 1/36.
Let A be the event that we roll doubles. A = {(1, 1), (2, 2), (3, 3), (4, 4), (5, 5), (6, 6)}
Let B be the event that the sum of the dice is between 7 and 10, inclusively.
Let C be the event that the dice sum to 2 or 7 or 8.
For reference, P(A) = 1/6, P(B) = 1/2, and P(C) = 1/3.
Are the events A, B, and C mutually independent?
Chapter 2, page 6
Law of Total Probability: If π΄π΄1 , π΄π΄2 , … , π΄π΄ππ are mutually exclusive and exhaustive events, and B is any
event, then
ππ(π΅π΅) = ππ(π΄π΄1 ∩ π΅π΅) + ππ(π΄π΄2 ∩ π΅π΅) + β― + ππ(π΄π΄ππ ∩ π΅π΅).
Equivalently,
ππ
ππ(π΅π΅) = ππ(π΄π΄1 )ππ(π΅π΅|π΄π΄1 ) + ππ(π΄π΄2 )ππ(π΅π΅|π΄π΄2 ) + β― + ππ(π΄π΄ππ )ππ(π΅π΅|π΄π΄ππ ) = οΏ½ ππ(π΄π΄ππ )ππ(π΅π΅|π΄π΄ππ ).
ππ=1
Bayes’ Rule: Let π΄π΄1 , π΄π΄2 , … , π΄π΄ππ be mutually exclusive and exhaustive events with prior probabilities
ππ(π΄π΄ππ ). Then the posterior probability of π΄π΄ππ given that an event π΅π΅ has occurred is
ππ(π΄π΄ππ |π΅π΅) =
ππ(π΄π΄ππ ∩ π΅π΅)
ππ(π΄π΄ππ )ππ(π΅π΅|π΄π΄ππ )
= ππ
,
∑ππ=1 ππ(π΄π΄ππ )ππ(π΅π΅|π΄π΄ππ )
ππ(π΅π΅)
ππ = 1, 2, … , ππ
The “prior probability” ππ(π΄π΄ππ ) is the probability of π΄π΄ππ before knowing that π΅π΅ occurred.
The “posterior probability” ππ(π΄π΄ππ |π΅π΅) is the probability of π΄π΄ππ after knowing that π΅π΅ occurred.
Bayes’ rule “updates” uncertainty about π΄π΄ππ after gaining information that B occurred.
Example: (modified Exercise 32, page 88) A quality-control program at a plastic bottle production line
involves inspecting finished bottles for flaws such as microscopic holes. The proportion of bottles that
actually have such a flaw is only 0.0002. If a bottle has a flaw, the probability is 0.995 that it will fail the
inspection. If a bottle does not have a flaw, the probability is 0.99 that it will pass the inspection.
If a randomly chosen bottle fails inspection, what is the probability that it has a flaw?
Chapter 2, page 7
Section 2.4 Random Variables
Outcomes of some experiments are numbers, outcomes of others are not.
Each outcome can be associated with a number by specifying a role of the association.
Definition: A random variable (RV) assigns a numerical value to each outcome in a sample space.
It is customary to denote random variables with uppercase letters.
Example: Let X be the number of letters in a randomly selected word.
Random variables can be discrete or continuous (or mixed).
Definition: A random variable is discrete if its set of possible values is a finite or countably infinite set of
individual points. This means that if the possible values are arranged in order, there is a gap between
each value and the next one.
Definition: A distribution of a random variable describes the values that the variable can take on and
the probabilities that it takes on those values.
Definition: The probability mass function (pmf) of a discrete random variable ππ specifies the
probability that a random variable ππ takes on value π₯π₯:
ππ(π₯π₯) = ππ(ππ = π₯π₯).
The pmf can be presented in tabular or graphical form,
or as a formula.
π₯π₯
ππ(π₯π₯)
1
0.4
2
0.3
3
0.2
4
0.1
Properties of pmfs:
Example: Let X = number of spam emails you receive during the next hour. Suppose that the pmf for the
random variable X is
π₯π₯
ππ(π₯π₯)
0
0.81
1
0.13
2
0.04
3
0.01
4
0.01
Note that the probability is zero for all other values of X under this model.
a. What is the probability that more than 3 spam messages arrive during the next hour?
b. What is the probability that at least 2 spam messages arrive during the next hour?
c. What is the probability that exactly 3 spam messages arrive during the next hour given that at least 2
spam messages arrive during the next hour?
Chapter 2, page 8
Definition: The cumulative distribution function (cdf) of a random variable ππ specifies the probability
that a random variable ππ takes on a value that is less than or equal to π₯π₯:
πΉπΉ(π₯π₯) = ππ(ππ ≤ π₯π₯).
Example: (Exercise 5, page 112) A survey of cars on a certain stretch of highway during the morning
commute hours showed that 70% had only one occupant, 15% had 2, 10% had 3, 3% had 4, and 2% had
5. Let X represent the number of occupants in a randomly chosen car.
a. Find the probability mass function of ππ.
b. Find the cumulative distribution function of ππ. Draw a graph of the cdf.
c. Use both the pmf and the cdf to find ππ(ππ ≤ 2.5).
Note: For a discrete random variable, the graph of πΉπΉ(π₯π₯) consists of a series of horizontal lines (called
“steps”) with jumps at each of the possible values of ππ. The size of the jump at any point π₯π₯ is equal to the
value of the probability mass function ππ(π₯π₯) at that point π₯π₯.
The Mean for a Discrete Random Variable
The mean of a random variable is the long-term average of its values, computed taking into account that
some values occur more frequently than others:
Example: Select one of the numbers 1 or 2 or 3 and write your choice here _____ .
π₯π₯Μ
=
=
Chapter 2, page 9
Definition: The mean of a discrete random variable ππ is given by
πΈπΈ[ππ] = ππππ = οΏ½ π₯π₯ β ππ(π₯π₯)
ππππππ π₯π₯
The mean of X is sometimes called the expectation or the expected value of X.
Note: The mean of X does not have to be a possible value of X, so, do not round an expected value.
The Variance for a Discrete Random Variable
The variance describes how much the values of a random variable are dispersed/vary around the mean
of the random variable. The variance is a common measure of variability.
Definition: The variance of a discrete random variable ππ is given by
ππππππ(ππ) = ππππ2 = πΈπΈ[(ππ − ππ)2 ] = οΏ½ (π₯π₯ − ππ)2 ππ(π₯π₯)
ππππππ π₯π₯
Note: A shortcut formula for the variance is given by ππππππ(ππ) = πΈπΈ[ππ 2 ] − (πΈπΈ[ππ])2
Fact: The variance of a random variable is always non-negative.
Definition: The standard deviation is the (positive) square root of the variance:
ππππ(ππ) = ππππ = οΏ½Var(ππ) = οΏ½ππππ2
The variance of a random variable is measured in the square of the units.
The standard deviation is measured in the original units of the random variable.
Example: (modified Exercise 2.4.3, page 112) A chemical supply company ships a certain solvent in 10gallon drums. Let X represent the number of drums ordered by a randomly chosen customer. Assume X
has the following probability mass function:
π₯π₯
ππ(π₯π₯)
1
0.10
2
0.10
3
0.60
4
0.15
5
0.05
a. What is the expected value of the number of drums ordered?
b. What is the standard deviation of the number of drums ordered?
c. What is the probability that the number of drums ordered is within two standard deviations of its
mean value?
Chapter 2, page 10
Continuous Random Variables
Definition: A random variable is continuous if its set of possible values is an interval (with finite or infinite
endpoints), for example: (0, 1], (a, b), [0, ∞), (−∞, ∞).
Distribution of a continuous random variable is described by the probability density function (pdf),
denoted by ππ(ππ).
Probabilities for a continuous random variable are the areas under the pdf:
ππ
ππ[ππ < ππ < ππ] = οΏ½ ππ(π₯π₯)ππππ
ππ
Reality: A continuous random variable is used as a model for a quantity that, in principle, takes values in
an interval (subset of the real line), even though every measurement scale is, in truth, discrete. Variables
such as length, income, and temperature are considered continuous random variables.
Example: The pH of a chemical compound is measured (perhaps with error). Possible values of X are in
the interval [0, 14], however the measuring device only reads to a few decimal places. We would still
treat X as a continuous random variable.
Example: (Exercise 2.4.15, page 114) The lifetime (in months) of a transistor in a certain application is a
random variable that can be modeled by the following
function:
−0.1π₯π₯
ππ(π₯π₯) = οΏ½0.1ππ
0
π₯π₯ > 0
π₯π₯ ≤ 0
Calculate the probability that a transistor will last at most 12 months.
Note: For any continuous random variable X and any number c, ππ(ππ = ππ) =
For any continuous RV X, ππ(ππ < ππ < ππ)
ππ(ππ < ππ ≤ ππ)
ππ(ππ ≤ ππ < ππ)
ππ(ππ ≤ ππ ≤ ππ).
Note: We can typically tell from the notation whether we are working with
a probability mass function (pmf) for discrete random variables: ππππ (π₯π₯), ππ(π₯π₯) or
a probability density function (pdf) for continuous random variables: ππππ (π₯π₯), ππ(π₯π₯)
Properties of a pdf:
∞
1. ππ(−∞ < ππ < ∞) = ∫−∞ ππ(π₯π₯)ππππ = 1
2. ππ(π₯π₯) ≥ 0 for all π₯π₯
Chapter 2, page 11
The simplest continuous distribution is the uniform distribution. Many computer simulation studies use
uniform distributions. The uniform distribution has two parameters, a and b, with a < b. The pdf of the
uniform distribution is
Example: (Example 4.63, page 272) When a motorist stops at a red
light at a certain intersection, the waiting time for the light to turn
green, in seconds, is uniformly distributed on the interval (0, 30). Find
the probability that the waiting time is between 12 and 18 seconds.
Interpreting the pdf
Question: In the last example, f(12) = 1/30. What does that mean?
What we just addressed is an important distinction between discrete and continuous RVs:
While probabilities for a discrete random variable are given by the pmf,
probabilities for a continuous random variable are given by areas under the pdf.
Example: Let T represent the time (in minutes) elapsed between when your alarm goes off and when
you actually get out of bed, and say that the distribution of this random variable is given by the
probability density function
1
(80 − 0.01π‘π‘ 3 ) for 0 < π‘π‘ < 20, (0 otherwise)
ππππ (π‘π‘) =
1200
a. What is ππ(ππ ≤ 0)?
b. What is ππ(ππ ≤ 20)?
c. For 0 < π‘π‘ < 20, what is ππ(ππ ≤ π‘π‘)?
Chapter 2, page 12
Cumulative Distribution Functions
Definition: The cumulative distribution function (cdf) of X is the function
π₯π₯
πΉπΉ(π₯π₯) = ππ[ππ ≤ π₯π₯] = οΏ½ ππππ (π‘π‘)πππ‘π‘
−∞
The following is a comparison of the pdf and the cdf for a continuous random variable X:
Example: The following cdf is used to model data in many different fields (industrial engineering,
weather forecasting, insurance, …):
0
π₯π₯ < 0
πΉπΉ(π₯π₯) = οΏ½
−π₯π₯ 2
1 − ππ
π₯π₯ ≥ 0
a. Find ππ(ππ > 1).
b. Find the probability density function of the random variable ππ.
Relationship between pdf and cdf:
If you know the pdf, you integrate it from −∞ to π₯π₯ to find the cdf:
If you know the cdf, you differentiate to find the pdf:
Both the pdf and the cdf carry equivalent information about the random variable X.
Chapter 2, page 13
Mean and Variance for Continuous Random Variables
The mean and variance for continuous random variables are analogous in meaning and calculation to
the mean and variance for discrete random variables.
Definition: Then the mean of a continuous random variable X is given by
∞
πΈπΈ[ππ] = ππππ = οΏ½ π₯π₯ ππ(π₯π₯)ππππ
−∞
As with discrete RVs, the mean of X is sometimes called the expectation, or expected value, of ππ.
Definition: The variance of a continuous random variable X is given by
∞
ππππππ(ππ) = ππππ2 = πΈπΈ[(ππ − ππ)2 ] = οΏ½ (π₯π₯ − ππ)2 ππ(π₯π₯)ππππ
Note: A shortcut formula for the variance is given by
−∞
ππππππ(ππ) = πΈπΈ[ππ 2 ] − (πΈπΈ[ππ])2
The standard deviation is the (positive) square root of the variance:
ππππ(ππ) = ππππ = οΏ½Var(ππ) = οΏ½ππππ2
Example: Say X represents the number of inches of snow on a day that it snows. X is a random variable
with pdf
ππ(π₯π₯) = 9π₯π₯ππ −3π₯π₯ for π₯π₯ > 0, (0 otherwise)
a. What is the expected amount of snow on a day that it snows?
b. What is the standard deviation of the amount of snow on a day that it snows?
Chapter 2, page 14
The Population Median and Percentiles
Definition: Let ππ be a continuous random variable with pdf ππ(π₯π₯) and cdf πΉπΉ(π₯π₯).
• The median of ππ is the point π₯π₯ππ that solves the equation
•
πΉπΉ(π₯π₯ππ ) = ππ(ππ ≤ π₯π₯ππ ) = οΏ½
π₯π₯ππ
−∞
ππ(π₯π₯)πππ₯π₯ = 0.5
More generally, if ππ is a number between 0 and 1, the 100pth percentile is the point π₯π₯100ππ that
solves the equation
πΉπΉοΏ½π₯π₯100ππ οΏ½ = πποΏ½ππ ≤ π₯π₯100ππ οΏ½ = οΏ½
(The median is the 50th percentile: π₯π₯ππ = π₯π₯50 .)
π₯π₯100ππ
−∞
ππ(π₯π₯)ππππ = ππ
Example: (Modified Exercise 2.4.15, page 114) The lifetime in months of a transistor in a certain application is
random with probability density function
−0.1π₯π₯
ππ(π₯π₯) = οΏ½0.1ππ
0
Calculate the median and IQR of this distribution.
π₯π₯ > 0
π₯π₯ ≤ 0
Chapter 2, page 15
Section 2.6 Jointly Distributed Random Variables
Definition: When two or more random variables are associated with each item in a population, the
random variables are said to be jointly distributed. If all the random variables are discrete, they are said
to be jointly discrete. If all the random variables are continuous, they are said to be jointly continuous.
Jointly Discrete Random Variables
Definition: If X and Y are jointly discrete random variables, the joint probability mass function (joint
pmf) is
ππππππ (π₯π₯, π¦π¦) = ππ[ππ = π₯π₯ ∩ ππ = π¦π¦]
Properties of joint pmf: For any jointly discrete random variables X and Y,
1. For all (π₯π₯, π¦π¦), ππππππ (π₯π₯, π¦π¦) ≥ 0
2.
οΏ½ ππππππ (π₯π₯, π¦π¦) = 1
ππππππ (π₯π₯,π¦π¦)
The probability that a randomly selected point (ππ, ππ) will be in some region π΅π΅ is computed as:
ππ[(ππ, ππ) is in region π΅π΅] =
οΏ½
ππππππ (π₯π₯,π¦π¦) ππππ π΅π΅
ππππππ (π₯π₯, π¦π¦)
Example: (Modified Exercise 2.6.12, page 156) Automobile engines and transmissions are produced on
assembly lines, and are inspected for defects after they come off their assembly lines. Those with
defects are repaired. Let ππ represent the number of engines, and ππ the number of transmissions that
require repairs in a one-hour time interval. The joint probability mass function of ππ and ππ is as follows:
π₯π₯ = 0
π₯π₯ = 1
π₯π₯ = 2
π₯π₯ = 3
π¦π¦ = 0
0.13
0.12
0.02
0.01
π¦π¦ = 1
0.10
0.16
0.06
0.02
a. How do we know that this is a valid joint pmf?
π¦π¦ = 2
0.07
0.08
0.08
0.02
π¦π¦ = 3
0.03
0.04
0.04
0.02
b. Find the probability that one engine and two transmissions produced within the next hour will
require repair.
c. Find the probability that one transmission and two engines produced within the next hour will
require repair.
d. Find the probability that exactly two auto parts (engines or transmissions only) produced within
the next hour will require repair.
Chapter 2, page 16
Definition: The marginal probability mass functions of jointly discrete RVs ππ and ππ are:
ππππ (π₯π₯) = ππ[ππ = π₯π₯] = οΏ½ ππππππ (π₯π₯, π¦π¦)
ππππππ π¦π¦
and
ππππ (π¦π¦) = ππ[ππ = π¦π¦] = οΏ½ ππππππ (π₯π₯, π¦π¦)
ππππππ π₯π₯
Example: For the engines and transmissions example, find marginal pmfs of ππ and ππ
π¦π¦ = 0
0.13
0.12
0.02
0.01
π¦π¦ = 1
0.10
0.16
0.06
0.02
π¦π¦ = 2
0.07
0.08
0.08
0.02
π¦π¦ = 3
0.03
0.04
0.04
0.02
π₯π₯
ππππ (π₯π₯)
0
1
2
3
Total
π¦π¦
ππππ (π¦π¦)
0
1
2
3
Total
π₯π₯ = 0
π₯π₯ = 1
π₯π₯ = 2
π₯π₯ = 3
ππππ (0) =
ππππ (1) =
ππππ (2) =
ππππ (3) =
Likewise,
Jointly Continuous Random Variables
Definition: Joint probability density function (joint pdf). of jointly continuous RVs ππ and ππ, denoted
ππππππ (π₯π₯, π¦π¦), is a surface above the π₯π₯π₯π₯-plane, that satisfies the following properties:
1. For any (π₯π₯, π¦π¦), ππππππ (π₯π₯, π¦π¦) ≥ 0
2.
π₯π₯=∞
οΏ½
π¦π¦=∞
οΏ½ ππππππ (π₯π₯, π¦π¦)ππππ ππππ = 1
π₯π₯=−∞ π¦π¦=−∞
The probability that a randomly selected point (ππ, ππ) will be in some region π΅π΅ is measured as the
volume under ππππππ (π₯π₯, π¦π¦) over the region π΅π΅:
ππ[(ππ, ππ) is in region π΅π΅] =
οΏ½ ππππππ (π₯π₯, π¦π¦) ππ(π΄π΄π΄π΄π΄π΄π΄π΄)
ππππππππππππ π΅π΅
In particular, if region B is a rectangle [ππ, ππ] × [ππ, ππ], then
π₯π₯=ππ π¦π¦=ππ
ππ[(ππ ≤ ππ ≤ ππ) ∩ (ππ ≤ ππ ≤ ππ)] = οΏ½
οΏ½ ππππππ (π₯π₯, π¦π¦)ππππ ππππ
π₯π₯=ππ π¦π¦=ππ
Question: Is it reasonable to have ππππππ (π₯π₯, π¦π¦) > 1 at some point (π₯π₯, π¦π¦)?
Chapter 2, page 17
Definition: The marginal probability density functions of ππ and ππ are:
ππππ (π₯π₯) =
ππππ (π¦π¦) =
π¦π¦=∞
οΏ½ ππππππ (π₯π₯, π¦π¦)ππππ
π¦π¦=−∞
π₯π₯=∞
οΏ½ ππππππ (π₯π₯, π¦π¦)ππππ
π₯π₯=−∞
Note: Your marginal distributions should be functions of the RV of interest and not have the other RV in
the equation. If it does, say if your marginal distribution of ππ has π¦π¦ in it, then you know that you have
done something wrong.
Example: Let the random variable ππ denote the time until a computer server connects to your machine
(in microseconds, ππππ), and ππ denote the time until the server authorizes you as a valid user (in
microseconds, ππππ). Each of these random variables measures the wait from a common start time and
ππ < ππ. Assume that the joint probability density function is
ππ(π₯π₯, π¦π¦) = ππππ −π₯π₯−2π¦π¦ for 0 < π₯π₯ < π¦π¦ < ∞,
(0 otherwise)
(From Montgomery and Runger, Applied Statistics and Probability for Engineers, 6e, Modified Example 5-2)
a. Determine the constant ππ such that ππ(π₯π₯, π¦π¦) is a valid joint pdf.
b. Calculate the probability that ππ < 1 and ππ < 2.
c. Find the marginal pdf for ππ.
Chapter 2, page 18
d. Calculate the probability that ππ < 2.
Means of Functions of Random Variables
Definition: If ππ and ππ are jointly distributed random variables, and β(ππ, ππ) is a function of ππ and ππ, then
• If ππ and ππ are jointly discrete with joint pmf ππππππ (π₯π₯, π¦π¦), then
•
πΈπΈ[β(ππ, ππ)] = ππβ(ππ,ππ) = οΏ½ β(π₯π₯, π¦π¦) ππππππ (π₯π₯, π¦π¦)
ππππππ (π₯π₯,π¦π¦)
If ππ and ππ are jointly continuous with joint pdf ππππππ (π₯π₯, π¦π¦), then
πΈπΈ[β(ππ, ππ)] = ππβ(ππ,ππ) =
π₯π₯=∞
οΏ½
π¦π¦=∞
οΏ½ β(π₯π₯, π¦π¦) ππππππ (π₯π₯, π¦π¦)ππππ ππππ
π₯π₯=−∞ π¦π¦=−∞
Example: Suppose that ππ and ππ are discrete random variables with joint pmf
π₯π₯ = 2
π₯π₯ = 6
. Compute πΈπΈοΏ½ππ/√πποΏ½
π¦π¦ = 1
0.36
0.23
π¦π¦ = 4
0.16
0.25
Example: Suppose that ππ and ππ are discrete random variables with joint pdf
Calculate πΈπΈ[ππππ].
ππππππ (π₯π₯, π¦π¦) = 0.75π₯π₯ 2 π¦π¦ for 0 < π₯π₯ < 2, 0 < π¦π¦ < 1; (0 otherwise)
Covariance and correlation coefficient
Definition: If ππ and ππ are jointly distributed random variables, then covariance of ππ and ππ is:
πΆπΆπΆπΆπΆπΆ(ππ, ππ) = πΈπΈ[(ππ − πΈπΈ[ππ])(ππ − πΈπΈ[ππ])] = πΈπΈ[ππππ] − πΈπΈ[ππ]πΈπΈ[ππ]
Properties of covariance:
1. πΆπΆπΆπΆπΆπΆ(ππ, ππ) =
2. πΆπΆπΆπΆπΆπΆ(ππ, ππ) =
Chapter 2, page 19
Definition: The correlation coefficient of ππ and ππ is:
ππππππ =
Properties of the correlation coefficient:
1. For any RVs ππ and ππ, −1 ≤ ππππππ ≤ 1
πΆπΆπΆπΆπΆπΆ(ππ, ππ)
οΏ½ππππππ[ππ]ππππππ[ππ]
2. ππππππ is unitless
3. ππππππ is defined only for quantitative variables
Example: Suppose that ππ and ππ are discrete random variables with joint pmf
π₯π₯ = 2
π₯π₯ = 6
π¦π¦ = 1
0.36
0.23
π¦π¦ = 4
0.16
0.25
Compute covariance and correlation coefficient of ππ and ππ
Conditional Distributions
If ππ and ππ are jointly distributed random variables, then knowing the value of ππ may change
probabilities regarding the random variable ππ.
Definition: Let ππ and ππ be jointly discrete random variables. The conditional probability mass function
of Y given X = x is
ππ[ππ = π¦π¦ ∩ ππ = π₯π₯] ππππππ (π₯π₯, π¦π¦)
ππππ|ππ=π₯π₯ (π¦π¦) = ππ[ππ = π¦π¦|ππ = π₯π₯] =
=
ππ[ππ = π₯π₯]
ππππ (π₯π₯)
Conditional Expectation
Definition: A conditional expected value of ππ given πΏπΏ = ππ is calculated using conditional pmf:
πΈπΈ[ππ|ππ = π₯π₯] = οΏ½ π¦π¦ ππππ|ππ=π₯π₯ (π¦π¦)
ππππππ π¦π¦
Chapter 2, page 20
Example: For the engines and transmissions example,
π¦π¦ = 0
0.13
0.12
0.02
0.01
π₯π₯ = 0
π₯π₯ = 1
π₯π₯ = 2
π₯π₯ = 3
0.28
ππππ (π¦π¦)
π¦π¦ = 1
0.10
0.16
0.06
0.02
0.34
π¦π¦ = 2
0.07
0.08
0.08
0.02
0.25
π¦π¦ = 3
0.03
0.04
0.04
0.02
0.13
ππππ (π₯π₯)
0.33
0.40
0.20
0.07
1
a. Given that one engine produced within the next hour will require repair, find the conditional pmf of
the number of transmissions produced within the next hour will require repair.
ππππ|ππ=1 (0) =
ππππ|ππ=1 (1) =
ππππ|ππ=1 (2) =
ππππ|ππ=1 (3) =
π¦π¦
ππππ|ππ=1 (π¦π¦)
0
1
2
3
Total
b. Calculate the conditional probability ππ(ππ > 0|ππ = 1).
c. How many transmissions produced within the next hour do we expect to require repair, if one engine
produced within the next hour will require repair?
Definition: Let ππ and ππ be jointly continuous random variables. The conditional probability density
function of Y given X = x is
ππππππ (π₯π₯, π¦π¦)
ππππ|ππ=π₯π₯ (π¦π¦) =
ππππ (π₯π₯)
To compute conditional probability that ππ will take value within an interval (ππ, ππ), given that ππ = π₯π₯, we
integrate the conditional density function ππππ|ππ=π₯π₯ (π¦π¦) over the interval (ππ, ππ):
π¦π¦=ππ
ππ[ππ < ππ < ππ|ππ = π₯π₯] = οΏ½ ππππ|ππ=π₯π₯ (π¦π¦) ππππ
π¦π¦=ππ
Chapter 2, page 21
Example: In the connection vs authentication example, the joint probability density function is
ππ(π₯π₯, π¦π¦) = 6ππ −π₯π₯−2π¦π¦ for 0 < π₯π₯ < π¦π¦ < ∞,
(0 otherwise)
Previously we found the marginal density function of ππ. An analogous calculation will give the marginal
density function of ππ:
ππππ (π₯π₯) = 3ππ −3π₯π₯ for π₯π₯ > 0 (0 otherwise)
a. Find the conditional probability density function for ππ given that ππ = π₯π₯.
b. Determine the probability that ππ exceeds 2, given that ππ = 1.5.
c. What do we expect authentication to be when connection is 1.5?
Independent Random Variables
The notion of independence for random variables is similar to the notion of independence for events.
Two random variables are independent if knowledge regarding one of them does not affect the
probabilities of the other.
Definition: Two random variables X and Y are independent, provided that
•
•
If X and Y are jointly discrete, then for all (π₯π₯, π¦π¦), ππππππ (π₯π₯, π¦π¦) = ππππ (π₯π₯)ππππ (π¦π¦)
If X and Y are jointly continuous, then for all (π₯π₯, π¦π¦), ππππππ (π₯π₯, π¦π¦) = ππππ (π₯π₯)ππππ (π¦π¦)
Facts: If ππ and ππ are independent random variables, then
•
•
If ππ and ππ are jointly discrete, then for all (π₯π₯, π¦π¦), ππππ|ππ=π¦π¦ (π₯π₯) = ππππ (π₯π₯)
If X and Y are jointly continuous, then for all (π₯π₯, π¦π¦), ππππ|ππ=π¦π¦ (π₯π₯) = ππππ (π₯π₯)
Chapter 2, page 22
Example: Revisiting the engines and transmissions example,
π₯π₯ = 0
π₯π₯ = 1
π₯π₯ = 2
π₯π₯ = 3
ππππ (π¦π¦)
Are ππ and ππ independent?
π¦π¦ = 0
0.13
0.12
0.02
0.01
0.28
π¦π¦ = 1
0.10
0.16
0.06
0.02
0.34
π¦π¦ = 2
0.07
0.08
0.08
0.02
0.25
π¦π¦ = 3
0.03
0.04
0.04
0.02
0.13
ππππ (π₯π₯)
0.33
0.40
0.20
0.07
1
Example: Revisiting the connection and authentication example,
ππ(π₯π₯, π¦π¦) = 6ππ −π₯π₯−2π¦π¦ for 0 < π₯π₯ < π¦π¦ < ∞,
Are ππ and ππ independent?
(0 otherwise)
Example: Let ππ(π₯π₯, π¦π¦) = ππ −π₯π₯−π¦π¦ , π₯π₯ ≥ 0, π¦π¦ ≥ 0 (0 otherwise) be the joint pdf for ππ and ππ.
Marginal pdfs for ππ and ππ were computed to be
ππππ (π₯π₯) = ππ −π₯π₯ for π₯π₯ ≥ 0 (0 otherwise) and ππππ (π₯π₯π₯π₯) = ππ −π¦π¦ for π¦π¦ ≥ 0 (0 otherwise), respectively.
Are ππ and ππ independent?
Covariance and Independence
If random variables ππ and ππ are independent, then
πΈπΈ[ππππ] =
Fact: Independent random variables are always uncorrelated, however uncorrelated random variables
may or may not be independent.
Chapter 2, page 23
Section 2.5 Linear Functions and Linear Combinations of Random Variables
Often, we perform arithmetic operations on random variables to come up with new random variables.
Whether we add or subtract a constant to a random variable, multiply or divide a random variable by a
non-zero constant, or add two or more random variables together, we can work with the new random
variable in an analogous fashion to working with the original random variables.
Linear Functions of a Random Variable
Definition: A linear function of a random variable ππ is an expression of the form
where ππ and ππ are constants, and ππ ≠ 0.
ππππ + ππ
Facts: Let X be a random variable and let a and b be constants, a ≠ 0. Then
πΈπΈ[ππππ + ππ] = ππππππ+ππ = ππππ[ππ] + ππ = ππππππ + ππ
2
ππππππ(ππππ + ππ) = ππππππ+ππ
= ππ2 ππππππ(ππ) = ππ2 ππππ2
ππππ(ππππ + ππ) = ππππππ+ππ = |ππ|ππππ(ππ) = |ππ|ππππ
In the facts above, the constant a “rescales” the random variable X and the constant b “shifts” the
random variable X.
Example: Temperatures in Ann Arbor average around 50 °F with a standard deviation of about 3 °F.
Because most of the world measures temperatures in Celsius (exceptions: Bahamas, Belize, Cayman
Islands, Palau, and the United States), it would be nice to convert the mean and standard deviation to
degrees Celsius. Calculate the mean and standard deviation for temperatures in Ann Arbor in Celsius.
Linear Combinations of Random Variables
Definition: A linear combination of random variables ππ1 , ππ2 , … , ππππ is an expression of the form
ππ1 ππ1 + ππ2 ππ2 + β― + ππππ ππππ
where ππ1 , ππ2 , … , ππππ are nonzero constants,
The mean of their linear combination is:
πΈπΈ[ππ1 ππ1 + ππ2 ππ2 + β― + ππππ ππππ ] =
Chapter 2, page 24
It is unimportant for computation of the mean of a linear combination of random variables if the
variables are independent or not. Variances, however, are calculated differently for linear combinations
of dependent vs. independent random variables (we will address dependent RVs in Section 2.6).
In-general, ππππππ[ππππ + ππππ] =
However if ππ and ππ are independent, then πΆπΆπΆπΆπΆπΆ(ππ, ππ) =
If ππ1 , ππ2 , … , ππππ are independent random variables, then the variance of their linear combination is:
ππππππ(ππ1 ππ1 + ππ2 ππ2 + β― + ππππ ππππ ) =
Example: For the difference of independent random variables X and Y:
πΈπΈ[ππ − ππ] =
ππππππ(ππ − ππ) =
Independence and Simple Random Samples
Recall from Chapter 1 that a simple random sample (SRS) of size n is a sample chosen by a method in
which each collection of n population items is equally likely to make up the sample.
When an SRS of size ππ is drawn from a population, each item in the sample can be thought of as a
random variable ππππ , with ππ1 , ππ2 , … , ππππ being independent and identically distributed (iid).
The items in an SRS may be treated as independent, except when the sample is a large proportion (more
than 5%) of a finite population.
Sample Total and Sample Mean
Soon, when we will be estimating parameters, we will be working with sample totals and sample means.
Definition: Suppose ππ1 , ππ2 , … , ππππ is an SRS from a population with mean ππππ and variance ππππ2 .
The sample total is defined as:
ππ = ππ1 + ππ2 + β― + ππππ
So, the sample total is the linear combination ππ1 ππ1 + ππ2 ππ2 + β― + ππππ ππππ in which ππ1 = ππ2 = β― = ππππ = 1.
Applying formulas for the mean and variance of a linear combination of RVs, we obtain:
πΈπΈ[ππ] = ππ ππ = πΈπΈ[ππ1 + ππ2 + β― + ππππ ] =
ππππππ(ππ) = ππππ2 = ππππππ[ππ1 + ππ2 + β― + ππππ ] =
ππππ(ππ) = ππππ =
Definition: Suppose ππ1 , ππ2 , … , ππππ is an SRS from a population with mean ππππ and variance ππππ2 .
The sample mean is defined as:
1
1
πποΏ½ = (ππ1 + ππ2 + β― + ππππ ) = ππ
ππ
ππ
So, the sample mean is the linear function ππππ + ππ, where ππ = 1/ππ and ππ = 0
Chapter 2, page 25
Applying formulas for the mean, variance and standard deviation of a linear function, we obtain:
1
πΈπΈ[πποΏ½] = πππποΏ½ = πΈπΈ οΏ½ πποΏ½ =
ππ
1
ππππππ(πποΏ½) = ππππ2οΏ½ = ππππππ οΏ½ πποΏ½ =
ππ
οΏ½
ππππ(ππ) = πππποΏ½ =
Example: The number of hours a call center employee works per day has an average of 10 hours and a
standard deviation of 0.25 hours. Assume that the number of hours an employee works on one day is
independent of the number of hours the employee works on any other day during the week. Suppose
that one employee works 4 days during the week.
a. Find the mean and standard deviation of the total number of hours the employee works during one
week.
b. Find the mean and standard deviation of the average number of hours the employee works per day
within a week.
The employee is payed $20 per hour.
c. Find the mean and the standard deviation of the total amount the employee is payed during a week.
d. Find the mean and the standard deviation of the average amount the employee is payed per day
within a week.
Note: The random variables ππ1 + ππ2 + ππ3 + ππ4 and 4ππ have the same mean, but different standard
deviations.
Chapter 2, page 26
0
You can add this document to your study collection(s)
Sign in Available only to authorized usersYou can add this document to your saved list
Sign in Available only to authorized users(For complaints, use another form )