Probability Lecture Notes: Introduction
1
What is Probability?
Probability is a numerical measure of uncertainty. It tells us how likely an event is to
occur.
0 ≤ P (A) ≤ 1
A probability close to 1 means the event is very likely. A probability close to 0 means
the event is unlikely.
Practical Example
If the weather forecast says that there is a 70% chance of rain, then the probability of
rain is
P (Rain) = 0.70.
2
Experiment
An experiment is a process that produces an outcome whose result cannot be predicted
with certainty before it occurs.
Examples
• Tossing a coin
• Rolling a die
• Selecting a student randomly
• Observing whether a customer repays a loan
In a bank, observing whether a customer repays a loan or defaults is a random experiment.
A company checks whether a product passes or fails a quality test. Identify the
experiment and possible outcomes.
1
Basic Probability
A First Course in Probability
Solution
The experiment is checking the quality status of the product.
The possible outcomes are
S = {Pass, Fail}.
3
Sample Space
The sample space is the set of all possible outcomes of a random experiment. It is usually
denoted by S.
Examples
For tossing a coin once:
S = {H, T }
For rolling a die once:
S = {1, 2, 3, 4, 5, 6}
A coin is tossed two times. Write the sample space.
The possible outcomes are
S = {HH, HT, T H, T T }.
Thus, the sample space contains 4 outcomes.
4
Events
An event is any subset of the sample space. Events are usually denoted by capital letters
such as A, B, or C.
Example
If a die is rolled once, then
S = {1, 2, 3, 4, 5, 6}.
Let A be the event of getting an even number. Then
A = {2, 4, 6}.
2
Basic Probability
A First Course in Probability
Practical Example
In a hospital, the event may be:
A = {Patient recovers after treatment}.
A die is rolled once. Find the probability of getting an even number.
The sample space is
S = {1, 2, 3, 4, 5, 6}.
The event of getting an even number is
A = {2, 4, 6}.
Therefore,
P (A) =
1
3
= .
6
2
Hence,
P (A) =
5
1
2
Axioms of Probability
Probability theory is based on three fundamental axioms.
Axiom 1: Non-negativity
For any event A,
P (A) ≥ 0.
Axiom 2: Probability of Sample Space
P (S) = 1.
Axiom 3: Additivity
If A and B are mutually exclusive events, then
P (A ∪ B) = P (A) + P (B).
Suppose a student can either get a distinction or a normal pass, but not both. Then
these events are mutually exclusive.
P (Distinction or Pass) = P (Distinction) + P (Pass).
.
A die is rolled once. Let
3
Basic Probability
A First Course in Probability
A = {getting 2},
B = {getting 5}.
Find P (A ∪ B).
The events A and B are mutually exclusive because both cannot occur at the same
time.
1
P (A) = ,
6
P (B) =
1
6
Using the additivity axiom,
P (A ∪ B) = P (A) + P (B)
P (A ∪ B) =
1 1
2
1
+ = = .
6 6
6
3
Hence,
P (A ∪ B) =
6
Rules of Probability
6.1
Complement Rule
1
3
If Ac denotes the complement of event A, then
P (Ac ) = 1 − P (A).
If
P (Rain) = 0.70,
then
P (No Rain) = 1 − 0.70 = 0.30.
The probability that a student passes an examination is 0.85. Find the probability
that the student fails.
Let A be the event that the student passes.
P (A) = 0.85
The probability of failing is
P (Ac ) = 1 − P (A) = 1 − 0.85 = 0.15.
Hence,
P (Ac ) = 0.15
4
Basic Probability
6.2
A First Course in Probability
Addition Rule
For any two events A and B,
P (A ∪ B) = P (A) + P (B) − P (A ∩ B).
In a class, 40% of students like cricket, 30% like football, and 10% like both. Find
the probability that a randomly selected student likes cricket or football.
Let C be the event that a student likes cricket and F be the event that a student likes
football.
P (C) = 0.40,
P (F ) = 0.30,
P (C ∩ F ) = 0.10.
Using the addition rule,
P (C ∪ F ) = P (C) + P (F ) − P (C ∩ F ).
P (C ∪ F ) = 0.40 + 0.30 − 0.10 = 0.60.
Hence,
P (C ∪ F ) = 0.60
7
Conditional Probability
Conditional probability is the probability of an event occurring given that another event
has already occurred.
If P (B) > 0, then
P (A|B) =
P (A ∩ B)
.
P (B)
Suppose a customer has already purchased a laptop. We may want to find the probability that the customer also buys a warranty.
P (Warranty|Laptop Purchased)
A box contains 5 red balls and 3 blue balls. Two balls are drawn without replacement.
Find the probability that the second ball is red given that the first ball is red.
Initially, there are 5 red balls and 3 blue balls.
If the first ball is red, then the remaining balls are:
4 red balls and 3 blue balls.
The total number of remaining balls is
7.
Therefore,
4
P (Second red|First red) = .
7
5
Basic Probability
A First Course in Probability
Hence,
4
7
8
Dependent and Independent Events
8.1
Independent Events
Two events A and B are independent if the occurrence of one does not affect the probability of the other.
Mathematically,
P (A ∩ B) = P (A)P (B).
Equivalently,
P (A|B) = P (A).
Tossing two coins is an example of independent events because the result of one toss
does not affect the result of the other toss.
A coin is tossed and a die is rolled. Find the probability of getting a head and an
even number.
Let H be the event of getting a head.
1
2
Let E be the event of getting an even number on the die.
P (H) =
E = {2, 4, 6}
3
1
=
6
2
Since the coin toss and die roll are independent,
P (E) =
P (H ∩ E) = P (H)P (E).
P (H ∩ E) =
Hence,
1
4
6
1 1
1
× = .
2 2
4
Basic Probability
8.2
A First Course in Probability
Dependent Events
Two events are dependent if the occurrence of one event affects the probability of the
other event.
Drawing cards without replacement is an example of dependent events because the
first draw changes the composition of the deck.
Stock prices of two companies in the same industry may be dependent because common market conditions affect both.
A bag contains 4 white balls and 6 black balls. Two balls are drawn without replacement. Find the probability that both balls are white.
Probability that the first ball is white:
4
.
10
After one white ball is drawn, there are 3 white balls left out of 9 balls.
Therefore,
P (W1 ) =
3
P (W2 |W1 ) = .
9
Thus,
P (W1 ∩ W2 ) = P (W1 )P (W2 |W1 ).
P (W1 ∩ W2 ) =
4
3
12
2
× =
= .
10 9
90
15
Hence,
2
15
9
Bayes’ Theorem
Bayes’ theorem is used to revise probabilities when new information becomes available.
If A1 , A2 , . . . , An are mutually exclusive and exhaustive events, then
P (B|Ai )P (Ai )
P (Ai |B) = Pn
.
j=1 P (B|Aj )P (Aj )
For two events A and B, Bayes’ theorem is
P (A|B) =
P (B|A)P (A)
.
P (B)
In medical testing, Bayes’ theorem helps find the probability that a person actually
has a disease given that the test result is positive.
A company uses a test to identify defective products. Suppose:
P (D) = 0.05
where D is the event that the product is defective.
The test correctly identifies a defective product with probability
7
Basic Probability
A First Course in Probability
P (T |D) = 0.90.
The test wrongly marks a good product as defective with probability
P (T |Dc ) = 0.10.
Find the probability that a product is actually defective given that the test says it is
defective.
We need to find
P (D|T ).
By Bayes’ theorem,
P (D|T ) =
P (T |D)P (D)
.
P (T )
Now,
P (T ) = P (T |D)P (D) + P (T |Dc )P (Dc ).
Also,
P (Dc ) = 1 − P (D) = 1 − 0.05 = 0.95.
Therefore,
P (T ) = 0.90(0.05) + 0.10(0.95).
P (T ) = 0.045 + 0.095 = 0.140.
Hence,
0.90(0.05)
0.045
=
= 0.3214.
0.140
0.140
Therefore, the probability that the product is actually defective given that the test
says defective is approximately
P (D|T ) =
0.3214
or
32.14% .
8
Lecture Notes on Random Variables
1
Introduction to Random Variables
In probability theory, a random variable is a numerical quantity associated with the
outcomes of a random experiment.
A random variable assigns a real number to each outcome in the sample space.
Examples
1. Tossing two coins:
Let X denote the number of heads obtained.
S = {HH, HT, T H, T T }
Then
X(HH) = 2,
X(HT ) = 1,
X(T H) = 1,
X(T T ) = 0
2. Rolling a die:
Let X be the number obtained on the die.
X ∈ {1, 2, 3, 4, 5, 6}
2
Types of Random Variables
Random variables are broadly classified into two types:
1. Discrete Random Variables
2. Continuous Random Variables
2.1
Discrete Random Variables
A random variable is called discrete if it takes countable values.
Examples include:
• Number of heads in coin tosses
• Number of defective items
• Number of customers arriving at a store
1
Random Variables
Sheldon Ross – First Course on Probability
Example
Suppose two coins are tossed.
Let X be the number of heads.
Possible values:
X ∈ {0, 1, 2}
2.2
Continuous Random Variables
A random variable is called continuous if it can take infinitely many values in an interval.
Examples include:
• Height of students
• Time taken to complete a task
• Temperature
• Daily rainfall
Example
Let X denote the lifetime of a battery measured in hours.
X ∈ [0, ∞)
3
Discrete Random Variables
A discrete random variable assumes finite or countably infinite values.
Practical Examples
• Number of students absent in a class
• Number of insurance claims
• Number of accidents in a city
4
Probability Mass Function (PMF)
For a discrete random variable X, the probability mass function (PMF) is defined as
p(x) = P (X = x)
The PMF satisfies:
1. p(x) ≥ 0
2.
X
p(x) = 1
x
2
Random Variables
4.1
Sheldon Ross – First Course on Probability
Example: Tossing Two Coins
Suppose two fair coins are tossed.
Let X be the number of heads obtained.
Step 1: Sample Space
S = {HH, HT, T H, T T }
Step 2: Values of X
0 for T T
X = 1 for HT, T H
2 for HH
Step 3: PMF Table
4.2
x
P (X = x)
0
1
2
1
4
1
2
1
4
Graph of PMF
P (X = x)
0.6
0.4
0.2
0
0
4.3
1
x
Solved Problem
A fair die is rolled once. Let X denote the number obtained.
Find the PMF of X.
Solution
Possible values:
X ∈ {1, 2, 3, 4, 5, 6}
Since the die is fair,
3
2
Random Variables
Sheldon Ross – First Course on Probability
1
P (X = x) = ,
6
x = 1, 2, 3, 4, 5, 6
Thus,
(
p(x) =
5
1
,
6
0,
x = 1, 2, 3, 4, 5, 6
otherwise
Cumulative Distribution Function (CDF)
The cumulative distribution function (CDF) of a random variable X is defined as
F (x) = P (X ≤ x)
The CDF gives the probability that the random variable is less than or equal to a
given value.
5.1
Example: Two Coin Tosses
Recall the PMF:
x
P (X = x)
0
1
2
1
4
1
2
1
4
Then,
0,
1,
F (x) = 43
,
4
1,
Graph of CDF
1
F (x)
5.2
x<0
0≤x<1
1≤x<2
x≥2
0.5
0
−1 −0.5
0
0.5
1
x
4
1.5
2
2.5
3
Random Variables
5.3
Sheldon Ross – First Course on Probability
Solved Problem
A fair die is rolled once. Find the CDF of X.
Solution
Since
1
P (X = x) = ,
6
x = 1, 2, 3, 4, 5, 6
The CDF is
0,
1
,
6
2
6,
F (x) = 36 ,
4
,
6
5
,
6
1,
6
x<1
1≤x<2
2≤x<3
3≤x<4
4≤x<5
5≤x<6
x≥6
Expectation of a Discrete Random Variable
The expectation or mean of a discrete random variable X is
X
E(X) =
xP (X = x)
x
Expectation represents the long-run average value of the random variable.
6.1
Example
Suppose
x
P (X = x)
0
1
2
0.2
0.5
0.3
Find E(X).
Solution
E(X) =
X
xP (X = x)
x
E(X) = 0(0.2) + 1(0.5) + 2(0.3)
E(X) = 0 + 0.5 + 0.6
5
Random Variables
Sheldon Ross – First Course on Probability
E(X) = 1.1
Hence,
E(X) = 1.1
6.2
Practical Interpretation
If this experiment is repeated many times, the average value of X will be approximately
1.1.
7
Properties of Expectation
7.1
Linearity Property
For constants a, b,
E(aX + b) = aE(X) + b
7.2
Solved Problem
Suppose E(X) = 5.
Find E(3X + 2).
Solution
Using linearity,
E(3X + 2) = 3E(X) + 2
E(3X + 2) = 3(5) + 2
E(3X + 2) = 17
Hence,
17
8
Variance of a Random Variable
Variance measures the spread or variability of a random variable around its mean.
The variance of X is
V ar(X) = E[(X − E(X))2 ]
An equivalent formula is
V ar(X) = E(X 2 ) − [E(X)]2
6
Random Variables
8.1
Sheldon Ross – First Course on Probability
Example
Suppose
x
P (X = x)
0
1
2
0.2
0.5
0.3
Find the variance of X.
Solution
First compute the mean:
E(X) = 0(0.2) + 1(0.5) + 2(0.3) = 1.1
Now compute
E(X 2 )
E(X 2 ) = 02 (0.2) + 12 (0.5) + 22 (0.3)
E(X 2 ) = 0 + 0.5 + 1.2
E(X 2 ) = 1.7
Therefore,
V ar(X) = E(X 2 ) − [E(X)]2
V ar(X) = 1.7 − (1.1)2
V ar(X) = 1.7 − 1.21
V ar(X) = 0.49
Hence,
V ar(X) = 0.49
9
Standard Deviation
The standard deviation of a random variable is the square root of variance.
SD(X) =
p
V ar(X)
For the previous example,
SD(X) =
√
7
0.49 = 0.7
Random Variables
10
Sheldon Ross – First Course on Probability
Practice Problems
1. A coin is tossed three times. Let X denote the number of heads. Construct the
PMF of X.
2. A random variable has PMF
P (X = x) =
x
,
10
x = 1, 2, 3, 4
Find:
(a) E(X)
(b) V ar(X)
3. A die is rolled twice. Let X denote the larger number obtained. Find the PMF of
X.
4. Construct the CDF corresponding to the following PMF:
x
P (X = x)
0
1
2
3
0.1
0.4
0.3
0.2
8
Lecture Notes on Bernoulli, Binomial and
Poisson Distributions
Based on A First Course in Probability by Sheldon Ross
1
Introduction to Probability Distributions
A probability distribution describes how probabilities are assigned to the possible values
of a random variable.
Discrete probability distributions are commonly used to model:
• Success or failure outcomes
• Number of defective items
• Number of arrivals
• Number of accidents or claims
In these notes, we study three important discrete distributions:
1. Bernoulli Distribution
2. Binomial Distribution
3. Poisson Distribution
2
Bernoulli Distribution
2.1
Introduction
A Bernoulli distribution models a random experiment having only two possible outcomes:
• Success
• Failure
Examples include:
• Tossing a coin
• Whether a customer purchases a product
• Whether a student passes an exam
• Whether a machine fails
1
Probability Distributions
2.2
Sheldon Ross – First Course on Probability
Definition
A random variable X is said to follow a Bernoulli distribution if
(
1, Success
X=
0, Failure
with
P (X = 1) = p
and
P (X = 0) = 1 − p
where
0 < p < 1.
We write
X ∼ Bernoulli(p).
2.3
Probability Mass Function
The PMF is
(
p,
x=1
P (X = x) =
1 − p, x = 0
2.4
Graph of Bernoulli PMF
Suppose p = 0.7.
P (X = x)
0.8
0.6
0.4
0.2
0
0
1
x
2
Probability Distributions
2.5
Sheldon Ross – First Course on Probability
Mean and Variance
For a Bernoulli random variable,
E(X) = p
and
V ar(X) = p(1 − p).
2.6
Solved Problem
A machine produces defective products with probability 0.08.
Let
(
1, if defective
X=
0, otherwise
Find:
1. P (X = 0)
2. Mean
3. Variance
Solution
Given
p = 0.08
Then
P (X = 1) = 0.08
and
P (X = 0) = 1 − 0.08 = 0.92.
Mean:
E(X) = p = 0.08
Variance:
V ar(X) = p(1 − p)
V ar(X) = 0.08(0.92) = 0.0736
Hence,
P (X = 0) = 0.92
3
Probability Distributions
Sheldon Ross – First Course on Probability
E(X) = 0.08
V ar(X) = 0.0736
3
Binomial Distribution
3.1
Introduction
The binomial distribution models the number of successes in n independent Bernoulli
trials.
Examples include:
• Number of heads in repeated coin tosses
• Number of defective items in a batch
• Number of customers who buy a product
• Number of students passing an exam
3.2
Conditions for Binomial Distribution
A random experiment follows a binomial distribution if:
1. There are fixed number of trials n
2. Each trial has two outcomes:
• Success
• Failure
3. Probability of success is constant
4. Trials are independent
3.3
Definition
A random variable X follows a binomial distribution if
X ∼ Binomial(n, p)
where:
• n = number of trials
• p = probability of success
4
Probability Distributions
3.4
Sheldon Ross – First Course on Probability
Probability Mass Function
The PMF is
n x
P (X = x) =
p (1 − p)n−x
x
for
x = 0, 1, 2, . . . , n.
3.5
Explanation of Formula
n
x
counts the number of ways of obtaining x successes in n trials.
px
represents probability of x successes.
(1 − p)n−x
represents probability of failures.
3.6
Example: Tossing 3 Coins
Suppose 3 fair coins are tossed.
Let X denote the number of heads.
Then,
X ∼ Binomial(3, 0.5)
3.7
PMF Table
x
P (X = x)
0
1
2
3
1
8
3
8
3
8
1
8
5
Probability Distributions
3.8
Sheldon Ross – First Course on Probability
Graph of Binomial PMF
P (X = x)
0.4
0.2
0
0
1
2
3
x
3.9
Mean and Variance
If
X ∼ Binomial(n, p)
then
E(X) = np
and
V ar(X) = np(1 − p).
3.10
Solved Problem
A multiple-choice test contains 5 questions. Each question has probability 0.6 of being
answered correctly.
Find the probability that exactly 3 questions are answered correctly.
Solution
Let X denote number of correct answers.
Then,
X ∼ Binomial(5, 0.6)
We need:
P (X = 3)
Using the PMF:
5
P (X = 3) =
(0.6)3 (0.4)2
3
Now,
6
Probability Distributions
Sheldon Ross – First Course on Probability
5
5!
= 10
=
3!2!
3
Thus,
P (X = 3) = 10(0.216)(0.16)
P (X = 3) = 0.3456
Hence,
P (X = 3) = 0.3456
3.11
Solved Problem
A factory produces bulbs with probability 0.1 of being defective. If 8 bulbs are selected
randomly, find:
1. Probability exactly 2 are defective
2. Mean
3. Variance
Solution
Let X denote number of defective bulbs.
X ∼ Binomial(8, 0.1)
Part 1
8
P (X = 2) =
(0.1)2 (0.9)6
2
Now,
8
= 28
2
Therefore,
P (X = 2) = 28(0.01)(0.531441)
P (X = 2) = 0.1488
Part 2: Mean
E(X) = np = 8(0.1) = 0.8
7
Probability Distributions
Sheldon Ross – First Course on Probability
Part 3: Variance
V ar(X) = np(1 − p)
V ar(X) = 8(0.1)(0.9) = 0.72
Hence,
P (X = 2) = 0.1488
E(X) = 0.8
V ar(X) = 0.72
4
Poisson Distribution
4.1
Introduction
The Poisson distribution models the number of occurrences of an event in a fixed interval
of time, area, distance, or volume.
Examples include:
• Number of phone calls arriving per minute
• Number of accidents in a city
• Number of typing errors in a page
• Number of customers arriving at a counter
4.2
Definition
A random variable X follows a Poisson distribution if
X ∼ P oisson(λ)
where
λ>0
is the average number of occurrences.
4.3
Probability Mass Function
The PMF is
P (X = x) =
e−λ λx
x!
for
x = 0, 1, 2, . . .
8
Probability Distributions
4.4
Sheldon Ross – First Course on Probability
Properties
For a Poisson random variable,
E(X) = λ
and
V ar(X) = λ.
Thus, for a Poisson distribution,
Mean = Variance
4.5
Graph of Poisson PMF
Suppose λ = 3.
P (X = x)
0.2
0.1
0
0
4.6
1
2
3
x
4
5
Solved Problem
A call center receives an average of 4 calls per minute.
Find the probability that exactly 3 calls arrive in a minute.
Solution
Let X denote number of calls.
X ∼ P oisson(4)
We need:
P (X = 3)
Using the PMF,
P (X = 3) =
P (X = 3) =
9
e−4 43
3!
e−4 (64)
6
6
Probability Distributions
Sheldon Ross – First Course on Probability
Using
e−4 ≈ 0.0183
we get
P (X = 3) = 0.1954
Hence,
P (X = 3) = 0.1954
4.7
Solved Problem
A hospital receives an average of 2 emergency cases per hour.
Find:
1. Probability of no emergency cases
2. Probability of at least one emergency case
Solution
Let
X ∼ P oisson(2)
Part 1
P (X = 0) =
e−2 20
0!
P (X = 0) = e−2
P (X = 0) = 0.1353
Part 2
P (X ≥ 1) = 1 − P (X = 0)
P (X ≥ 1) = 1 − 0.1353
P (X ≥ 1) = 0.8647
Hence,
P (X = 0) = 0.1353
P (X ≥ 1) = 0.8647
10
Probability Distributions
5
Sheldon Ross – First Course on Probability
Relationship Between Binomial and Poisson
The Poisson distribution can approximate the binomial distribution when:
• n is large
• p is small
• λ = np
Thus,
Binomial(n, p) ≈ P oisson(np)
6
Applications
6.1
Bernoulli Distribution
Applications:
• Pass/fail experiments
• Product defect analysis
• Customer purchase behavior
6.2
Binomial Distribution
Applications:
• Election polling
• Quality control
• Medical trials
• Reliability analysis
6.3
Poisson Distribution
Applications:
• Queueing theory
• Traffic analysis
• Insurance claims
• Network traffic modeling
11
Probability Distributions
7
Sheldon Ross – First Course on Probability
Summary Table
Distribution
PMF
Mean
Variance
Bernoulli
px (1 − p)1−x
p
p(1 − p)
Binomial
n x
p (1−p)n−x
x
Poisson
8
e−λ λx
x!
np
np(1 − p)
λ
λ
Practice Problems
1. A biased coin has probability 0.7 of heads. Find:
(a) PMF of the Bernoulli random variable
(b) Mean
(c) Variance
2. A student guesses answers to 10 multiple-choice questions where probability of
correct answer is 0.25. Find:
(a) Probability exactly 4 answers are correct
(b) Probability at least one answer is correct
3. A website receives an average of 5 requests per minute. Find:
(a) Probability exactly 2 requests arrive
(b) Probability no request arrives
4. A factory produces products with defect probability 0.02. If 100 products are
inspected:
(a) Find probability exactly 3 are defective using Binomial distribution
(b) Approximate using Poisson distribution
12
Lecture Notes on Continuous Random
Variables
1
Introduction to Continuous Random Variables
A random variable is called a continuous random variable if it can take infinitely
many values over an interval.
Unlike discrete random variables, continuous random variables are measured rather
than counted.
Examples
• Height of students
• Weight of a product
• Time taken to complete a task
• Lifetime of a machine
• Daily rainfall
• Temperature
Important Property
For a continuous random variable X,
P (X = x) = 0
for every value x.
Thus probabilities are assigned over intervals:
P (a < X < b)
rather than at individual points.
1
Continuous Random Variables
Sheldon Ross – First Course on Probability
2
Probability Density Function (PDF)
2.1
Definition
A continuous random variable X has a probability density function (PDF) f (x) if
Z b
f (x) dx
P (a ≤ X ≤ b) =
a
for all real numbers a and b.
2.2
Properties of PDF
A valid PDF satisfies:
1.
f (x) ≥ 0
2.
Z ∞
f (x) dx = 1
−∞
2.3
Interpretation
The PDF describes the relative likelihood of the random variable near a point.
Important:
f (x) ̸= P (X = x)
For continuous random variables,
P (X = x) = 0.
3
Example of a Probability Density Function
Suppose
(
2x, 0 < x < 1
f (x) =
0,
otherwise
3.1
Checking Validity
We verify:
Z 1
0
1
2x dx = x2 0 = 1
Hence, this is a valid PDF.
2
Continuous Random Variables
3.2
Sheldon Ross – First Course on Probability
Graph of PDF
f (x)
2
1
0
0
0.2
0.4
0.6
x
0.8
4
Finding Probabilities Using PDF
4.1
Solved Problem
Suppose
(
2x, 0 < x < 1
f (x) =
0,
otherwise
Find:
P (0.2 < X < 0.8)
Solution
Using the definition,
Z 0.8
P (0.2 < X < 0.8) =
2x dx
0.2
0.8
= x2 0.2
= 0.82 − 0.22
= 0.64 − 0.04
= 0.60
Hence,
P (0.2 < X < 0.8) = 0.60
3
1
1.2
Continuous Random Variables
Sheldon Ross – First Course on Probability
5
Cumulative Distribution Function (CDF)
5.1
Definition
The cumulative distribution function (CDF) of a continuous random variable X is defined
as
F (x) = P (X ≤ x)
For continuous random variables,
Z x
F (x) =
f (t) dt
−∞
5.2
Properties of CDF
1.
0 ≤ F (x) ≤ 1
2. F (x) is non-decreasing
3.
lim F (x) = 0
x→−∞
4.
lim F (x) = 1
x→∞
6
Example of CDF
Suppose
(
2x, 0 < x < 1
f (x) =
0,
otherwise
Find the CDF.
Solution
We compute:
Z x
F (x) =
f (t) dt
−∞
Case 1: x ≤ 0
F (x) = 0
4
Continuous Random Variables
Sheldon Ross – First Course on Probability
Case 2: 0 < x < 1
Z x
2t dt
F (x) =
0
x
= t2 0
= x2
Case 3: x ≥ 1
F (x) = 1
Thus,
0, x ≤ 0
F (x) = x2 , 0 < x < 1
1, x ≥ 1
6.1
Graph of CDF
F (x)
1
0.5
0
−0.2
0
0.2
0.4
0.6
0.8
1
1.2
x
7
Expectation of a Continuous Random Variable
7.1
Definition
The expectation or mean of a continuous random variable is
Z ∞
E(X) =
xf (x) dx
−∞
Expectation represents the long-run average value of the random variable.
8
Example: Finding Expectation
Suppose
5
Continuous Random Variables
Sheldon Ross – First Course on Probability
(
2x, 0 < x < 1
f (x) =
0,
otherwise
Find E(X).
Solution
Using the formula,
Z 1
x(2x) dx
E(X) =
0
Z 1
=2
x2 dx
0
3 1
x
=2
3 0
=
2
3
Hence,
E(X) =
2
3
9
Variance of a Continuous Random Variable
9.1
Definition
Variance measures the spread of the random variable around the mean.
V ar(X) = E(X 2 ) − [E(X)]2
where
2
Z ∞
E(X ) =
x2 f (x) dx
−∞
10
Example: Finding Variance
Suppose
(
2x, 0 < x < 1
f (x) =
0,
otherwise
Find V ar(X).
6
Continuous Random Variables
Sheldon Ross – First Course on Probability
Solution
We already found:
E(X) =
2
3
Now compute:
Z 1
2
E(X ) =
x2 (2x) dx
0
Z 1
=2
x3 dx
0
4 1
x
=2
4 0
1
2
=
Therefore,
V ar(X) = E(X 2 ) − [E(X)]2
1
= −
2
2
2
3
=
1 4
−
2 9
=
9−8
18
=
1
18
Hence,
V ar(X) =
11
1
18
Standard Deviation
The standard deviation is the square root of variance.
SD(X) =
p
V ar(X)
For the previous example,
r
SD(X) =
7
1
18
Continuous Random Variables
Sheldon Ross – First Course on Probability
12
Uniform Distribution
12.1
Definition
A continuous random variable X has a uniform distribution on (a, b) if
(
1
, a<x<b
b−a
f (x) =
0,
otherwise
12.2
Graph of Uniform PDF
For a = 0 and b = 2,
f (x) =
1
2
0.8
f (x)
0.6
0.4
0.2
0
−0.5
12.3
0
0.5
1
x
1.5
2
2.5
Mean and Variance
For a uniform random variable,
E(X) =
a+b
2
V ar(X) =
(b − a)2
12
and
13
Solved Problem: Uniform Distribution
Suppose
X ∼ U nif orm(2, 8)
Find:
1. P (3 < X < 6)
2. Mean
3. Variance
8
Continuous Random Variables
Sheldon Ross – First Course on Probability
Solution
The PDF is
1
1
=
8−2
6
f (x) =
Part 1
Z 6
P (3 < X < 6) =
3
1
dx
6
1
= (6 − 3)
6
=
3
1
=
6
2
Part 2: Mean
E(X) =
2+8
=5
2
Part 3: Variance
V ar(X) =
=
(8 − 2)2
12
36
=3
12
Hence,
P (3 < X < 6) =
1
2
E(X) = 5
V ar(X) = 3
14
Applications of Continuous Random Variables
Continuous random variables are widely used in:
• Reliability engineering
• Queueing theory
• Financial modeling
• Inventory systems
• Machine learning
• Medical statistics
• Weather forecasting
9
Continuous Random Variables
15
16
Sheldon Ross – First Course on Probability
Summary Table
Concept
Description
Continuous Random
Variable
PDF
CDF
Expectation
Variance
Standard Deviation
Uniform Distribution
Can take infinitely many values over intervals
Function used to compute probabilities
Cumulative probability function
Long-run average value
Measures spread around the mean
Square root of variance
Equal density over an interval
Practice Problems
1. Suppose
(
3x2 , 0 < x < 1
f (x) =
0,
otherwise
Find:
(a) Verify that f (x) is a PDF
(b) P (0.2 < X < 0.6)
(c) CDF
2. Suppose
(
4x3 , 0 < x < 1
f (x) =
0,
otherwise
Find:
(a) E(X)
(b) V ar(X)
3. Suppose X ∼ U nif orm(1, 5).
Find:
(a) P (2 < X < 4)
(b) Mean
(c) Variance
4. A random variable has PDF
(
k(1 − x2 ), −1 < x < 1
f (x) =
0,
otherwise
Find:
10
Continuous Random Variables
Sheldon Ross – First Course on Probability
(a) Value of k
(b) P (0 < X < 0.5)
11
Lecture Notes on Uniform and Exponential
Distributions
1
Introduction
Continuous probability distributions are used to model measurements, waiting times,
lifetimes, and many naturally occurring random phenomena.
In these notes, we study:
1. Uniform Distribution
2. Exponential Distribution
along with:
• Probability Density Functions
• Cumulative Distribution Functions
• Derivation of expectation and variance
• Relationship between Poisson and Exponential distributions
2
Uniform Distribution
2.1
Introduction
A continuous random variable X is said to follow a uniform distribution if all values
in an interval are equally likely.
Examples:
• Random arrival time within an hour
• Randomly selecting a point on a line segment
• Random number generation
1
Continuous Distributions
2.2
Sheldon Ross – First Course on Probability
Definition
A random variable X has a uniform distribution on (a, b) if its PDF is
1 , a<x<b
f (x) = b − a
0,
otherwise
We write
X ∼ U nif orm(a, b)
2.3
Why is it Called Uniform?
The density is constant over the interval (a, b).
Thus every interval of equal length has equal probability.
3
Graph of Uniform Distribution
Suppose
X ∼ U nif orm(0, 4)
Then
f (x) =
1
4
for 0 < x < 4.
0.4
f (x)
0.3
0.2
0.1
0
−1
4
0
1
2
x
CDF of Uniform Distribution
The cumulative distribution function is
F (x) = P (X ≤ x)
For the uniform distribution,
2
3
4
5
Continuous Distributions
Sheldon Ross – First Course on Probability
x≤a
0,
x−a
, a<x<b
F (x) =
b−a
1,
x≥b
4.1
Example
If
X ∼ U nif orm(0, 4)
then
x≤0
0,
x
, 0<x<4
F (x) =
4
1, x ≥ 4
5
Derivation of Expectation of Uniform Distribution
Suppose
X ∼ U nif orm(a, b)
Then
f (x) =
1
b−a
The expectation is
Z b
E(X) =
xf (x) dx
a
Substituting the PDF,
Z b 1
E(X) =
x
dx
b−a
a
Z b
1
=
x dx
b−a a
2 b
1
x
=
b−a 2 a
2
1
b − a2
=
b−a
2
Factorizing:
b2 − a2 = (b − a)(b + a)
Thus,
3
Continuous Distributions
Sheldon Ross – First Course on Probability
E(X) =
(b − a)(a + b)
2(b − a)
Hence,
a+b
2
E(X) =
6
Derivation of Variance of Uniform Distribution
We use
V ar(X) = E(X 2 ) − [E(X)]2
Z b
2
E(X ) =
x2 f (x) dx
a
1
=
b−a
Z b
x2 dx
a
3 b
1
x
=
b−a 3 a
b3 − a 3
=
3(b − a)
Using
b3 − a3 = (b − a)(b2 + ab + a2 )
we get
E(X 2 ) =
b2 + ab + a2
3
Now,
V ar(X) = E(X 2 ) − [E(X)]2
b2 + ab + a2
=
−
3
a+b
2
Simplifying,
V ar(X) =
4
(b − a)2
12
2
Continuous Distributions
Sheldon Ross – First Course on Probability
7
Solved Problems on Uniform Distribution
7.1
Solved Problem 1
Suppose
X ∼ U nif orm(2, 8)
Find:
1. P (3 < X < 6)
2. E(X)
3. V ar(X)
Solution
The PDF is
f (x) =
1
1
=
8−2
6
Part 1
Z 6
P (3 < X < 6) =
3
1
dx
6
1
1
= (6 − 3) =
6
2
Part 2
E(X) =
2+8
=5
2
Part 3
V ar(X) =
(8 − 2)2
36
=
=3
12
12
Hence,
P (3 < X < 6) =
E(X) = 5
V ar(X) = 3
5
1
2
Continuous Distributions
Sheldon Ross – First Course on Probability
8
Exponential Distribution
8.1
Introduction
The exponential distribution is widely used to model waiting times.
Examples include:
• Time between customer arrivals
• Lifetime of electronic components
• Waiting time for buses
• Time until machine failure
8.2
Definition
A random variable X has an exponential distribution with parameter λ > 0 if
(
λe−λx , x > 0
f (x) =
0,
otherwise
We write
X ∼ Exponential(λ)
9
Graph of Exponential Distribution
Suppose
λ=1
Then
f (x) = e−x
f (x)
1
0.5
0
0
1
2
3
x
6
4
5
Continuous Distributions
10
Sheldon Ross – First Course on Probability
CDF of Exponential Distribution
F (x) = P (X ≤ x)
Thus,
Z x
F (x) =
λe−λt dt
0
Integrating,
x
F (x) = −e−λt 0
= 1 − e−λx
Hence,
(
1 − e−λx , x > 0
F (x) =
0,
x≤0
11
Derivation of Expectation of Exponential Distribution
We compute
Z ∞
xλe−λx dx
E(X) =
0
Using integration by parts:
Let
u = x,
dv = λe−λx dx
du = dx,
v = −e−λx
Then
Thus,
∞
E(X) = −xe−λx 0 +
Z ∞
e−λx dx
0
The first term becomes 0.
Hence,
∞
1 −λx
E(X) = − e
λ
0
=
1
λ
Thus,
E(X) =
7
1
λ
Continuous Distributions
12
Sheldon Ross – First Course on Probability
Derivation of Variance of Exponential Distribution
We use
V ar(X) = E(X 2 ) − [E(X)]2
2
Z ∞
E(X ) =
x2 λe−λx dx
0
Using integration by parts twice,
E(X 2 ) =
2
λ2
E(X) =
1
λ
Since
we get
2
V ar(X) = 2 −
λ
=
2
1
λ
1
λ2
Hence,
V ar(X) =
13
1
λ2
Memoryless Property
The exponential distribution has the memoryless property:
P (X > s + t|X > s) = P (X > t)
Interpretation:
The future waiting time does not depend on how long we have already waited.
14
Relationship Between Poisson and Exponential
Distributions
Suppose arrivals follow a Poisson process with rate λ.
Then:
P (N (t) = k) =
where:
8
e−λt (λt)k
k!
Continuous Distributions
Sheldon Ross – First Course on Probability
• N (t) = number of arrivals by time t
• λ = average arrival rate
Let T denote the waiting time until the first arrival.
We compute:
P (T > t)
The event T > t means:
No arrivals occur in the interval (0, t).
Thus,
P (T > t) = P (N (t) = 0)
Using the Poisson distribution,
P (N (t) = 0) =
e−λt (λt)0
0!
= e−λt
Hence,
P (T > t) = e−λt
Therefore,
FT (t) = P (T ≤ t) = 1 − P (T > t)
= 1 − e−λt
Differentiating,
fT (t) = λe−λt
which is exactly the exponential density.
Thus,
T ∼ Exponential(λ)
15
Solved Problems on Exponential Distribution
15.1
Solved Problem 1
Customer arrivals follow an exponential distribution with rate
λ=2
Find the probability that the waiting time exceeds 1 hour.
9
Continuous Distributions
Sheldon Ross – First Course on Probability
Solution
We use
P (X > t) = e−λt
Thus,
P (X > 1) = e−2
= 0.1353
Hence,
P (X > 1) = 0.1353
16
Solved Problem 2
Suppose
X ∼ Exponential(0.5)
Find:
1. Mean
2. Variance
Solution
E(X) =
1
1
=
=2
λ
0.5
Variance:
V ar(X) =
1
1
=
=4
2
λ
(0.5)2
Hence,
E(X) = 2
V ar(X) = 4
17
Applications
17.1
Uniform Distribution
Applications:
• Random number generation
10
Continuous Distributions
Sheldon Ross – First Course on Probability
• Simulation
• Scheduling problems
• Randomized algorithms
17.2
Exponential Distribution
Applications:
• Queueing theory
• Reliability engineering
• Survival analysis
• Inventory systems
• Telecommunication systems
18
Summary Table
Distribution
Uniform
PDF
Mean
1
b−a
a+b
2
λe−λx
1
λ
Exponential
19
Variance
Practice Problems
1. Suppose
X ∼ U nif orm(1, 7)
Find:
(a) P (2 < X < 5)
(b) Mean
(c) Variance
2. Suppose
X ∼ Exponential(3)
Find:
11
(b − a)2
12
1
λ2
Continuous Distributions
Sheldon Ross – First Course on Probability
(a) P (X > 1)
(b) Mean
(c) Variance
3. Show that if arrivals follow a Poisson process with rate λ, then the waiting time
until the second arrival follows a Gamma distribution.
4. Suppose the lifetime of a machine follows an exponential distribution with mean 5
years. Find:
(a) λ
(b) Probability machine survives more than 3 years
12
Lecture Notes on Normal Distribution
1
Introduction
The normal distribution is one of the most important probability distributions in statistics
and probability theory.
It arises naturally in many real-world situations involving measurements and averages.
Examples include:
• Heights of individuals
• Weights of products
• Measurement errors
• Test scores
• IQ scores
• Financial returns
The normal distribution is also called the Gaussian distribution.
2
Normal Distribution
2.1
Definition
A continuous random variable X is said to follow a normal distribution with parameters
µ and σ 2 if its probability density function is
(x − µ)2
1
exp −
f (x) = √
2σ 2
2πσ 2
for
−∞ < x < ∞
We write
X ∼ N (µ, σ 2 )
where:
• µ = mean
• σ 2 = variance
• σ = standard deviation
1
Normal Distribution
3
Sheldon Ross – First Course on Probability
Shape of the Normal Distribution
The normal curve has the following properties:
1. Bell-shaped
2. Symmetric about the mean
3. Mean = Median = Mode
4. Total area under the curve equals 1
5. Tails extend indefinitely
4
Graph of Normal Distribution
Suppose
X ∼ N (0, 1)
0.5
f (x)
0.4
0.3
0.2
0.1
0
−4
5
−3
−2
−1
0
x
1
2
3
Properties of the Normal Distribution
If
X ∼ N (µ, σ 2 )
then:
E(X) = µ
and
V ar(X) = σ 2
2
4
Normal Distribution
5.1
Sheldon Ross – First Course on Probability
Important Intervals
Approximately:
• 68% of observations lie within:
µ±σ
• 95% lie within:
µ ± 2σ
• 99.7% lie within:
µ ± 3σ
This is called the Empirical Rule.
6
Cumulative Distribution Function (CDF)
The cumulative distribution function is
F (x) = P (X ≤ x)
For the normal distribution, the CDF does not have a simple closed form.
Thus probabilities are usually obtained using:
• Standard normal tables
• Statistical software
• Calculators
7
Standard Normal Distribution
7.1
Definition
A normal random variable with:
µ=0
and
σ2 = 1
is called the standard normal distribution.
It is denoted by:
Z ∼ N (0, 1)
Its PDF is
1
2
f (z) = √ e−z /2
2π
3
Normal Distribution
8
Sheldon Ross – First Course on Probability
Standardization
Any normal random variable can be converted into a standard normal variable using:
X −µ
σ
This process is called standardization.
Z=
8.1
Why Standardization?
It allows us to use a single standard normal table for all normal distributions.
9
Example of Standardization
Suppose
X ∼ N (50, 9)
Find:
P (X < 53)
Solution
Given:
µ = 50,
σ=3
Standardize:
Z=
53 − 50
3
Z=1
Thus,
P (X < 53) = P (Z < 1)
From standard normal tables:
P (Z < 1) = 0.8413
Hence,
P (X < 53) = 0.8413
4
Normal Distribution
Sheldon Ross – First Course on Probability
10
Solved Problems on Normal Distribution
10.1
Solved Problem 1
The heights of students are normally distributed with:
µ = 170 cm
and
σ = 6 cm
Find the probability that a randomly selected student has height less than 178 cm.
Solution
Let X denote height.
X ∼ N (170, 36)
We need:
P (X < 178)
Standardize:
Z=
178 − 170
6
Z=
8
= 1.33
6
Thus,
P (X < 178) = P (Z < 1.33)
From standard normal tables:
P (Z < 1.33) = 0.9082
Hence,
P (X < 178) = 0.9082
10.2
Solved Problem 2
Suppose test scores are normally distributed with:
µ = 75,
σ = 10
Find the probability that a student scores between 70 and 90.
5
Normal Distribution
Sheldon Ross – First Course on Probability
Solution
We need:
P (70 < X < 90)
Standardize:
Z1 =
70 − 75
= −0.5
10
Z2 =
90 − 75
= 1.5
10
Thus,
P (70 < X < 90) = P (−0.5 < Z < 1.5)
Using standard normal tables:
P (Z < 1.5) = 0.9332
P (Z < −0.5) = 0.3085
Therefore,
P (−0.5 < Z < 1.5) = 0.9332 − 0.3085
= 0.6247
Hence,
P (70 < X < 90) = 0.6247
11
Normal Approximation to the Binomial Distribution
11.1
Introduction
When n is large, the binomial distribution can be approximated using a normal distribution.
This approximation simplifies probability calculations.
11.2
Conditions
If
X ∼ Binomial(n, p)
then for large n,
X ≈ N (np, np(1 − p))
6
Normal Distribution
Sheldon Ross – First Course on Probability
provided:
np ≥ 5
and
n(1 − p) ≥ 5
12
Example of Normal Approximation
Suppose:
X ∼ Binomial(100, 0.5)
Find:
P (X ≤ 60)
using the normal approximation.
Solution
Mean:
µ = np = 100(0.5) = 50
Variance:
σ 2 = np(1 − p) = 100(0.5)(0.5) = 25
Thus,
σ=5
Approximate:
X ≈ N (50, 25)
Apply continuity correction:
P (X ≤ 60) ≈ P (Y ≤ 60.5)
Standardize:
Z=
60.5 − 50
5
Z = 2.1
Therefore,
P (X ≤ 60) ≈ P (Z ≤ 2.1)
From standard normal tables:
7
Normal Distribution
Sheldon Ross – First Course on Probability
P (Z ≤ 2.1) = 0.9821
Hence,
P (X ≤ 60) ≈ 0.9821
13
Solved Problem on Normal Approximation
A factory produces items with probability 0.2 of being defective.
If 200 items are selected, find the probability that fewer than 50 are defective using
the normal approximation.
Solution
Let
X ∼ Binomial(200, 0.2)
Mean:
µ = np = 200(0.2) = 40
Variance:
σ 2 = np(1 − p) = 200(0.2)(0.8) = 32
Thus,
σ=
√
32 = 5.657
We need:
P (X < 50) = P (X ≤ 49)
Using continuity correction:
P (X ≤ 49) ≈ P (Y ≤ 49.5)
Standardize:
Z=
49.5 − 40
5.657
Z = 1.68
Thus,
P (X < 50) ≈ P (Z < 1.68)
From standard normal tables:
P (Z < 1.68) = 0.9535
8
Normal Distribution
Sheldon Ross – First Course on Probability
Hence,
P (X < 50) ≈ 0.9535
14
Applications of the Normal Distribution
The normal distribution is widely used in:
• Quality control
• Financial modeling
• Machine learning
• Social sciences
• Medical statistics
• Measurement theory
• Sampling distributions
15
Summary Table
Concept
Description
Normal Distribution
Parameters
Standard Normal
Distribution
Standardization
Bell-shaped continuous distribution
Mean µ, Variance σ 2
N (0, 1)
Z=
Empirical Rule
Normal Approximation
X −µ
σ
68%, 95%, 99.7% intervals
Binomial(n, p) ≈ N (np, np(1 − p))
Continuity Correction
16
Adjusts discrete-to-continuous approximation
Practice Problems
1. Suppose
X ∼ N (40, 16)
Find:
9
Normal Distribution
Sheldon Ross – First Course on Probability
(a) P (X < 44)
(b) P (35 < X < 45)
2. The lifetimes of bulbs are normally distributed with mean 1000 hours and standard
deviation 120 hours.
Find:
(a) Probability lifetime exceeds 1100 hours
(b) Probability lifetime lies between 900 and 1200 hours
3. Suppose
X ∼ Binomial(80, 0.4)
Use normal approximation to find:
(a) P (X ≤ 35)
(b) P (X > 40)
4. A fair coin is tossed 200 times.
Use the normal approximation to find the probability that the number of heads lies
between 90 and 110.
10
Lecture Notes on Joint Probability
Distributions
1
Introduction
In many real-life situations, we study two or more random variables simultaneously.
Examples include:
• Height and weight of individuals
• Demand and price of a product
• Rainfall and crop yield
• Income and expenditure
• Number of customers and waiting time
To study relationships between variables, we use:
• Joint distributions
• Marginal distributions
• Conditional distributions
• Covariance
2
Joint Probability Distribution (Discrete Case)
2.1
Definition
Suppose X and Y are discrete random variables.
The joint probability mass function (joint PMF) is
p(x, y) = P (X = x, Y = y)
It gives the probability that:
• X=x
• and Y = y
occur simultaneously.
1
Joint Probability Distributions
2.2
Sheldon Ross – First Course on Probability
Properties
1.
p(x, y) ≥ 0
2.
XX
x
3
p(x, y) = 1
y
Example of Joint PMF
Suppose two fair coins are tossed.
Define:
• X = number of heads
• Y = number of tails
Possible outcomes:
S = {HH, HT, T H, T T }
3.1
Joint PMF Table
X
Y
0
1
2
4
0
1
2
0
0
0
1
4
1
4
0
1
2
0
0
Graphical Intuition of Joint PMF
P (X, Y )
The joint PMF can be visualized as a 3D probability surface.
0.4
0
2
0.5
1
X
1.5
1
1.5
2 0
2
0.5
Y
Joint Probability Distributions
Sheldon Ross – First Course on Probability
5
Joint Cumulative Distribution Function
5.1
Definition
The joint cumulative distribution function (joint CDF) is
F (x, y) = P (X ≤ x, Y ≤ y)
It gives the probability that:
• X is less than or equal to x
• and Y is less than or equal to y
6
Marginal Distributions
6.1
Definition
The marginal distribution of one variable is obtained by summing over the other variable.
6.2
Marginal PMF of X
pX (x) =
X
p(x, y)
y
6.3
Marginal PMF of Y
pY (y) =
X
p(x, y)
x
7
Example of Marginal Distribution
Using the previous table:
X
PX (x)
0
1
2
1
4
1
2
1
4
Y
PY (y)
0
1
2
1
4
1
2
1
4
Similarly,
3
Joint Probability Distributions
Sheldon Ross – First Course on Probability
8
Expectation of Joint Random Variables
8.1
Expectation of X
E(X) =
XX
x
8.2
Expectation of Y
E(Y ) =
XX
x
9
x p(x, y)
y
y p(x, y)
y
Solved Problem: Expectation
Using the previous joint PMF:
Find E(X).
Solution
E(X) =
XX
x
x p(x, y)
y
1
1
1
+1
+2
=0
4
2
4
=0+
1 1
+
2 2
=1
Hence,
E(X) = 1
10
Variance
The variance of X is
V ar(X) = E(X 2 ) − [E(X)]2
where
E(X 2 ) =
XX
x
11
x2 p(x, y)
y
Solved Problem: Variance
Using the previous example:
Find V ar(X).
4
Joint Probability Distributions
Sheldon Ross – First Course on Probability
Solution
First compute:
E(X) = 1
Now,
1
1
1
2
2
+1
+2
E(X ) = 0
4
2
4
2
2
=0+
1
+1
2
= 1.5
Thus,
V ar(X) = 1.5 − (1)2
= 0.5
Hence,
V ar(X) = 0.5
12
Covariance
12.1
Definition
Covariance measures the relationship between two random variables.
Cov(X, Y ) = E(XY ) − E(X)E(Y )
12.2
Interpretation
• Positive covariance:
Variables increase together
• Negative covariance:
One variable increases while the other decreases
• Zero covariance:
No linear relationship
13
Solved Problem: Covariance
Using the earlier example:
Find Cov(X, Y ).
5
Joint Probability Distributions
Sheldon Ross – First Course on Probability
Solution
We know:
E(X) = 1
and
E(Y ) = 1
Now compute:
E(XY ) =
XX
x
xy p(x, y)
y
From the table:
XY = 0
for all possible outcomes.
Thus,
E(XY ) = 0
Therefore,
Cov(X, Y ) = 0 − (1)(1)
= −1
Hence,
Cov(X, Y ) = −1
14
Joint Probability Density Function (Continuous
Case)
14.1
Definition
Suppose X and Y are continuous random variables.
The joint probability density function (joint PDF) is denoted by
f (x, y)
such that
ZZ
P ((X, Y ) ∈ A) =
f (x, y) dx dy
A
6
Joint Probability Distributions
14.2
Sheldon Ross – First Course on Probability
Properties
1.
f (x, y) ≥ 0
2.
Z ∞Z ∞
f (x, y) dx dy = 1
−∞
15
−∞
Graphical Intuition of Joint PDF
Suppose
f (x, y) = x + y
on the unit square.
The joint PDF can be visualized as a surface.
f (x, y)
2
1
0
0
0.2
0.4
0.6
x
16
0.8
1 0
0.2
0.4
0.6
y
Example of Joint PDF
Suppose
(
2, 0 < x < 1, 0 < y < x
f (x, y) =
0, otherwise
16.1
Region of Support
The density exists in the triangular region:
0<y<x<1
7
0.8
1
Joint Probability Distributions
17
Sheldon Ross – First Course on Probability
Verification of PDF
We check:
Z 1Z x
2 dy dx
0
0
Z 1
2x dx
=
0
1
= x2 0
=1
Hence this is a valid joint PDF.
18
Joint CDF in Continuous Case
The joint CDF is
F (x, y) = P (X ≤ x, Y ≤ y)
For continuous random variables,
Z x Z y
F (x, y) =
f (u, v) dv du
−∞
−∞
19
Marginal Density Functions
19.1
Marginal Density of X
Z ∞
fX (x) =
f (x, y) dy
−∞
19.2
Marginal Density of Y
Z ∞
fY (y) =
f (x, y) dx
−∞
20
Solved Problem: Marginal Density
Suppose
(
2, 0 < y < x < 1
f (x, y) =
0, otherwise
Find the marginal density of X.
8
Joint Probability Distributions
Sheldon Ross – First Course on Probability
Solution
We integrate over y:
Z x
fX (x) =
2 dy
0
= 2x
for
0<x<1
Hence,
(
2x, 0 < x < 1
fX (x) =
0,
otherwise
21
Expectation in Continuous Case
21.1
Expectation of X
Z ∞Z ∞
E(X) =
xf (x, y) dx dy
−∞
21.2
−∞
Expectation of Y
Z ∞Z ∞
yf (x, y) dx dy
E(Y ) =
−∞
22
−∞
Solved Problem: Expectation
Suppose
(
2, 0 < y < x < 1
f (x, y) =
0, otherwise
Find E(X).
Solution
Z 1Z x
E(X) =
2x dy dx
0
0
Z 1
=
2x(x) dx
0
Z 1
=2
x2 dx
0
3 1
x
=2
3 0
9
Joint Probability Distributions
Sheldon Ross – First Course on Probability
=
2
3
Hence,
E(X) =
23
2
3
Covariance in Continuous Case
Cov(X, Y ) = E(XY ) − E(X)E(Y )
where
Z Z
E(XY ) =
xyf (x, y) dx dy
24
Independence of Random Variables
24.1
Discrete Case
X and Y are independent if
p(x, y) = pX (x)pY (y)
24.2
Continuous Case
X and Y are independent if
f (x, y) = fX (x)fY (y)
25
Applications
Joint distributions are widely used in:
• Finance
• Machine learning
• Risk management
• Queueing theory
• Reliability engineering
• Economics
• Weather forecasting
10
Joint Probability Distributions
26
Sheldon Ross – First Course on Probability
Summary Table
Concept
Definition
Joint PMF
P (X = x, Y = y)
Joint PDF
f (x, y)
Joint CDF
P (X ≤ x, Y ≤ y)
Marginal Distribution
Expectation
Variance
Covariance
Independence
27
Distribution of one variable alone
Average value
Spread around the mean
Measures joint linear relationship
Product form of joint distribution
Practice Problems
1. Two fair dice are rolled.
Let:
• X = first die
• Y = second die
Find:
(a) Joint PMF
(b) Marginal distributions
(c) Cov(X, Y )
2. Suppose
(
6xy, 0 < x < 1, 0 < y < 1
f (x, y) =
0,
otherwise
Verify whether this is a valid joint PDF.
3. Suppose
(
8xy, 0 < x < 1, 0 < y < x
f (x, y) =
0,
otherwise
Find:
11
Joint Probability Distributions
Sheldon Ross – First Course on Probability
(a) Marginal densities
(b) E(X)
(c) E(Y )
4. Show whether the following variables are independent:
f (x, y) = e−(x+y) ,
12
x > 0, y > 0
Lecture Notes on Conditional Distribution
and Conditional Expectation
1
Introduction
In many real-world problems, we revise probabilities and expectations when additional
information becomes available.
Examples include:
• Expected sales given advertising expenditure
• Expected stock return given market condition
• Probability of disease given test result
• Waiting time given number of customers already present
These ideas lead to:
• Conditional distributions
• Conditional expectation
• Law of total expectation
• Variance and covariance properties
2
Conditional Distribution (Discrete Case)
2.1
Definition
Suppose X and Y are discrete random variables.
The conditional probability mass function of X given Y = y is
P (X = x|Y = y) =
P (X = x, Y = y)
P (Y = y)
provided
P (Y = y) > 0.
2.2
Interpretation
Conditional distribution tells us:
How probabilities change once we know the value of another variable.
1
Conditional Distribution and Expectation
3
Sheldon Ross – First Course on Probability
Example of Conditional Distribution
Suppose the joint PMF is:
Y =0
Y =1
X=0
X=1
0.2
0.1
0.3
0.4
Find:
P (X = 1|Y = 1)
Solution
First compute:
P (Y = 1) = 0.1 + 0.4 = 0.5
Then,
P (X = 1|Y = 1) =
=
P (X = 1, Y = 1)
P (Y = 1)
0.4
0.5
= 0.8
Hence,
P (X = 1|Y = 1) = 0.8
4
Conditional Density Function (Continuous Case)
4.1
Definition
Suppose X and Y are continuous random variables.
The conditional density of X given Y = y is
fX|Y (x|y) =
f (x, y)
fY (y)
provided
fY (y) > 0.
4.2
Interpretation
Conditional density describes the distribution of X after fixing Y = y.
2
Conditional Distribution and Expectation
5
Sheldon Ross – First Course on Probability
Example of Conditional Density
Suppose
(
2, 0 < y < x < 1
f (x, y) =
0, otherwise
Find:
fY |X (y|x)
Solution
First compute the marginal density of X:
Z x
fX (x) =
2dy = 2x
0
for
0 < x < 1.
Therefore,
fY |X (y|x) =
=
f (x, y)
fX (x)
1
2
=
2x
x
for
0 < y < x < 1.
Hence,
fY |X (y|x) =
6
Conditional Expectation
6.1
Definition (Discrete Case)
1
x
The conditional expectation of X given Y = y is
X
E(X|Y = y) =
xP (X = x|Y = y)
x
6.2
Definition (Continuous Case)
For continuous random variables,
Z ∞
xfX|Y (x|y) dx
E(X|Y = y) =
−∞
3
Conditional Distribution and Expectation
7
Sheldon Ross – First Course on Probability
Intuition Behind Conditional Expectation
Conditional expectation is:
The expected value of X after learning information about Y .
It represents an updated average.
7.1
Practical Interpretation
Examples:
• Expected sales given advertising budget
• Expected rainfall given cloud conditions
• Expected return given economic indicators
8
Example of Conditional Expectation
Suppose:
Y =0
Y =1
X=1
X=3
0.2
0.1
0.3
0.4
Find:
E(X|Y = 1)
Solution
First compute:
P (Y = 1) = 0.1 + 0.4 = 0.5
Then,
P (X = 1|Y = 1) =
0.1
= 0.2
0.5
P (X = 3|Y = 1) =
0.4
= 0.8
0.5
Therefore,
E(X|Y = 1) = 1(0.2) + 3(0.8)
= 0.2 + 2.4
4
Conditional Distribution and Expectation
Sheldon Ross – First Course on Probability
= 2.6
Hence,
E(X|Y = 1) = 2.6
9
Properties of Conditional Expectation
9.1
Linearity
E(aX + b|Y ) = aE(X|Y ) + b
9.2
If X and Y are Independent
E(X|Y ) = E(X)
because knowing Y provides no additional information about X.
9.3
Taking Out What is Known
If g(Y ) depends only on Y ,
E[g(Y )X|Y ] = g(Y )E(X|Y )
10
Law of Total Expectation
10.1
Statement
E(X) = E[E(X|Y )]
This is also called the:
• Tower property
• Iterated expectation property
11
Intuition Behind E(X) = E(E(X|Y ))
The formula means:
First compute the expected value of X within each group defined by Y , then
average these expectations across groups.
5
Conditional Distribution and Expectation
11.1
Sheldon Ross – First Course on Probability
Practical Example
Suppose:
• Y = department
• X = salary
Then:
• E(X|Y ) = average salary within a department
• E(E(X|Y )) = overall average salary across all departments
12
Proof of Law of Total Expectation
For discrete random variables,
E(E(X|Y )) =
X
E(X|Y = y)P (Y = y)
y
Now,
E(X|Y = y) =
X
xP (X = x|Y = y)
x
Substituting:
!
=
X X
y
xP (X = x|Y = y) P (Y = y)
x
Using
P (X = x|Y = y)P (Y = y) = P (X = x, Y = y)
we get
=
XX
y
xP (X = x, Y = y)
x
!
=
X
X
x
x
P (X = x, Y = y)
y
=
X
xP (X = x)
x
= E(X)
Hence,
E(X) = E(E(X|Y ))
6
Conditional Distribution and Expectation
Sheldon Ross – First Course on Probability
13
Variance of Linear Combination
13.1
Statement
For random variables X and Y ,
V ar(aX + bY ) = a2 V ar(X) + b2 V ar(Y ) + 2abCov(X, Y )
14
Proof of Variance Formula
Let
Z = aX + bY
Then,
V ar(Z) = E[(Z − E(Z))2 ]
Since
E(Z) = aE(X) + bE(Y )
we get
Z − E(Z) = a(X − E(X)) + b(Y − E(Y ))
Thus,
V ar(Z) = E [a(X − E(X)) + b(Y − E(Y ))]2
Expanding,
= E[a2 (X − E(X))2 + b2 (Y − E(Y ))2
+2ab(X − E(X))(Y − E(Y ))]
Using linearity of expectation,
= a2 E[(X − E(X))2 ]
+b2 E[(Y − E(Y ))2 ]
+2abE[(X − E(X))(Y − E(Y ))]
Recognizing variance and covariance,
= a2 V ar(X) + b2 V ar(Y ) + 2abCov(X, Y )
Hence,
V ar(aX + bY ) = a2 V ar(X) + b2 V ar(Y ) + 2abCov(X, Y )
7
Conditional Distribution and Expectation
15
Sheldon Ross – First Course on Probability
Special Case: Independent Variables
If X and Y are independent,
Cov(X, Y ) = 0
Thus,
V ar(aX + bY ) = a2 V ar(X) + b2 V ar(Y )
16
Practical Relevance of Variance Formula
16.1
Finance
Portfolio risk depends on:
• Variance of each asset
• Covariance between assets
Diversification reduces risk when covariance is low or negative.
16.2
Operations Management
Total production uncertainty depends on:
• Variability of individual production units
• Dependence between units
16.3
Machine Learning
Prediction uncertainty depends on:
• Variability of models
• Correlation between prediction errors
17
Solved Problem on Variance Formula
Suppose:
V ar(X) = 4
V ar(Y ) = 9
Cov(X, Y ) = 2
Find:
V ar(3X + 2Y )
8
Conditional Distribution and Expectation
Sheldon Ross – First Course on Probability
Solution
Using:
V ar(aX + bY ) = a2 V ar(X) + b2 V ar(Y ) + 2abCov(X, Y )
Substitute values:
= 32 (4) + 22 (9) + 2(3)(2)(2)
= 36 + 36 + 24
= 96
Hence,
V ar(3X + 2Y ) = 96
18
Conditional Variance
Conditional variance measures variability after conditioning on another variable.
V ar(X|Y ) = E[(X − E(X|Y ))2 |Y ]
19
Law of Total Variance
V ar(X) = E[V ar(X|Y )] + V ar(E(X|Y ))
Interpretation:
• First term:
Average variability within groups
• Second term:
Variability between group means
20
Applications
Conditional expectation and covariance are widely used in:
• Finance
• Risk management
• Insurance
• Machine learning
• Queueing theory
• Econometrics
• Signal processing
9
Conditional Distribution and Expectation
21
Sheldon Ross – First Course on Probability
Summary Table
Concept
Formula
Conditional PMF
P (X = x|Y = y) =
P (X = x, Y = y)
P (Y = y)
Conditional Density
fX|Y (x|y) =
f (x, y)
fY (y)
Conditional Expectation
E(X|Y )
Law of Total Expectation
E(X) = E(E(X|Y ))
Covariance
Cov(X, Y ) = E(XY ) − E(X)E(Y )
Variance Formula
V ar(aX+bY ) = a2 V ar(X)+b2 V ar(Y )+2abCov(X, Y )
Law of Total Variance
V ar(X) = E[V ar(X|Y )] + V ar(E(X|Y ))
22
Practice Problems
1. Suppose the joint PMF is:
X=1
X=2
0.3
0.1
0.2
0.4
Y =0
Y =1
Find:
(a) P (X = 2|Y = 1)
(b) E(X|Y = 1)
10
Conditional Distribution and Expectation
Sheldon Ross – First Course on Probability
2. Suppose:
V ar(X) = 5,
V ar(Y ) = 7,
Cov(X, Y ) = 3
Find:
V ar(2X − 3Y )
3. Show that if X and Y are independent, then
E(X|Y ) = E(X)
4. Prove the law of total expectation for continuous random variables.
5. A portfolio contains two assets:
• Asset A weight = 0.6
• Asset B weight = 0.4
Suppose:
V ar(A) = 0.04,
V ar(B) = 0.09,
Find the variance of the portfolio return.
11
Cov(A, B) = 0.01
0
You can add this document to your study collection(s)
Sign in Available only to authorized usersYou can add this document to your saved list
Sign in Available only to authorized users(For complaints, use another form )