Chapter Two
Random Variable
Course: Probability and Statistics
The notion of Random Variables
• A random variable (also called random quantity, or stochastic variable) is a
mathematical formalization of a quantity or object which depends
on random events.
• Essentially, a random variable is a function that assigns numerical values to the
outcomes of a random experiment.
• This allows us to work with probabilities in a mathematical way.
• Instead of dealing with vague outcomes like "heads" or "tails," we can assign
them numbers (e.g., 1 for heads, 0 for tails).
2.2 The Three Types of Random Variables
Random variables can be classified into three main types, each serving different
purposes in statistical analysis and probability theory.
1 Discrete random variable
• A discrete random variable takes on a finite or countably infinite number of
distinct values. These are typically associated with counting processes.
• Discrete random variables are often described using a probability mass function
(PMF), which assigns probabilities to each possible value. The sum of all
probabilities equals 1.
• Examples: The number of students in a classroom 20, 21, 22, etc., the outcome of
rolling a die (1, 2, 3, 4, 5, or 6).
• Common Distributions: Binomial, Poisson, Geometric.
2 Continuous Random Variables
• A continuous random variable can take any value within a continuous range or
interval (e.g., all real numbers between 0 and 1, or all positive real numbers).
• There are no "gaps", which would correspond to numbers which have a finite
probability of occurring.
• Continuous random variables are described using a probability density function
(PDF), which indicates the likelihood of the variable falling within a particular
interval
• Examples: The height of a person (e.g., 5.7 feet, 5.72 feet, etc.), the time it takes
for a bus to arrive (e.g., 3.2 minutes, 3.25 minutes, etc.).
3. Mixed Random Variables
• A mixed random variable has both discrete and continuous components.
• It combines features of the two types above, with some probability concentrated at
specific points (discrete) and some spread over intervals (continuous).
• The cumulative distribution function (CDF) is used to describe it fully, as it may
have jumps (discrete) and smooth increases (continuous).
• Example: The time until a machine fails, where there’s a chance it fails
immediately (discrete at 0) or after some continuous time (e.g., between 1 and 10
hours).
2.3 The cumulative distribution function(cdf)
• The cumulative distribution function, or briefly the distribution function, for a
random variable X is defined by:
• F(x)=P(X≤ x)
Where x is any real number, i.e.,-∞<x<∞ .
• The cumulative distribution function F(x) has the following properties:
Example 1
• Find the cdf of the random variable X which is defined as the number of heads in three tosses of
a fair coin.
• Solution: X takes on only the values 0, 1, 2 and 3 with probabilities 1/8, 3/8, 3/8 and 1/8
respectively. Thus, Fx (x) is simply the sum of the probabilities of the outcomes from the set {0,
1, 2, 3} that are less than or equal to x.
Example 2: Suppose X is the outcome of rolling a fair six sided die. The possible values of X are
1, 2, 3, 4, 5, 6, each with the probability of 1⁄6.
• Fx (x) gives the probability that X≤x. For a discrete random variable like this the CDF is a step
function that increases at each possible value of X.
Cont..
• So the CDF can be written as:
2.4 The probability density function
• The probability density function (pdf), denoted f, of a continuous random variable X satisfies
the following:
• Example 3: Suppose a bus arrives at a stop at a random time between 10:00 AM and 10:10 AM,
and all times in that interval are equally likely.
• What’s the probability the bus arrives between 10:02 and 10:05? That’s from x=2 to x=5.
2.5 Conditional CDF's and PDF's
• Conditional cumulative distribution functions (CDFs) and probability density functions (PDFs)
are essential concepts in probability theory and statistics, particularly when dealing with the
relationships between random variables.
Conditional CDF’s:
• The conditional cumulative distribution function of a random variable X given another random
variable Y is defined as:
• This function provides the probability that the random variable X takes on the value less than or
equal to x given that Y has taken on the specific value y.
• It is calculated by integrating the conditional PDF of X given Y:
Conditional PDF’s
• The conditional probability density function of a random variable X given another
random variable Y is defined as:
ππ,π (π₯, π¦)
ππ|π π₯|π¦ =
ππ¦ (π¦)
• Where ππ,π (π₯, π¦) is the joint PDF of X and Y, and ππ¦ (π¦) is the marginal PDF of Y.
This formula allows us to update our understanding of the distribution of X based
on the information provided by Y.
Some importance of Random Variable
1.Bernoulli Random Variable:
• This is a discrete random variable that takes on two possible outcomes, typically labeled as 1
(success) and 0 (failure). The cdf πΉπ (x) of the Bernoulli r.v. X is given by:
2.Binomial Random Variable:
• This variable represents the number of successes in a fixed number of independent Bernoulli
trials.
• For example, if you flip a coin 10 times, the binomial random variable would count how many
times you get heads.
• It is characterized by two parameters: the number of trials (n) and the probability of success (k)
in each trial
• A r.v. X is called a binomial r.v. with parameters (n, k) if its pmf is given by:
3.Poisson Random Variable:
• This discrete random variable is used to model the number of events occurring within a fixed
interval of time or space, given that these events happen with a known constant mean rate and
independently of the time since the last event.
Some examples of Poisson r.v.'s include
1. The number of telephone calls arriving at a switching center during various intervals of time
2. The number of customers entering a bank during various intervals of time
A r.v. X is called a Poisson r.v. with parameter λ(> 0) if its pmf is given by:
Example
The number of traffic accidents per week in a small city has a Poisson distribution with mean
equal to 3. What is the probability of exactly 2 accidents occur in 2 weeks?
Solution: The mean traffic accident is 3. Thus, the mean accidents in two weeks are λ= (3) (2) =6.
cont.
4 Normal (Gaussian) Random Variable:
• This is a continuous random variable that follows a normal distribution.
• It is defined by two parameters: the mean (μ) and the standard deviation (σ). Many natural
phenomena, such as heights or test scores, can be modeled using a normal random variable.
5.Exponential Random Variable:
• This continuous random variable is often used to model the time until an event occurs, such as the time
until a radioactive particle decays or the time until the next customer arrives at a service point.
• It is characterized by a rate parameter (λ). Mean is 1/λ. A r.v. X is called an exponential r.v. with
parameter λ (>0) if its pdf is given by
Example
• Assume that the length of phone calls made at a particular telephone booth is exponentially
distributed with a mean of 3 minutes. If you arrive at the telephone booth just as Chris was
about to make a call, find the following
1. The probability that you will wait more than 5 minutes before Chris is done with the call.
2. The probability that Chris’ call will last between 2 minutes and 6 minutes.
Solution:
• Let X be a random variable that denotes the length of calls made at the telephone booth. Since
the mean length of calls is 1/λ = 3, we have that the PDF of X is given by
1 The probability that you will wait more than 5 minutes is the probability that X is greater than 5
minutes, which is given by
2. The probability that the call lasts between 2 and 6 minutes is given by
6 Uniform Random Variable:
• This variable can be either discrete or continuous and is characterized by all
outcomes being equally likely.
• For example, rolling a fair die results in a uniform random variable where each
face has an equal probability of landing face up.
• A r.v. X is called a uniform r.v. over (a, b) if its pdf is given by
2.7 Functions of a random variable
• Functions of random variables are an important concept in probability and statistics, allowing us to
derive new random variables from existing ones through mathematical functions.
• Here’s an overview of how this works:
• Let X be a random variable, and let Y be a new random variable that is a function of X. That is, Y =
g(X). We are interested in computing the PDF or PMF of Y when the PDF or PMF of X is given.
• For example, let g(X) = X + 5. Then
• FY(y) = P[Y ≤ y] = P[X + 5 ≤ y]
• Finding the PMF of a Function of a Random Variable
• To find the PMF of Y, we can use the relationship between Y and X:
• Identify the range of Y: π
π ={g(x)|x ∈π
π (π)}
• Calculate the PMF of Y using:
• ππ (π) = π(π = π¦) = π(π(π₯) = π¦) =
π₯:π(π₯)=π¦ ππ (π₯)
• This approach allows us to derive the probabilities associated with the new random variable Y based on
the probabilities of X.
Example 8
Let X be discrete random variable withPX (k)=1/5 for k=-1, 0,1,2,3. Let Y=2|X|. Find the range and PMF of Y.
2.8 The expected value of random variables
• Remember that the expected value of a discrete random variable can be obtained
as
• Now, by replacing the sum by an integral and PMF by PDF, we can write the
definition of expected value of a continuous random variable as:
conti..
cont.
2.8 Variance
The variance of r.v X denoted by ππ₯2 =πππ(π) = πΈ(π 2 ) − (πΈ(π))2
Example 12:
• Let X be a continuous random variable with PDF
• Find expected value and variance of X?
• Sol
2.9 The Markov and Chebyshev inequalities
Chebyshev Inequality
• Let X be a random variable with pdf f(x). If μ and σ > 0 are the mean and standard deviation of
X, then:
• For any nonzero real positive number k.
• Example: Suppose a random variable X represents daily sales (in dollars) with πΈ[X]=1000
and π 2 =200
Example
Markov Inequality
• The Markov inequality applies to random variables that take only non-negative values. If X is a
random variable that takes only non-negative values, then for any value a > 0,
• This inequality can be proved as follows.
•
Example
2.10 Testing the fit of a distribution to data
• This involves both graphical and statistical methods to evaluate how well a theoretical
distribution matches observed data
Methods
1.Graphical Methods:
• QQ Plots: Compare the quantiles of the sample data against the quantiles of a theoretical
distribution.
• Histograms: Compare the empirical distribution to the theoretical distribution visually.
Statistical methods
a. Chi-Squared Goodness-of-Fit Test
ο·
Compares observed frequencies with expected frequencies under a hypothesized
distribution.
ο·
Suitable for categorical or binned continuous data.
Test Statistic:π₯ 2 =
(ππ− πΈπ )2
c. Anderson–Darling Test
πΈπ
Where:
ο·
ππ : Observed frequency
ο·
πΈπ : Expected frequency
ο·
A more sensitive version of the K–S test.
ο·
Weighs the tails of the distribution more heavily.
b. Kolmogorov–Smirnov (K–S) Test
ο·
Non-parametric test for continuous distributions.
ο·
Compares empirical CDF of the data with the CDF of the hypothesized distribution.
Statistic:
π·π = supβ‘
|πΉπ π₯ − πΉ(π₯)|
Where:
ο·
Fn x is the empirical CDF
ο·
F x is the hypothesized CDF
2.11 Transform methods
• Transform methods are techniques used to manipulate random variables and their
distributions.
• It facilitate the study of relationships between distributions and their properties.
They include:
• Transformation of Variables: This involves applying a function to a random variable to
obtain a new random variable with a desired distribution.
• Rejection Sampling: A method for generating observations from a distribution by using a
proposal distribution.
• These methods are crucial for deriving new distributions from known ones and for
simplifying complex problems.
2.12 The characteristic function
The characteristic function of a random variable is a powerful tool in probability theory. Let
ππ (π₯) be the PDF of the continuous random variable X. The characteristic function of X is
defined by:
If X is a discrete random variable with PMF ππ π₯ , the characteristic function is given by
∞
ππ (π₯)π ππ€π₯
Φπ π€ =
π₯ =−∞
Note that Φπ 0 = 1, which is a test of whether a given function of w is a true characteristic
function of a random variable.
cont.
Example 15
Find the characteristic function of the exponential random variable X with the PDFππ π₯ =
ππ −ππ₯ π₯ ≥ 0.
Solution:
The characteristic function uniquely determines the distribution of a random variable and is
useful for:
•Analyzing sums of independent random variables.
•Establishing convergence in distribution.
Cont.
The probability generating function (PGF) is particularly useful for discrete random variables.
It is defined as:
∞
πΊπ π = πΈ π π =
π(π = π)π π
π =0
Where s is a non-negative real number. The PGF provides insights into the moments of the
distribution and can be used to derive the distribution of sums of independent random variables.
2.14 The Laplace Transform of the PDF
The Laplace transform is a powerful mathematical tool used to transform a function of time (or
another independent variable) into a function of a complex variable, typically denoted s. It’s
widely used in engineering, physics, and mathematics to solve differential equations, analyze
systems, and work with probability distributions. For a function f(t) defined for t≥0, the Laplace
transform is given by:
Cont.
Where s is a complex number with a real part Re(s) sufficiently large to ensure the integral
converges. The Laplace transform of the PDF is closely related to the moment-generating
function or the characteristic function of the random variable, but with a specific form:
Where ππ (π‘) is the pdf of the random variable X. The Laplace transform is useful for:
ο·
Solving differential equations related to stochastic processes.
ο·
Analyzing the behavior of random variables in the time domain.
Cont..