Probability and Statistics
Dr. Ishapathik Das, IIT Tirupati
To introduce the fundamentals of probability theory and the basic
techniques of statistics. To demonstrate methods to solve applied problems
of probability and applications of statistics.
Textbook(s): 1. Bertseka D and Tsitsiklis J, Introduction to Probability,
Athena Scientific (2008).
Reference(s): 1. Chung K L, Elementary Probability Theory with Stochastic
Process, Springer Verlag (1974).
2. Drake A, Fundamentals of Applied Probability Theory, McGraw-Hill (1967).
3. Kreyszig E, Advanced Engineering Mathematics, John Wiley & Sons (2010).
4. Ross S, A First course in Probability, Prentice Hall of India (2009).
Probability and Statistics
Dr. Ishapathik Das, IIT Tirupati
Probability: Probability models and axioms, conditioning and Bayes' rule,
independence discrete random variables; probability mass functions;
expectations, examples, multiple discrete random variables: joint PMFs,
expectations, conditioning, independence, continuous random variables,
probability density functions, expectations, examples, multiple continuous
random variables, transformation of random variables, covariance and
correlation, iterated expectations, convolution; notion of convergence, weak
law of large numbers, central limit theorem. Statistics: Concepts of Statistical
Inference, Point Estimation, Methods of Estimation, Confidence Intervals,
Testing of Hypotheses, Bayesian Statistical Inference.
Probability and Statistics
Dr. Ishapathik Das, IIT Tirupati
Probability and Statistics
Dr. Ishapathik Das, IIT Tirupati
❑ A phenomenon is a fact, occurrence, or
circumstance observed or observable.
❑ Example: Natural Phenomena such as
weather,
fog,
thunder,
tornadoes,
biological processes, decomposition, etc.
❑ In scientific usage, a phenomenon is any
event that is observable, including the use
of instrumentation to observe, record, or
compile data.
Probability and Statistics
Dr. Ishapathik Das, IIT Tirupati
Probability and Statistics
Dr. Ishapathik Das, IIT Tirupati
❑ There exists a mathematical model that allows “perfect”
prediction the phenomena’s outcome.
❑ Many examples exist in Physics, Chemistry (the exact
sciences).
❑
Predicting the amount of money in a bank account.
➢
If you know the initial deposit, and the interest rate, then:
➢
You can determine the amount in the account after one
year
Probability and Statistics
Dr. Ishapathik Das, IIT Tirupati
❑ No mathematical model exists that allows “perfect” prediction the
phenomena’s outcome.
❑ may be divided into two groups.
1. Random phenomena :
– Unable to predict the outcomes, but in the long-run, the outcomes
exhibit statistical regularity.
2. Haphazard phenomena
– unpredictable outcomes, but no long-run, exhibition of statistical
regularity in the outcomes.
Probability and Statistics
Dr. Ishapathik Das, IIT Tirupati
Probability and Statistics
Dr. Ishapathik Das, IIT Tirupati
▪ Unable to predict the outcomes, but in the long-run, the outcomes
exhibit statistical regularity.
▪ Example:
➢ Unable to predict outcome but in the long run can one can determine that each
outcome will occur 1/6 of the time.
➢ Use symmetry. Each side is the same. One side should not occur more
frequently than another side in the long run. If the die is not balanced this may not
be true.
Probability and Statistics
Dr. Ishapathik Das, IIT Tirupati
❑ Unpredictable outcomes, but no long-run, exhibition of statistical
regularity in the outcomes.
❑ Example:
➢ we don’t have die, instead someone is choosing numbers from 1 to 6.
➢ it is impossible to know any number someone might choose .
➢ it is not possible to know the probability of observing any value of 1 to 6.
➢ we can not know whether someone has a favorite number to choose more
likely than others.
➢ we don’t have any idea the process by which the person is choosing the
numbers.
Probability and Statistics
Dr. Ishapathik Das, IIT Tirupati
Probability and Statistics
Dr. Ishapathik Das, IIT Tirupati
▪ The set of all possible outcomes of a random phenomena is
called the sample space S.
▪ Examples:
Probability and Statistics
Dr. Ishapathik Das, IIT Tirupati
Any subset of the sample space S is called the Event.
Probability and Statistics
Dr. Ishapathik Das, IIT Tirupati
Probability and Statistics
Dr. Ishapathik Das, IIT Tirupati
Probability and Statistics
Dr. Ishapathik Das, IIT Tirupati
Probability and Statistics
Dr. Ishapathik Das, IIT Tirupati
Probability and Statistics
Dr. Ishapathik Das, IIT Tirupati
Probability and Statistics
Dr. Ishapathik Das, IIT Tirupati
Probability and Statistics
Dr. Ishapathik Das, IIT Tirupati
Probability and Statistics
Dr. Ishapathik Das, IIT Tirupati
Probability and Statistics
Dr. Ishapathik Das, IIT Tirupati
Probability and Statistics
Dr. Ishapathik Das, IIT Tirupati
➢ Both the classical and frequency approaches have serous
drawbacks.
➢ The words “equally likely” are vague.
➢ The “large number” involved is vague.
➢ Mathematicians have been led to an axiomatic approach to
probability.
Probability and Statistics
Dr. Ishapathik Das, IIT Tirupati
➢ Suppose we have a sample space S.
➢ If S is discrete, all subsets corresponds to events and conversely.
➢ If S is non discrete, only special subsets (called measurable)
correspond to events.
➢ To each event A in the class C of events, we associate a real
number P(A).
➢ The P is called a probability function, and
➢ P(A) the probability of the event, if the following axioms are
satisfied.
Probability and Statistics
Dr. Ishapathik Das, IIT Tirupati
Probability and Statistics
Dr. Ishapathik Das, IIT Tirupati
Probability and Statistics
Dr. Ishapathik Das, IIT Tirupati
Probability and Statistics
Dr. Ishapathik Das, IIT Tirupati
Chennai and Mumbai are two of the cities
competing in IPL (there are also many others).
The organizers are narrowing the competition to
the final 5 cities. There is a 20% chance that
Chennai will be amongst the final 5. There is a
35% chance that Mumbai will be amongst the
final 5, and an 8% chance that both Chennai
and Mumbai will be amongst the final 5. What is
the probability that Chennai or Mumbai will be
amongst the final 5.
Statistical Inference
Dr. Ishapathik Das, IIT Tirupati
Probability and Statistics
Dr. Ishapathik Das, IIT Tirupati
Probability and Statistics
Dr. Ishapathik Das, IIT Tirupati
Probability and Statistics
Dr. Ishapathik Das, IIT Tirupati
➢ Suppose
before observing the outcome of a random experiment
you are given information regarding the outcome
➢ How should this information be used in prediction of the outcome.
➢ Namely, how should probabilities be adjusted to take into account
this information
➢ Usually the information is given in the following form: You are told
that the outcome belongs to a given event. (i.e. you are told that an
event has occurred)
Probability and Statistics
Dr. Ishapathik Das, IIT Tirupati
• Three prisoners , A, B, C are in jail. One of
them is to be executed and the other two will
be set free. Prisoner A asked the guard : one of
my partners B or C will be set free. Could you
please tell me which one of them will be set
free?
• Guard thought a while and told A : If I do not
tell you, then your chance of death is 1/3. But if
I tell you, then there are only two left and you
are one of them to be killed. Your chance of
death will be 1/2. Do you really want to increase
your chance of death ?
Probability and Statistics
Dr. Ishapathik Das, IIT Tirupati
Probability and Statistics
Dr. Ishapathik Das, IIT Tirupati
Probability and Statistics
Dr. Ishapathik Das, IIT Tirupati
Roll a balanced die once and record the
number on the top face.
Let E be the event that a 1 shows on the top
face.
Let F be the event that the number on the top
face is odd.
What is P(E)?
What is the Probability of the event E
if we are told that the number on the top face is
odd,
that is, we know that the event F has occurred?
Probability and Statistics
Dr. Ishapathik Das, IIT Tirupati
➢ Key idea: The original sample space no longer applies.
The new or reduced sample space is S={1, 3, 5}.
➢ Notice that the new sample space consists only of the outcomes in F.
P(E occurs given that F occurs) = 1/3.
Probability and Statistics
Dr. Ishapathik Das, IIT Tirupati
Probability and Statistics
Dr. Ishapathik Das, IIT Tirupati
Probability and Statistics
Dr. Ishapathik Das, IIT Tirupati
Probability and Statistics
Dr. Ishapathik Das, IIT Tirupati
Probability and Statistics
Dr. Ishapathik Das, IIT Tirupati
Probability and Statistics
Dr. Ishapathik Das, IIT Tirupati
Probability and Statistics
Dr. Ishapathik Das, IIT Tirupati
Probability and Statistics
Dr. Ishapathik Das, IIT Tirupati
Bayes’ Theorem
❑ Consider a manufacturing firm that receives shipment of parts
from two suppliers.
❑ Let A1 denote the event that a part is received from supplier 1; A2
is the event the part is received from supplier 2
We get 65 percent of our
parts from supplier 1 and 35
percent from supplier 2.
Thus:
P(A1) = 0.65 and P(A2) = 0.35
Probability and Statistics
Dr. Ishapathik Das, IIT Tirupati
Quality levels
Percentage
Good Parts
Percentage
Bad Parts
Supplier 1
98
2
Supplier 2
95
5
Let G denote that a part is good and B denote
the event that a part is bad. Thus we have the
following conditional probabilities:
P(G | A1 ) = .98 and P(B | A2 ) = .02
P(G | A2 ) = .95 and P(B | A2 ) = .05
Probability and Statistics
Dr. Ishapathik Das, IIT Tirupati
Tree Diagram
Step 1
Supplier
Step 2
Condition
G
A1
Experimental
Outcome
(A1, G)
B
(A1, B)
A2
(A2, G)
G
B
(A2, B)
Probability and Statistics
Dr. Ishapathik Das, IIT Tirupati
Each of the experimental
outcomes is the intersection of 2
events. For example, the
probability of selecting a part
from supplier 1 that is good is
given by:
P( A1  G) = P( A1 ) P(G | A1 )
Probability and Statistics
Dr. Ishapathik Das, IIT Tirupati
Tree Diagram
Step 1
Supplier
Step 2
Condition
Probability of Outcome
P( A1  G) = P( A1 ) P(G | A1 ) = .6370
P(G | A1)
.98
P(A1)
.65
P(A2)
P(B | A2)
P( A1  B) = P( A1 ) P( B | A1 ) = .0130
.02
P(B | A2)
P( A2  G) = P( A2 ) P(G | A2 ) = .3325
.95
.35
P(B | A2)
.05
Probability and Statistics
Dr. Ishapathik Das, IIT Tirupati
P( A2  B) = P( A2 ) P(G | A2 ) = .0175
A bad part broke one
of our machines—so
we’re through for the
day. What is the
probability the part
came from supplier 1?
We know from the law of conditional probability that:
P( A1  B )
P ( A1 | B) =
P( B)
(1)
Observe from the probability tree that:
P( A1  B) = P( A1 ) P( B | A1 )
Probability and Statistics
(2)
Dr. Ishapathik Das, IIT Tirupati
The probability of selecting a bad
part is found by adding together
the probability of selecting a bad
part from supplier 1 and the
probability of selecting bad part
from supplier 2.
That is:
P( B) = P( A1  B) + P( A2  B)
= P( A1 ) P( B | A1 ) + P( A2 ) P( B / A2 )
Probability and Statistics
(3)
Dr. Ishapathik Das, IIT Tirupati
Bayes’ Theorem for 2 Events
By substituting equations (2) and (3) into (1), and
writing a similar result for P(B | A2), we obtain Bayes’
theorem for the 2 event case:
P( A1 ) P( B | A1 )
P( A1 | B) =
P( A1 ) P( B | A1 ) + P( A2 ) P( B | A2 )
P( A2 ) P( B | A2 )
P( A2 | B) =
P( A1 ) P( B | A1 ) + P( A2 ) P( B | A2 )
Probability and Statistics
Dr. Ishapathik Das, IIT Tirupati
Do the Math
P ( A1 ) P ( B | A1 )
P ( A1 | B ) =
P ( A1 ) P ( B | A1 ) + P ( A2 ) P ( B | A2 )
=
(.65)(.02)
.0130
=
= .4262
(.65)(.02) + (.35)(.05) .0305
P ( A2 ) P( B | A2 )
P ( A2 | B ) =
P ( A1 ) P( B | A1 ) + P ( A2 ) P( B | A2 )
(.35)(.05)
.0175
=
=
= .5738
(.65)(.02) + (.35)(.05) .0305
Probability and Statistics
Dr. Ishapathik Das, IIT Tirupati
Bayes’ Theorem
Probability and Statistics
Dr. Ishapathik Das, IIT Tirupati
Example
There are 100 patients in a hospital with a certain
disease. Of these, 10 are selected to undergo a drug
treatment that increases the percentage cured rate from
50 percent to 75 percent. What is the probability that
the patient received a drug treatment if the patient is
known to be cured?
Probability and Statistics
Dr. Ishapathik Das, IIT Tirupati
Probability and Statistics
Dr. Ishapathik Das, IIT Tirupati
Probability and Statistics
Dr. Ishapathik Das, IIT Tirupati
Probability and Statistics
Dr. Ishapathik Das, IIT Tirupati
Probability and Statistics
Dr. Ishapathik Das, IIT Tirupati
Probability and Statistics
Dr. Ishapathik Das, IIT Tirupati
Probability and Statistics
Dr. Ishapathik Das, IIT Tirupati
For the previous example, find the distribution function and the sketch of
the distribution function of X.
Probability and Statistics
Dr. Ishapathik Das, IIT Tirupati
Probability and Statistics
Dr. Ishapathik Das, IIT Tirupati
Probability and Statistics
Dr. Ishapathik Das, IIT Tirupati
Probability and Statistics
Dr. Ishapathik Das, IIT Tirupati
Probability and Statistics
Dr. Ishapathik Das, IIT Tirupati
Probability and Statistics
Dr. Ishapathik Das, IIT Tirupati
Probability and Statistics
Dr. Ishapathik Das, IIT Tirupati
Probability and Statistics
Dr. Ishapathik Das, IIT Tirupati
Probability and Statistics
Dr. Ishapathik Das, IIT Tirupati
Probability and Statistics
Dr. Ishapathik Das, IIT Tirupati
• Many instances of binomial
distributions can be found in real life.
• For example, if a new drug is
introduced to cure a disease, it either
cures the disease (it’s successful) or
it doesn’t cure the disease (it’s a
failure).
• If you purchase a lottery ticket, you’re
either going to win money, or you
aren’t.
• Basically, anything you can think of
that can only be a success or a
failure can be represented by a
binomial distribution.
Probability and Statistics
Dr. Ishapathik Das, IIT Tirupati
Probability and Statistics
Dr. Ishapathik Das, IIT Tirupati
• The number of aircraft/road accidents
in any time interval.
• The number of arrivals in any service
facility like at an ATM, bank, railway
station, petrol pump,…
• The number of patients in a doctor's
clinic.
• The number of visitors to a mall,
exhibition,…
• The number of services completed
per unit time at a bank or any other
service facility.
Probability and Statistics
Dr. Ishapathik Das, IIT Tirupati
Probability and Statistics
Dr. Ishapathik Das, IIT Tirupati
• Represents a situation where all
outcomes in a range are equally
likely.
• The position of a particular molecule
in a room.
• The point on a car tyre where the
next puncture will occur.
• The distance from the origin after
throwing a dart to target on a board.
Probability and Statistics
Dr. Ishapathik Das, IIT Tirupati
Probability and Statistics
Dr. Ishapathik Das, IIT Tirupati
Probability and Statistics
Dr. Ishapathik Das, IIT Tirupati
• The gamma distribution is often
concerned with the amount of time
until some specific event occurs.
• The amount of time (beginning now)
until an earthquake occurs.
• The length, in minutes, of long
distance business telephone calls.
• The amount of time, in months, a car
battery lasts.
• The time between two customers
come at a computer center.
Probability and Statistics
Dr. Ishapathik Das, IIT Tirupati
Probability and Statistics
Dr. Ishapathik Das, IIT Tirupati
• Height
• Shoe size
• Birth weight
• IQ
• Income Distribution
• Technical Stock Market
• Student’s report
Probability and Statistics
Dr. Ishapathik Das, IIT Tirupati
Probability and Statistics
Dr. Ishapathik Das, IIT Tirupati
Probability and Statistics
Dr. Ishapathik Das, IIT Tirupati