Frequently Asked Questions: Basic Probability
1. What does “correlation” mean?
If two events are correlated, then they tend to change together. Knowing
something about the one event tells you something about the other. If two events are
independent, then the occurrence or nonoccurrence of the one tells you nothing about
the other. Consider this example. I look out the window and tell you “I see a person
coming out of the Wright building. What is your subjective probability that that person is
female? Write it down on paper.” Then I tell you that the person’s last name is “Jones”
and I invite you to revise your subjective probability. Since in your experience there has
been no relationship between having a last name of “Jones” and being of the one or the
other sex, your new subjective probability is unchanged. You consider the two events
(being of the one sex or the other and having a last name of “Jones”) to be independent,
that is, not correlated. Then I add, “and Jones is wearing a skirt, you may revise your
subjective probability again, if you wish.” Now you probably would revise your
subjective probability, upwards, since in your experience people who wear skirts are
usually female. You think of being female and wearing a skirt as positively correlated
(they tend to go together), not independent. Likewise, you think of wearing a skirt and
being male as negatively correlated – knowing that the one event has happened makes
the other event less likely.
2. How does one decide which approach to use when estimating a
probability – empirical, rational, or subjective?
It is both a matter of personal preference and the data that are available.
Suppose I told you that I have a box of poker chips. Some are red, some are white,
some are blue. I ask you to estimate the probability of drawing, on a single trial, a blue
chip. You ask me if you can first experiment with the box of chips. I say “sorry, no.”
You think about it (the rational approach). You make the assumption that the three
types of chips are all equally likely to be drawn (there are equal numbers of them in the
box, they are all the same size and shape, and they are randomly distributed within the
box). You know (or assume) that drawing a blue chip and drawing a not blue chip are
mutually exclusive events – a chip cannot be both blue and not blue, and the same is
true of red versus not red and white versus not white. You also assume that the
drawing blue, drawing red, and drawing white are mutually exhaustive events – there
are no other events that can happen. Accordingly, you reason that P(blue) + P(red) + P
(white) = 1 or 100%. Since you have assumed that the three events are equally likely,
that is, P(blue) = P(red) = P(white), each of the probabilities must be 1/3.
Now I offer you 3:1 odds against the drawn chip being blue (this is just another
way of saying 1:3 odds in favor of the drawn chip being blue). You are thinking that is a
good bet (better than fair) if betting on blue. If you are right, and the P(blue) is 1/3, then
2:1 odds would be fair. You take the bet. You put in $1, I put in $2. You draw a chip
from the box. It is white. You lose a dollar. Hey, what do you expect, you probably
would not hit blue on the first draw, bet again. We go through ten bets, and you get only
one blue chip. You won $2 on that one bet but lost $9 on the other nine. That does not
sound fair. I want you to keep on betting. You say “only if I can experiment with the box
first.” Reluctantly, I say “OK.”
You do a 1,000 trial experiment. On each trial you draw one chip, note its color,
and then put it back in the box and shake the box well. In 1,000 trials you draw blue
chips only 102 times. From that result, you estimate the probability of drawing a blue
chip to be 102/1,000 = about 10%. This is the empirical approach. After you finish, I
say “OK, lets bet.” You say “OK, but the bet won’t be fair unless the odds are 9:1
against getting a blue chip.”
The subjective approach is whatever it is that humans do when making
decisions that involve the likelihood of something happening or not. Although there are
strong common threads here, there are also individual differences. Generally speaking,
humans make a lot of errors in this domain. There is a large and interesting research
literature on this topic, and I would love to talk about it, but, frankly, we just do not have
the time to talk about that right now.
3. What is the relationship between the addition rule and mutual exclusion?
Suppose that I ask you to bet on whether the next chip drawn from the box will
be blue or red. From your experiment you know that the P(blue) = 10% and the P(red)
= 30%. The addition rule is P(A  B) = P(A) + P(B) if A and B are mutually
exclusive. You did not see any chips that were both red and blue, so you estimate the
P(blue or red) to be 10% + 30% = 40%. A fair bet would be 60:40 or 3:2 odds against
the chip being blue or red.
Now I bring out a new box of chips. I promise you that for this box P(blue) = 10%
and P(red) = 30%. What is the probability that a randomly drawn chip will be blue or
red? What I do not tell you is that 5% of the chips are red with blue polka dots, that is
they are both blue and red. Now being blue and being red are not mutually exclusive.
In this case the simplified addition rule (above) will not work, we need the more general
addition rule, P(A  B) = P(A) + P(B) - P(A  B). P(blue or red) = 10% + 30% -5% =
35%,
Look back at my handout “Basic Probability, Including Contingency Tables” for
another example of the addition rule when events are not mutually exclusive, where we
would have decided that 127% of female students are going to hell if we did not take
into account that smoking and being sexually active are not mutually exclusive.
Consider the diagram to the right. The area of the
red circle on the left represents P(A). The area of the
blue circle on the right represents P(B). The overlap
between the two circles, with the purple dots, represents
P(AB), the joint occurrence of A and B – events that are
both A and B. A and B are not mutually exclusive. If we
were to calculate P(A  B) = P(A) + P(B) we would count
the overlap area twice. To correct for this, we need to
subtract out the overlap area once, that is, P(A  B) = P(A) + P(B)  P(A  B).