ICOTS 2, 1986: I.W. Kelly and F.W. Zwiers
MUTUALLY EXCLUSIVE AND INDEPENDENCE:
UNRAVELLING BASIC MISCONCEPTIONS IN PROBABILITY THEORY
I.W. Kelly
University of Saskatchewan
F.W. Zwiers
Environment Canada
T w o c e n t r a l concepts in p r o b a b i l i t y t h e o r y a r e those o f "independence"
a n d o f "mutually exclusive" e v e n t s a n d t h e i r alternatives. In t h i s a r t i c l e
we p r o v i d e f o r t h e i n s t r u c t o r suggestions t h a t can b e used t o e q u i p
s t u d e n t s w i t h an i n t u i t i v e , comprehensive u n d e r s t a n d i n g o f these basic
concepts.
L e t u s examine each o f these concepts i n t u r n along w i t h
common s t u d e n t misunderstandings.
1. Mutually Exclusive versus Non Mutually Exclusive Events
Events a r e m u t u a l l y exclusive when t h e occurrence o f one o f t h e e v e n t s
r u l e s o u t t h e p o s s i b i l i t y o f t h e occurrence o f t h e other events o f concern.
T h e outcomes, f o r example, on t h e toss o f a single die a r e 1, 2, 3, 4, 5 o r
6. T h e outcomes a r e all m u t u a l l y e x c l u s i v e because when a d i e i s tossed
a n d a number t u r n s up, all t h e o t h e r numbers cannot occur. On t h e o t h e r
hand, t h e e v e n t s "eating r o c k y mountain o y s t e r s f o r b r e a k f a s t " a n d "eati n g vegetable soup f o r b r e a k f a s t " a r e & mutually exclusive since it i s
p o s s i b l e t h a t one has b o t h f o r breakfast, however improbable t h a t m i g h t
be.
T h e d i s t i n c t i o n between c o n t r a d i c t i o n a n d c o n t r a r i e t y should b e i n t r o d u c e d
a t t h i s p o i n t t o students. T w o statements are contradictories when t h e y
cannot b o t h b e t r u e , and cannot b o t h b e false. T w o statements a r e
t r a r i e s if t h e y can b o t h b e false a n d a third statement, d i f f e r e n t f r o m
both, can b e t r u e . For example, "It i s r a i n i n g outside t h i s b u i l d i n g a t t h i s
moment" a n d "It i s not r a i n i n g o u t s i d e t h i s b u i l d i n g a t t h i s moment" a r e
contradictories. However, t h e statements " A l l mathematicians a r e v e r y i n t e l l i g e n t " a n d "No mathematicians a r e v e r y intelligent" are c o n t r a r i e s since
a l t h o u g h b o t h cannot b e t r u e , it i s almost c e r t a i n l y t h e case t h a t b o t h a r e
false. Some mathematicians a r e v e r y i n t e l l i g e n t a n d others are not.
con-
T h e r e a r e m u t u a l l y exclusive e v e n t s o f b o t h t y p e s . I n t h e d i e t o s s i n g
example a l r e a d y described we c o u l d d i v i d e t h e sample space i n t o t w o parts,
say, "even" a n d "odd". These e v e n t s are m u t u a l l y exclusive a n d c o n t r a d i c t o r y . B y t a k i n g t h e sample space t o b e t h e outcomes 1, 2, 3, 4, 5 a n d 6
we have i m p l i c i t l y made an assumption about t h e operation o f d i e tossing:
namely, t h a t it i s impossible f o r t h e d i e t o come t o r e s t on a p o i n t o r an
edge. If t h e sample space were expanded t o include these possibilities t h e n
t h e e v e n t s "even" a n d "odd" a r e contraries. T h e reason f o r i n t r o d u c i n g
these concepts is, o f course, t o e n s u r e t h a t t h e s t u d e n t i s q u i t e clear
a b o u t t h e f a c t t h a t a p a i r of m u t u a l l y e x c l u s i v e events a r e n o t necessarily
complementary events.
ICOTS 2, 1986: I.W. Kelly and F.W. Zwiers
The die example presented above has somewhat of a contrived air about it
because most of us would accept t h e proposition t h a t t h e probability of a
die coming t o rest on a point o r an edge is zero. However, things are not
always t h a t clear c u t i n nature. Amongst humans there are males, females,
and other borderline cases (e.g., Klinefelter's syndrome) which occur w i t h
nonzero probability. Thus, t h e events "male" and "female" are i n f a c t contraries.
Another point t h a t should be considered i n class is t h e fact t h a t v e r y often
i n the real world it is not always clear whether o r not two events are
mutually exclusive.
The examples we present t o students when teaching
t h e concept are, understandably, clearcut. For example, if we obtain a six
on a single toss of a f a i r die, it rules out the possibility of obtaining a
one, two, three, four, o r five. I n the real world we often do not have
enough information t o be sure. Not all students are aware t h a t "Clark
Kent'' and "Superman" are
mutually exclusive. It would be useful t o
draw examples from nature where t h e distinction between contradictory,
contrary, mutually exclusive and non-mutually exclusive are not always
clear.
not
2. Independent versus Dependent Events
Events are independent when the occurrence (or nonoccurrence) o f one o f
the events carries no information about the occurrence (or nonoccurrence)
of the other event. Mathematically, two events A and B are considered t o
be independent if P(A n B) = P(A)- P(B). For example, if the probability
t h a t Obadiah has escargot f o r breakfast tomorrow is 0.4 and the probability t h a t it will rain tomorrow is 0.3, then the probability t h a t both events
will occur tomorrow is (0.4)*(0.3)=0.12.
When we have more than two
events, t h e situation becomes a b i t more complicated
all possible combinations of component events must follow the multiplication rule. T h a t is,
each combination must also involve independent events.
-
Students have several difficulties w i t h the distinction between independent
and dependent events. The f i r s t parallels the problem t h a t arises with
mutually exclusive events, namely, determining when events i n the real
world are independent o r dependent. Once again, we seldom help students
t o bridge the gap between the fuzzy distinctions apparent i n nature and
t h e v e r y r i g i d distinctions made i n mathematics. If we toss a pair of dice,
t h e outcome t h a t occurs on one due obviously (for most of us) does not i n fluence what outcome will occur on the other die. I n other cases one often
needs expertise i n a particular area t o make a reasoned judgement whether
particular events are independent or not. Many years of research were
required t o demonstrate t h a t there is a dependent relationship between
smoking and l u n g disease (see also, Ayton & Wright, 1985).
A second common misunderstanding involves interpreting a dependent relationship between events as a causal relationship. There are, of course, s i t uations where this is plausible, f o r example, having tuberculosis is dependent on having tuberculosis bacilli i n one's body. However, there are many
examples of dependent relationships between events where no causal relationship is involved. For example, having a f i r e is dependent on t h e p r e -
ICOTS 2, 1986: I.W. Kelly and F.W. Zwiers
sence o f oxygen, although t h e l a t t e r does n o t cause t h e former. T h i s is
something we can help students w i t h simply by presenting examples.
A n issue t h a t sometimes arises w i t h students who have done reading i n
physics o r philosophy concerns whether events can ever really be independent. For example, some w r i t e r s (e.g., Capra, 1975) take a holistic approach t o t h e universe i n which t h e universe i s considered t o be a web o f
relationships where all t h i n g s communicate intimately w i t h one another and
all being i s shared. Statisticians sometimes suggest similar things, adding
t o student confusion. Hays (1981 p. 293), f o r example, tells us t h a t
"There is s u r e l y nothing on e a r t h t h a t is completely independent o f a n y t h i n g else". A student coming across such a statement would be u n d e r standably confused. T h e student m i g h t reason t h a t if nothing is completely
independent o f a n y t h i n g else, t h e n how can we a p p l y p r o b a b i l i t y formulas
t h a t assume independence o f events? T h e answer i s t h a t application o f
such formulas doesn't need t o assume t h a t t h e events are completely independent of each other, o n l y t h a t a n y relationship is negligible. For
example, e v e r y b o d y i n t h e universe has some interactive gravitational a t t r a c t i o n w i t h e v e r y other body, b u t t h e gravitational attraction between a
human being and a star l i g h t years away is so minute t h a t it can be considered non-existent. Underwood (1957, p . 6) p u t it well:
The length of an astronomer's toenails isn't related to phases of the
moon; the color of the secretary's hair isn't related to the height to
which the corn grows in an Iowa field, and a pygmy tribe in New
Guinea has little influence on the alcoholic consumption of a t r u c k
drive/: in Brooklyn.
As a final point, we should p o i n t o u t t h a t independence is a d i f f i c u l t idea
t h a t is clearly understood i n relation t o specific examples
we t a l k about
conducting an experiment i n class and then t a l k about whether t w o events
(describing sets o f possible outcomes o f t h e experiment) are independent.
We use phrases such as "events A and B are independent if knowledge
about whether A has occurred provides us w i t h no knowledge about
whether B has occurred". I n a v e r y subtle way an element o f time i s
hinted a t i n such a statement and it often confuses students. it is t h e r e f o r e important t o emphasize t o students t h a t whatever t h e temporal relationship between two (or more) independent events, a knowledge o f t h e
occurrence (or nonoccurrence) o f a n y o f t h e events provides one w i t h no
knowledge o f t h e f u t u r e
past outcomes o f a n y o f t h e other events.
-
or
I n o r d e r t o understand how t h i s problem w i t h time can arise we w i l l consider t h e simple example o f a coin tossing experiment i n which a f a i r coin
i s tossed six times. Associated w i t h t h e experiment is a sample space, t h e
l i s t o f all possible outcomes o f t h a t experiment, and a collection of events.
T h e latter represent more general descriptions o f outcomes of t h e e x p e r i ment and can usually occur i n more t h a n one way. For example, t h e event
"three heads come u p i n t h e six tosses of a coin, can occur i n 20 ways.
Now suppose t h a t you are blindfolded before t h e experiment is conducted
and t h a t you have t h e o p p o r t u n i t y t o b e t t h a t t h e last two tosses o f t h e
coin will come u p heads. Quick mental arithmetic tells you t h a t t h i s can
happen one time i n f o u r . Now t h e coin is tossed six times and, s t i l l b l i n d folded, y o u are t o l d t h a t t h e outcome was t h r e e heads and g i v e n an o p p o r -
ICOTS 2, 1986: I.W. Kelly and F.W. Zwiers
t u n i t y t o revise your bet. You realize t h a t there are only f o u r ways i n
which t h e outcome of six tosses of a coin can result i n a total of three
heads with a head on each of the last two tosses. Therefore, the relative
chances of being a winner once you've been given some knowledge of the
outcome of the experiment has been reduced from one-in-four t o one-infive. The events "three heads i n six tosses of a coin" and "the last two
tosses i n a sequence of six tosses are heads" are not independent.
The element of time i n t h e above example relates not t o t h e way i n which
the experiment was conducted, b u t t o t h e way i n which we t h i n k . We were
given some knowledge about the outcome of the experiment, namely t h a t
three heads had occurred. This i n t u r n gives us some knowledge about the
likelihood of other events t h a t may have occurred. We now t h i n k t h a t it is
less likely t h a t there was a head on each of t h e last two tosses than before
the experiment was conducted. This is essentially what we mean when we
say t h a t two events are dependent - knowledge t h a t one event has occurred conveys information about whether o r not t h e second event also
occurred. The events "heads on t h e f i r s t toss" and "heads on the t h i r d
toss" are independent, because when we re-evaluate the likelihood t h a t
second event occurred i n l i g h t o f the occurrence of the f i r s t event, we see
t h a t we have no more knowledge about t h e outcome of the second event.
The element of time which we mentioned is associated with t h i s process of
re-evaluation of probabilities after some information about the outcome of
the experiment is available. These a posteriori, o r conditional probabilities can only be applied t o decision making, such as whether t o continue a
bet o r raise the ante i n a poker game, after some information about t h e o u t come of the experiment is available.
These probabilities are evaluated b y conceptually repeating the experiment
with a restricted sample space. We feel t h a t students have some i n k l i n g of
what goes on, b u t t h a t we don't explain these concepts t o them carefully
enough. We suspect t h a t t h e y do feel t h a t something happens as time goes
on b u t don't really understand its mechanics. We should point out, using
simple examples such as coin tossing o r card games, t h a t we construct new
probability models f o r the experiment as it progresses (or after the fact)
which are conditional upon information which we have received about the
outcome of the experiment t o t h a t point. With these ideas about conditional
probability i n place it can be shown t h a t two events, say A and B, are i n dependent if the a p r i o r i probability of A is equal t o the a posterio r i probability of A given B. We t h i n k t h a t such an approach is much
easier f o r the student t o understand than the standard approach which is
found in most t e x t books. The latter consists of stating the usual definition of independence, i.e., t h a t A and B are independent if
P (A n B) = P (A)
P (B)
and then presenting a few examples t o illustrate the t r u t h of the definition. We often confuse the student through our use of language, as discussed above, and the failure t o l i n k t h e ideas of independence and conditional probability.
ICOTS 2, 1986: I.W. Kelly and F.W. Zwiers
3. Confusion Between Independent Events and Mutually Exclusive Events
Students not only have difficulty with the notions of mutually exclusive
and independent events; they v e r y often confuse the two. Most of the confusion arises because we, as instructors, do not take the time t o relate the
two concepts.
We frequently answer the question "If A and B are mutually exclusive,
does it follow that they are not independent?" With the reply, ,"well, no.
For example,
and then t r o t out the pathological example of a pair of
events of probability zero without f u r t h e r explanation. Except i n certain
pathological cases, the concept of mutually exclusive events is the exact
antithesis of the concept of independent events. A nice illustration is p r o vided b y Hays (1981, p. 43-44). Suppose that all men are either "bald" o r
have a "full head of hair". These are mutually exclusive. Let us say the
probability of selecting a bald man from the population is 0.60; the probab i l i t y of selecting a hairy man would be 0.40. If these two events were i n dependent, then the probability of selecting a man who i s both bald with a
head f u l l of hair would be equal t o (0.60). (0.40). B u t the probability of
such a joint event is, of course, zero. Mutually exclusive events are ( a l most) never independent. Ad hoc explanations tend t o deal with specific
aberrations t h a t may lead t o f u r t h e r confusions and moreover avoid dealing
with the fundamental principles upon which the student's question i s
based.
. . ."
B y planning f o r instruction of these two fundamental concepts we can i n sure t h a t the student's understanding i s built up systematically. This p r o vides a firmer foundation upon which the student can acquire a grasp of
probability theory.
References
Ayton P., E Wright, G. (1985). Thinking with probabilities. Teaching statistics, 2(2), 37-40.
Capra, F. (1975). The tao of physics. Berkeley: Shambala Books.
Hays, W.
Winston.
L.
(1981).
Statistics.
Underwood, B. J. (1957).
Appleton-Century-Crofts.
New York,
Psychological
N.Y.:
research.
Holt,
Rinehart &
New York,
N. Y. :