WESTERN INSTITUTE FOR SOCIAL RESEARCH
3220 Sacramento Street. Berkeley, CA 94702 (415) 655-2830
This paper was written to be used in conjunction with seminars on action-research initially
given at WISR from 1980-83, as part of a nationwide demonstration project to “extend the
teaching, learning and use of action-research throughout the larger community” under a
grant from the US Department of Education’s Fund for the Improvement of
Postsecondarfy Education
The Uses, Misuses and Abuses of Statistics
Two meanings of "STATISTICS":
A numerical "fact"-in this sense, "statistics" involves counting things (e.g., she has five
brothers and sisters) or using some standard of' “measurement” (e.g., he is six feet tall and has an
IQ of 105).
A field of study which uses mathematical methods to help people to make sense out of
large (and/or complicated) masses of information--in this sense, statistical techniques are used to
describe groups of people, types of programs and situations, and the like, and to draw inferences
(or make indirect conclusions--"calculated" judgments) about similarities and differences
between two or more groups of people, groups of events and circumstances, or types of
programs, for example.
SOME OF THE ADVANTAGES AND USES OF STATISTICS ARE:
To provide a simplified way of talking about complex qualities (by describing the
qualities in quantitative terms). Examples are the uses of IQ tests to measure intelligence, use of
"age" to estimate the extent of a person's experience and/or social or developmental status, use of
blood pressure to estimate cardiovascular health, the use of psychological testing to characterize
mental health, and the use of course grades to characterize one's learning and academic
achievement.
To summarize information about groups in a shorthand way, or to estimate characteristics
of a population (a very large group of people or situations) without having to study the entire
group (but instead by studying a smaller sample). This saves time and helps one get to a main,
manageable point. Examples of this are seen in opinion polls, in psychological studies that try to
generalize about certain "types" of people by studying a small group of people who seem to be
representative of that "type." In this way, statistics address "The problem of making meaningful
statements about the world on the basis of examination of only a small part of it." (M.G. Kendall,
"The History of Statistical Method.").
To provide standardized (easily repeatable) procedures for detecting relationships
between groups –for example, for comparing two groups of people in terms of their health or
mental health, or for comparing two different treatment programs to see if one is more
"effective" than another in accomplishing certain goals. Assuming we use such statistical
procedures with an open mind, we may detect patterns in a complicated mass of information,
which might otherwise go unnoticed or be "incorrectly" interpreted because of our biases. The
mathematical methods provide us with some standards for evaluating the extent to which we
should be confident in our perception of certain patterns in the data (e.g., a perception that
certain family circumstances are a central factor in giving rise to a particular type of mental
illness).
SOME REMINDERS ABOUT DEMYSTIFYING STATISTICS:
We are going to learn how to think about numbers as though they were words or
concepts—because in a fundamental way, that's all they are. In other words, we shall try to reach
for qualitative understandings of quantitative formulations.
Statistical methods and numerical measurements are no substitute for our own judgment.
We need to use our experience and previous observations, our open-mindedness and curiosity,
our intuition and ability to make interesting speculations, our theoretical knowledge and
understanding of the facts in our fields, and probably good old fashioned common sense, too.
We will need to continually ask ourselves such questions as: what do we want to measure and
why? What is a "good" measure of this? What different, real world interpretations of events
might account for this particular statistical result? Or, for that matter, why was this particular
statistical method appropriate (in light of our purposes and the nature of whatever it is we are
studying)?
SOME OF THE LIMITATIONS OF STATISTICS ARE:
They can oversimplify. For example, grades summarize the complexity of all that a
person learns in a class into one symbol (even though they may have learned much in one area,
and very little in another way). Among other things, IQ tests can oversimplify because there are
many different types of human intellect, and consequently, ways of exercising "intelligence"
cannot be so neatly summarized in one concept or measure of "intelligence." And, because of the
critiques of critically-minded people, it has been shown that there are many cultural biases and
other problems of “validity” (a statistical concept we’ll learn more about) that call into question
the accuracy, the meaningfulness, and usefulness of these “intelligence” tests. Similarly, costbenefit equations reduce profound human events and conditions (e.g., health, learning, and even
life and death) to dollars and cents. The tendency to do this kind of quantitative
oversimplification can be traced in part to certain historical trends and forces, and the efforts by
some to view “dollars and cents” as the main criterion of what is of “value” in our lives.
However, it is not an obvious "fact" or "truth" that decision-making about what is valuable in life
should be boiled down to units of economic currency. [Cite the infamous example where the
producers of the Pinto automobile went ahead with making a dangerous vehicle that might
explode on impact from the rear because their studies indicated that the cost of lawsuits due to
injuries and loss of life would likely be less than the cost of re-designing the vehicle so that it
would be more safe.]
Estimates about groups are not so useful in helping us to make predictions or judgments
about individuals, or specific events, without additional information about the particular
individual or circumstances with which we are concerned. For example, we have never met Joe
but have to greet him in an airport. Joe plays on a basketball team and their average height is 6''
7". Do we look for someone quite tall, at the risk of overlooking a shorter person who has a look
on his face that he is looking for someone he has never met but who is supposed to be meeting
him in the airport? Or, to take a different example, suppose that people who get scores on the
MMPI [personality tests that assesses the likelihood of an emotional disturbance] like Bob are
highly disturbed. Should we then assume that Bob is very disturbed, as well? How do we use this
information without being blinded by other, perhaps contradictory information?
Mathematical methods are just aids to help us in drawing inferences and making
judgments about how to make sense out of information, they can’t do it for us.. For example, we
have to understand why based on present information and past knowledge available to us, a
particular method is the “right” one to use. Further, we need to critically examine the
“reasonableness” of whatever conclusions we draw from mathematical computations. If we
knew nothing about freeway driving, what might we conclude from the statistical “finding” that
a very large percentage of cars start blinking on the right side of their tail lights just as they reach
a certain 100 yard stretch of the freeway? To consider a different example: we could compare
through statistical tests the exercise habits of hospitalized people with those not hospitalized in a
given year to test the importance of exercise for health. But this might not be as useful to the
objective of examining the relation between exercise and health as making the same comparisons
as suggested above but with separate comparisons for each age group. There are of course many
other ways we could improve our thinking about how to “statistically test” the importance of
exercise to health. In other words, our decisions about which statistical methods to use and how
to use them should be influenced by our information and knowledge.