DEVELOPING LITERACY IN
QUANTITATIVE
RESEARCH METHODS
Dr Christina Hughes
University of Warwick
C.L.Hughes@warwick.ac.uk
These materials have two inter-related aims. The primary aim is
to develop students' literacy in the use and reading of research
that uses quantitative data. The second is to enhance students'
confidence in their understandings of such approaches. To
achieve these aims the package will introduce students to a
number of basic statistical techniques that are used in social
research. In addition the materials will explore some common
concepts that underpin quantitative social research.
The specific objectives are:
· To develop understandings of the relationship between
different types of quantitative data and their implications for
descriptive and inferential statistical techniques;
· To develop understandings of the statistical techniques of:
measures of central tendency, measures of dispersion;
· To explore the meanings of correlation and causality in
relation to quantitative social research;
· To explore uses, and misuses, of official statistics.
Quantitative techniques are most commonly associated with
survey and experimental research designs. As the name
suggests, quantitative research is concerned with the collection
and analysis of data in numeric form. It tends to emphasize
relatively large-scale and representative sets of data, and is often
(problematically) presented or perceived as being about the
gathering of `facts'. Because of strong associations that are made
between statistics as social facts and dominant ideas of science
as objective and detached, quantitative strategies are often
viewed as more valid.
Many small-scale research studies that use questionnaires as a
form of data collection will not need to go beyond the use of
descriptive statistics and the exploration of the interrelationships
between pairs of variables. It will be adequate to say that so
many respondents (either the number or the proportion of the
total) answered given questions in a certain way; and that the
answers given to particular questions appear to be related. Such
an analysis will make wide use of proportions and percentages,
and of the various measures of central tendency (averages) and
of dispersion (ranges).
You may, however, wish or need to go beyond this level of
analysis, and make use of inferential statistics or multivariate
methods of analysis. There are dozens of inferential statistics
available: three commonly used examples are Chi-square;
Kolmogorov-Smirnov and Student's t-test. The functions of
these statistics vary but they are typically used to compare the
measurements you have collected from your sample for a
particular variable with another sample or a population in order
that a judgement may be made on how similar or dissimilar they
are. It is important to note that all of these inferential statistics
make certain assumptions about both the nature of your data and
how it was collected. This means that you have to be clear
whether your data is, for example, nominal, ordinal, interval or
ratio. If these assumptions do not hold these measures should
not be used.
Multivariate methods of analysis may be used to explore the
interrelationships among three or more variables
simultaneously. Commonly used examples include multiple
regression, cluster analysis and factor analysis. While you do
not need to have an extensive mathematical knowledge to apply
these techniques, as they are all available as part of computer
software packages, you should at least have an understanding of
their principles and purposes.
One key point to be aware of when carrying out quantitative
analysis is the question of causality. One of the purposes of
analysis is to seek explanation and understanding. We would
like to be able to say that something is so because of something
else. However, just because two variables of which you have
measurements appear to be related, this does not mean that they
are. Statistical associations between two variables may be a
matter of chance, or due to the effect of some third variable. In
order to demonstrate causality, you also have to find, or at least
suggest, a mechanism linking the variables together.
[Extracted from Blaxter, Hughes and Tight, 1996]
Bibliography
This bibliography includes texts that are useful for students
new to quantitative techniques and those that are useful for
the more advanced. The asterisk (*) indicates those that are
introductory. The key publishers of methodology texts are
Sage, Routledge and Open University Press. If you wish to
extend your reading or keep up to date with developments you
should put your name on these publishers' catalogue mailing
lists. There are also a number of journals that are primarily
concerned with developments in methodology. These include:
The International Journal of Social Research Methodology
and Social Research Online ( http://www.socresonline.org.uk).
In addition, secondary sources produced by the Office for
National Statistics for the Government Statistical Service can
be obtained from The Office for National Statistics, 1
Drummond Gate, London, SW1V 2QQ or through the
STATBASE on-line directory.
Black, T (1999) Doing Quantitative Research in the Social
Sciences: An Integrated Approach to Research Design,
Measurement and Statistics, London, Sage
Blaxter, L, Hughes, C and Tight, M (1996) How to Research,
Buckingham, Open University Press*
Bowling, A (1997) Research Methods in Health:
Investigating Health and Health Services, Buckingham, Open
University Press*
Bryman, A and Cramer, D (1990) Quantitative Data Analysis
for Social Scientists, London, Routledge
Calder, J (1996) Statistical Techniques, in R Sapsford and V
Jupp (Eds) Data Collection and Analysis, London, Sage, pp
225-261
Cramer, D (1994) Introducing Statistics for Social Research:
Step-by-step calculations and computer techniques using
SPSS, London, Routledge
Denscombe, M (1998) The Good Research Guide: For small
scale social research projects, Buckingham, Open University
Press*
De Vaus, D (1991) Surveys in Social Research, Sydney, NSW,
Allen and Unwin
Hek, G, Judd, M and Moule, P (1996) Making Sense of
Research: An Introduction for Nurses, London, Cassell*
Hinton, P (1995) Statistics Explained: A guide for social
science students, London, Routledge*
Leary, M (1991) Introduction to Behavioural Research
Methods, Belmont, Calif, Wadsworth Publishing
Levitas, R and Guy, W (1996) Interpreting Official Statistics,
London, Routledge
Persell, C and Maisel, R (1995) How Sampling Works,
Newbury Park, Calif, Pine Forge
Pilcher, D (1990) Data Analysis for the Helping Professions:
A Practical Guide, Newbury Park, Calif, Sage
Sapsford, R (1996) Extracting and Presenting Statistics, in R
Sapsford and V Jupp (Eds) Data Collection and Analysis,
London, Sage, pp 184-224
Solomon, R and Winch, C (1994) Calculating and Computing
for Social Science and Arts Students, Buckingham, Open
University Press*
Stanley, L (Ed) (1990) Feminist Praxis, London, Routledge
Townsend, P (1996) The Struggle for Independent Statistics on
Poverty, in R Levitas and W Guy (Eds) Interpreting Official
Statistics, London, Routledge, pp 26-44
Traub, R (1994) Reliability for the Social Sciences: Theory
and Application, Thousand Oaks, Calif, Sage
Wright, D (1997) Understanding Statistics: An introduction
for the social sciences, London, Sage*
TYPES OF QUANTITATIVE DATA
Nominal data
Nominal data come from counting things and placing them in a
category. They are the lowest level of quantitative data in the
sense that they allow little by way of statistical manipulation
compared with the other types. Typically there is a head count
of members of a particular category, such as female/male or
African Caribbean/South Asian. These categories are based
simply on names; there is no underlying order to the names.
Used for the following descriptive statistics: proportions,
percentages, ratios.
Ordinal data
Like nominal data, ordinal data are based on counts of things
assigned to specific categories but in this case the categories
stand in some clear, ordered, ranked relationship. The categories
are `in order'. This means that the data in each category can be
compared with the data in the other categories as being higher or
lower than, more or less than, etc. those in other categories. The
most obvious examples of ordinal data come from the use of
questionnaires in which respondents are asked to respond to a
five-point Likert scale. It is worth stressing that rank order is all
that can be inferred. With ordinal data we do not know the
cause of the order or by how much they differ.
Used for the following descriptive statistics: proportions,
percentages, ratios.
Interval data
Interval data are like ordinal data but the categories are ranked
on a scale. This means that the `distance' between the categories
is a known factor and can be pulled into the analysis. The
researcher can not only deal with the data in terms of `more
than' or `less than' but also say how much more or how much
less. The ranking of the categories is proportionate and this
allows for direct contrast and comparison. Calendar years are
one example. This allows the researcher to use addition and
subtraction (but not multiplication and division) to contrast the
difference between various periods.
Used for the following descriptive statistics: measures of central
tendency (mode, median, mean)
Ratio data
Ratio data are like interval data except that the categories exist
on a scale which has a `true zero' or an absolute reference point.
When the categories concern things like incomes, distances and
weights they give rise to ratio data because the scales have a
zero point. Calendar years, in the previous example, do not exist
on such a scale because the year 0 does not denote the beginning
of all time and history. The important thing about the scale
having a true zero is that the researcher can compare and
contrast the data for each category in terms of ratios, using
multiplication and division, rather than being restricted to the
use of addition and subtraction as is the case with interval data.
Ratio data are the highest level of data in terms of how
amenable they are to mathematical manipulation.
Used for the following descriptive statistics: measures of central
tendency (mode, median, mean)
[adapted from Blaxter, Hughes and Tight, 1996 and Denscombe,
1998]
TYPES OF QUANTITATIVE DATA
EXAMPLES
Are the following nominal, ordinal, ratio or interval data?
· The income levels of social workers;
· The examination scores of members of this course;
· The sex of your research participants;
· The birth position of members of a family;
· Exam grades received at school;
· Number of exam passes;
· The temperatures of different geographical zones;
· The size of families in the UK;
· IQ scores;
Illustrative Issue
A Likert scale is written to convey equidistant points along an
axis:
*-----------------*---------------- *---------------- *---------------- *
Very Fairly Important Not very Not at all
Important Important Important Important
Are the meanings ascribed by research respondents similarly
equidistant?
Is such data interval or nominal?
TYPES OF QUANTITATIVE DATA
A CAUTIONARY COMMENT
Very important 1
Fairly important 2
Not very important 3
Not at all important 4
The problem is that the `real' distance between the ratings
numbered 3 and 4 for a respondent may be much greater than
the distance they perceive between the items numbered 1 and 2.
The `real' distances between each of the ratings may also vary
from person to person. In theory, therefore, such data should be
treated as ordinal data. Most researchers take a pragmatic
approach, however, and continue with the practice of treating
ratings and psychological tests as interval data.
One way of dealing with data that are difficult to `type' correctly
is through the use of models. Scientists use models of weather
systems to study the relationships between different factors in
order to understand better what the contributory factors are. In
the same way, statisticians produce statistical models based on
their current understanding of the problem. When they do not
quite work as expected, they modify some of their assumptions.
If the assumption of an interval scale does not work, then
further analyses can be carried out on the assumption of an
ordinal scale. Over the years, reviews of the statistical evidence
suggested that the assumption of equality of equal intervals
within rating scales is justified. But where such assumptions are
made, there is always the possibility of misinterpretation of the
data. The important point is to be clear always that there are
different types of data, and that this will affect the type of
analyses that can be used on them. (Calder, 1996: 229)
MEASURES OF CENTRAL TENDENCY
OR MID-POINTS AND AVERAGES
There are three types of average and these are collectively called
`measures of central tendency'. These are the mean, the median
and the mode.
The mean (or arithmetic average)
This is the most common meaning of `average'. It includes the
total spread and finds the mid-point. To calculate the mean:
1. Add together the total of all the values for the category
2. Divide this total by the number of cases
· The mean cannot be used with nominal data. For example, you
cannot `average' names, sexes, nationalities and occupations.
· The mean is affected by extreme values, or outliers. Because
the mean includes all values the average can be pulled toward
the value of the outlier or toward the more extreme values.
· The mean can lead to strange descriptions, such as 2.4 person
households.
Example: Calculate the mean from the following:
•
1 4 7 11 12 17 17 47
The median or mid-point
The median is the mid-point of the range. To calculate:
1. Place the values in ascending/descending rank order
2. Find the mid-point number
3. With even numbers of values the mid-point is half-way
between the two middle values
· The median can be used with ordinal data as well as interval
and ratio data.
· The median is not affected by extreme values or outliers.
· The median works well with a low number of values.
· The main disadvantage is that you can do no further
calculations with the median.
Example: Calculate the median from the following:
•
1 4 7 11 12 17 17 47
The Mode
The mode is the value that is most common. To calculate:
1. Arrange the data in ascending/descending order;
2. Identify the value that occurs more frequently than any other.
· The mode can be used with nominal, ordinal, interval and ratio
data. It has the widest possible scope therefore.
· It is unaffected by outliers or extreme values.
· It does not allow any further mathematical calculations.
· There may not be any `most common' values or there may be
more than one.
Example: Calculate the mode of the following:
1 1 4 4 7 11 12 17 17 17 47
MEASURES OF DISPERSION
Given some of the problems in the accuracy of conveying
meaning with measures of central tendency, measures of
dispersion are an important adjunct in any description of the
data. Measures of dispersion are used to indicate how widely the
data is spread and how evenly the data is spread. In other words,
how far from the central point is the data dispersed?
There are three main measures of dispersion: the range, fractiles
and standard deviation.
The range
This is the simplest, and a very effective, way of describing the
spread of the data. To calculate the range:
· Substract the minimum value in the distribution from the
maximum value.
Although effective, the range can still be affected by the value
of any outliers. In consequence it can give a misleading
impression of the spread of the data. This is why is it important
to include a note of the highest and lowest score in your written
presentation of data.
Example: Calculate the range from the following:
3 4 7 11 12 17 17 47
Fractiles
To take account of the spread of values across the whole range,
fractiles (eg quartiles/quarters, deciles/tenths,
percentiles/hundredths) are used. These divide the range into
smaller, equidistant ranges. Fractiles are used with median
values. To calculate:
1. Subdivide the range into equal parts (eg quartiles, deciles,
percentiles)
2. Find the median (mid point) value;
3. Working from the median point divide your data into the
relevant fractiles.
Fractiles can eliminate the high and low values that affect
measures of central tendency. For example, by focusing on the
cases that fall between the second and third quartile reasearchers
know that they are dealing with the half of the values that fall in
the middle. In addition it allows the comparison of values
between fractiles. For example, the top ten percent of earners
can be compared with the bottom.
Example: The following is income data of social workers.
Divide the data into quartiles. Find the median that occurs in
each quartile. Find the median that occurs between the second
and third quartile. How would you present this data? What
would you say about the validity of these data?
Income per annum (thousands):
15 16 17 21 22 27 27 47
Standard Deviation (SD)
The standard deviation is used with the arithmetic mean. The
standard deviation uses all the values in the range to calculate
the spread of the data. It is a measure of the distance of the
scores from your mean. The larger the standard deviation the
more spread out the range is. To calculate:
1. Find the mean
2. Subtract the mean from all your values
3. Square all the results (to turn your minuses into pluses)
4. Add all these `squared numbers' together
5. Divide this by the number of your values minus one
6. Find the square root of this
· The standard deviation can be used for further statistical
analysis
· Because of this standard deviation is an immensely important
aspect of social research
· The standard deviation can only be used with interval and ratio
data. It is meaningless when used with nominal and even ordinal
data.
Exercise: Find the standard deviation of the following:
•
1 4 7 11 12 17 17 47
CORRELATION
Correlation
How closely are two variables connected? This question is
answered in statistical terms with correlation. For example, do
the students who spend the most time studying achieve the
highest marks? Do those who spend least time studying get the
lowest marks? These question are asking us to compare two
variables: study time and examination performance. We are
asking to what extent is there a relationship between these two
variables.
If the answer was that that those who spend most time studying
do achieve the highest marks we would say that there is a
positive correlation between the two variables. In other words
we would be saying that as the score increases on one variable it
also increases on the other variable. In addition, if those who
study least achieve the lowest marks, we would also say that
there is a positive correlation between the two variables.
However, if we found that the more students spent studying the
lower their marks, this would be described as a negative
correlation. There is, for example, a negative correlation
between the variables of smoking and health. The more a person
smokes the less healthy that person is likely to be. If there is no
relation between two variables then we would say that the
variables are uncorrelated. For example, if the hypothesis was
that wearing jeans improved exam scores and the results
suggested that some students who wore jeans had high scores
and some who wore jeans had low scores, some students who
did not wear jeans had high scores and some who did not wear
jeans had low scores the results are likely to show no
correlation.
To calculate correlation one plots the scores on a scatter
diagram. This requires you to plot the scores of the two
variables along the axes of a graph and mark the results. If a
straight line can be drawn there is a correlation. The direction of
the lines indicates whether this is a positive (up) correlation or a
negative (down) correlation.
The two most commonly used correlation statistics are
Spearman's rank correlation coefficient that works for ordinal
data and Pearsons's product moment correlation coefficient that
works for interval and ratio data.
When reading statistical research you are likely to find the
following signs:
· +1 this equals a perfect positive correlation (as one variable
goes up so does the other)
· 0 this means there is no relationship between the variables
· -1 this equals a perfect negative correlation (as one variable
goes up the other goes down)
· In practice any correlation coefficient between 0.3 (weak) and
0.7 (strong) suggests a reasonable correlation.
Example: Do the following data indicate a correlation?
Student Study Time Examination Mark
1 40 58
2 43 73
3 18 56
4 10 47
5 25 58
6 33 54
7 27 45
8 17 32
9 30 68
10 47 69
(from Hinton, 1995)
CORRELATION AND CAUSATION
CORRELATION DOES NOT MEAN CAUSATION
If two things go together it is easy to assume that they are
causally related in some way. Is this the case? Even if the
thickness of a caterpillar's coat correlates closely with the
severity of the winter weather, can we conclude that caterpillars
cause bad weather?
Three criteria are required to achieve causality in statistical
research:
· Covariation
· Directionality
· Elimination of extraneous variables
Covariation
To conclude that two variables are causally related they need to
covary or correlate. If one variable causes the other then
changes in the values of one variable should be associated with
changes in the values of the other. This is, of course, the
definition of correlation.
Directionality
To infer that two variables are causally related we much show
that the presumed cause precedes the presumed effect in time.
However in most correlational research both variables are
measured at the same time. There is therefore no way to
determine the direction of causality. Has X causes Y or Y
caused X?
Elimination of Extraneous variables
The third criterion for inferring causality is that all extraneous
factors that might influence the relationship between the two
variables are eliminated. Correlational research never satisfies
this requirement completed. Two variables may be correlated
not because they are causally related to one another but because
they are both related to a third variables. For example, does
loneliness cause depression? Maybe but a third variable - the
quality of a person's social network - may reduce both loneliness
and depression.
Example: Does smoking cause cancer?
There is a wealth of research that suggests a strong correlation
between smoking and cancer. Does smoking cause cancer?
[adapted from Leary, 1991]
USING OFFICIAL DATA SETS
There are a number of important, and useful, data sets collected
by government and which can be used for secondary analyses.
These include:
· Census of Employment
· Census of Population
· Labour Force Survey
· General Household Survey
· Family Expenditure Survey
The annual publication Social Trends is a useful source for
those who are seeking some simple statistics. Social Trends
compiles its analyses from these data sets. In addition, the
ESRC keeps data archives of both quantitative and qualitative
research that can be consulted.
Care should be taken in the use of statistics however. For
example, in a discussion of poverty statistics, Townsend notes
how successive governments in the UK have chosen to avoid
using the term `poverty'. As he further notes (1996: 26):
Statistics don't fall out of the skies. Like words - of which they
are of course an extension - they are constructed by human
beings influenced by culture and the predispositions and
governing ideas of the organisations and groups within which
people work. Statistical methodologies are not timeless
creations. They are the current expression of society's attempts
to interpret, represent and analyse information about economic
and social (and other) conditions. As the years pass they change
- not just because there may be technical advances but because
professional, cultural, political and technical conventions
change in terms of retreat as well as advance ... [Thus] Every
student of social science ... needs to be grounded in how
information about social conditions is acquired. Statistics form
a substantial part of such information. Acquiring information is
much more than looking up handbooks of statistics. We have to
become self-conscious about the process of selection.
Levitas and Guy (1996) contextualise these concerns in terms of
the following:
There are developments which may make official data more
easily accessible to academic experts [on-line access]. They do
not make data more easily available to the public in the interests
of informed political debate. Moreover, the (relative) ease of
conducting secondary analysis carries the danger of forgetting
that the concepts used in any research derive from the questions
and interests of its original intentions. The extent to which
secondary analysis can bend data sets to the service of
sometimes quite different agendas is necessarily limited. (p 3)
...The debates ... show that the insistence on the neutrality and
objectivity of facts still dominates discussion of official statistics
and their production. The presentation of statistics in particular
ways for political ends, and the abolition of inconvenient
measures, continue. It is understandable that professional
statisticians should try to counter this by appeals to objectivity.
But it is also abundantly clear that the definitions used in
official statistics still produce measures which embody the
interests of the state rather than of citizens. It is therefore only
with the utmost care that such data can be interpreted for
democratic purposes. (p 6)
The edited text by Levitas and Guy (1996) outlines the kinds of
data sets that are available. It also contains discussions of the
use, and misuse, of government statistics in the following areas:
poverty, unemployment, social class, health, safety at work,
working women, ethnicity, disability and crime. Another useful
text is that of Stanley L (Ed) (1990) Feminist Praxis, London,
Routledge. Amongst the range of issues discussed, this contains
discussions on the ways in which statistics collected on the
homeless are `compromised' by the processes of turning raw
data into statistical information. A chapter by Liz Stanley (A
Referral Was Made) discusses the politics of objectivity
influences the presentation of a social service's case.
USING OFFICIAL DATA SETS
EXERCISE
1. How would you interpret the following statement?
"Statistics on patterns of household disposable income are
provided in Households below Average Income reports ... The
best response to low household income is to sustain economic
recovery and to assist those in greatest need" (Reported in
Townsend, 1996: 27-28)
2. How would you interpret the following conversation?
Ms Corston: Is the Prime Minister aware that Social Trends
1994, a Government publication, reveals that as a direct
consequence of Tory Government policy since 1979 the average
disposable income of the richest 20 per cent of households has
increased by £6,000 a year while the 20 per cent of households
at the bottom of the income scale have had their average
disposable income cut by £3,000 a year? Does that reveal the
hypocrisy of the Prime Minister's professed commitment to
creating a nation at ease with itself?
The Prime Minister: The hon. Lady [Ms Corston] was being
selective in what she said - [Interruption]. She was selective
from the report. The net disposable income of people at all
ranges of income has increased and the proportion of total tax
paid by those on top incomes has increased, not been reduced.
(Reported in Townsend, 1996: 40)