Introduction to Quantitative research and data management
Robin Beaumont robin@organplayers.co.uk
02/01/2012
Quantitative Research
Contents
1.
Quantitative research
2.
Theory, Methods and Tools
2.1
Methods
2.2
Operationalisation
2.3
Sample size
2.4
Tools
8
Data
11
3.
Exercise 4.
3.1
7
8
11
11
12
The Social Scientists View of Data 13
Exercise 6.
13
Exercise 7.
14
3.3
5
5
The Statisticians View of Data
Exercise 5.
3.2
9
Concepts
Exercise 8.
14
15
4.
Hierarchy of datatypes
5.
Ranking Data
16
17
5.1
Magnitude and Ranking
5.2
Rating Scales
17
17
6.
Collecting and organising your data
7.
Multiple Choice Questions (MCQs) 20
8.
Summary
21
9.
References
22
11.
FAQs (Frequently asked Questions)
19
24
11.1 If you repeat the same research what does it mean? 24
11.2 Qualitative and quantitative research ask different questions - wrong 24
11.3 Qualitative research does not use hypotheses - wrong
24
11.4 Qualitative research is high in validity whereas quantitative research is high in reliability - wrong 25
11.5 You can combine the two philosophical approaches - not really
11.6 There is one hierarchy of Evidence - no
25
11.7 The basis of knowledge is belief or the scientific method!
27
11.8 Quantum theory validates the world 2 viewpoint - no!
27
12.
References
28
Robin Beaumont robin@organplayers.co.uk Document1
page 2
25
Quantitative Research
1. The elements of Quantitative research
There are many ways of carrying out research and depending upon your discipline the methods and tools you use will
vary greatly. In the scientific disciplines a quantitative approach is usually adopted (see the appendix to this chapter for a
description of the philosophical aspects to this approach.)
All quantitative research has the following elements:
Experimenter / Investigator / Researcher
Subject / Client / Volunteer / Cohort / Group / Sample
Data (Primary or Secondary data)
Measuring Instrument (e.g. thermometer, ergometer, ECG, questionnaire)
In addition the following may also be present:
Treatment(s) / Intervention / Test group(s)
Controls / placebo / comparator
Depending upon the degree of rigor required the above basic elements are developed in differing ways, for example
knowing the effectiveness of the treatment, having based our judgement on historical results, will dictate the number of
subjects required in subsequent studies to obtain valid results.
An actual example will help demonstrate some of these elements.
Robin Beaumont robin@organplayers.co.uk Document1
page 3
Quantitative Research
The UK Medical Research Council (MRC) randomised trial of Streptomycin for TB
1949-1950
Streptomycin was a antibiotic which demonstrated some effectiveness in the
treatment of TB in a small group of patients. Subsequently the committee
decided that there should be a series of multi-centre trials at tuberculosis units
throughout the UK. The initial trials involved patients with the most serious
forms of the disease - miliary and meningitic (both previously almost uniformly
fatal), and very advanced pulmonary tuberculosis (with a high but not uniform
mortality). Any recovery of patients with miliary or meningitic tuberculosis
proved convincingly the worth of the new drug in those conditions (MRC 1948b).
In contrast, patients often recovered from pulmonary tuberculosis, even from
very advanced disease.
It was decided to limit the patients participating in the trial to those aged
between 15 and 30 with "acute progressive bilateral pulmonary tuberculosis of
presumably recent origin, bacteriologically proved ...". Both the streptomycin
and control group would receive the standard treatment for this type of disease bed rest. As there were more patients with pulmonary tuberculosis than there
were hospital or sanatorium beds to accommodate them, those allocated to bed
rest alone in the trial received priority for admission. If streptomycin proved
valuable these patients would receive it later, when supplies improved.
Meanwhile, they would avoid any unknown ill effects of the new drug.
When a consultant physician identified a potentially eligible patient, the patient’s
details were sent to Marc Daniels at the national coordinating centre for the
trial. If the patient was judged to meet the eligibility criteria, admission was
arranged to the next available hospital bed in the nearest participating centre.
Each gender in each centre was allotted a numbered series of envelopes, bearing
only the name of the hospital. Each envelope contained a card indicating
‘S(treptomycin)’ or ‘C(ontrol)’. The numerical order of the envelopes was based
on a series of random numbers. When a patient was approved for the trial the
next envelope for that centre and for gender was opened. Streptomycin and
control patients were usually admitted to different wards but otherwise treated
exactly the same.
Neither group of patients knew that they were in a trial, which remained
confidential throughout its 15 month duration. Progress was assessed with
monthly chest X-rays, graded by three specialists who remained ignorant of
(blind to) the identities of the allocation of patients to streptomycin with bed
rest or bed rest alone. Any difference of opinion, usually slight, was resolved by
discussion. Monthly direct smear and culture of sputum was also reported by
bacteriologists who also remained blind to the treatment group. Fever, weight
and sedimentation rates were also recorded regularly.
The results of the trial showed that streptomycin was helpful. During the first 6
months after admission to the study, there were 4 deaths among 55 patients
who had been allocated streptomycin, compared with 15 among 52 patients
allocated to bed rest alone, and this difference was reflected in radiological and
other improvements. During the subsequent 6 months, the radiological and
mortality differences were less marked (there were 8 more deaths in the
streptomycin group and 9 more in the groups treated with bed rest alone).
from: http://www.jameslindlibrary.org/illustrating/articles/the-mrc-randomizedtrial-of-streptomycin-and-its-legacy-a-view
Crofton J (2004). The MRC randomized trial of streptomycin and its legacy: a
view from the clinical front line. © John Crofton, 13 Spylaw Bank Road, Colinton,
Edinburgh EH13 0JW. Email: eapretty@breathemail.net.
Robin Beaumont robin@organplayers.co.uk Document1
page 4
Quantitative Research
2. Theory, Methods and Tools
The research method (the process), the Tools used and the philosophical
foundations (World View) are distinct but related aspects of any
quantitative research so let's consider each of these separately.
Theory
Methods
Tools
2.1 Methods
The overriding aim of quantitative research is to achieve the highest possible level of Objectivity, which is achieved using
the following:
achieved by minimising the dangerous possibility of by using
Blinding techniques subject /researcher interaction along with the following methods:
standardisation would be the aimed for and achieved by appropriate Sampling, Randomisation, Grouping and
scale development (all forming part of the research protocol). You would aim for a method that would allow
replication.
Comparisons - the design would usually include one or more comparator group(s). Such comparator groups are
usually real that is a control, placebo or alternative intervention(s) but might be theoretical that is statistically
defined.
Measurement - taking measurement would allow the development of hypotheses estimating the values of
possible variables (relating to the objective real world). You would define objective valid reliable measures by a
process of operationalisation producing validated standardised scales the result of which would be data
consisting of numbers. A specific sample size
will be stated to ensure a level of certainty for
any measure obtained. The data would be
interpreted using formal statistical theory
involving probability theory and making
inferences from your sample to a defined
population (the process of generalisation).
Interpretation - The possibility of Individual
interpretation of the results would be
minimised. Others following the study protocol
and analysing the data would come to the
same conclusions.
Documentation - you would produce a
protocol defining the process, again to aid
Robin Beaumont robin@organplayers.co.uk Document1
page 5
Quantitative Research
replication, and also report your results in a standard way using validated techniques and layout.
Another way of looking at the above description is to consider the world view 1 gold standard research approach, the RCT
(Randomised Control Trial), which can be expressed in the form of a flow diagram (Schulz, Altman & Moher 2010)
Sampling - we attempt to ensure that our sample is a representative sample (usually random) from some theoretical
population we are interested in generalising our findings to by using mathematical procedures (inferential statistics).
In quantitative research a representative sample is essential for a large number of statistical procedures to produce valid
results and if it is not the results might simply be garbage. Depending upon the research design various eligibility criteria
might be applied to allow more control (i.e. a more homogenous sample), for example in the previous discussion
concerning power the researcher selected, newly diagnosed hypertensive males (aged between 50 to 55 yrs).
Random allocation - A second level of control is achieved by Randomisation which usually means random allocation to a
particular group. This ensures that each subject entering the trial has equal chances of being in any of the treatment
groups. One group may be a placebo or control group which is another important aspect as it allows the measurement or
any effect/change over and about that brought about by any specific effects the other groups are subjected to. For an
excellent introduction to the placebo effect I recommend you read Ben Goldacres' book Bad science which includes a
whole chapter on the subject.
Where it is not possible to randomise (often due to spurious ethical reasons) the alternative might be to use some type of
matching where subjects in the intervention group are matched on important characteristics that have been shown to
effect the outcome either prospectively (case control) or retrospectively (historical control). Another alternative in some
situations is a crossover design where the subjects receive multiple treatments.
The above simple design can be adapted and expanded in numerous imaginative ways, cluster designs (where the
randomisation is at the group level) and stepped wedge designs to name but a few.
Blinding - Blinding in this context means that the person does not know which group they below to and is another
important aspect of many trials. Blinding can be at one of three levels, the lowest level is that of the subject only not
knowing, the second level is that of the subject and immediate researcher not knowing and the third level is where the
data analysis is also carried out blind.
Where it is necessary to make compromises regarding randomisation and/or blinding the quantitative researcher has at
their disposal alternative designs such as the clinical controlled trial and the pragmatic controlled trial. If it is not possible
to apply an intervention of some sort then a observational study can be undertaken where group(s) are usually
repe
atedl
y
meas
ured
over
time.
Hier
archi
es
and
netw
orks
of
data
- it is possible to analyse complex structures such as patients within wards within hospitals within districts and
ascertaining the effects at each level using advanced statistical techniques (i.e. multilevel modelling or other similar
techniques).
The mindmap opposite shows some of the possible methods used in quantitative research.
Robin Beaumont robin@organplayers.co.uk Document1
page 6
Quantitative Research
Exercise 1.
I have attempted to only give you a very brief introduction to methods used by quantitative researchers to find out more
please glance over Gerard E. Dallal's excellent article entitles Some Aspects of Study Design at
http://www.jerrydallal.com/LHSP/STUDY.HTM
I have mentioned two very important aspects above operationalisation and sample calculations. We will now take a quick
look at each.
2.2 Operationalisation
Operationalisation is the process of taking a concept and making it into one or more measurements. Some concepts are
uni-dimensional that is only require one scale to adequately measure them such as temperature or height while others
are more complex, being multidimensional, such as intelligence, empathy, anxiety and happiness often requiring a battery
of measures along different dimensions to adequately measure and capture them. From a world one viewpoint this
process of scale development and validation is one of the great achievements of quantitative research.
.......................................................
Robin Beaumont robin@organplayers.co.uk Document1
page 7
Quantitative Research
2.3 Sample size
Sample size considerations are vitally important in quantitative research as with too smaller sample the measure you
obtain will vary too widely to produce and useful results and too larger sample may be economically wasteful. Sample
size requirements are calculated by measuring a statistical concept call power, where we usually aim for a power level of
at least .8
Let's consider an example. A fictitious researcher wishes to investigate the effects of a new blood pressure reducing drug
which has shown in a previous study with male Scottish, newly diagnosed hypertensive subjects (age range 50 to 55 yrs)
to reduce the diastolic blood pressure by 10 mmHg (SD=6.5) the current researcher wishes to see if it has a similar effect
upon a similar sample in London she carries out the necessary calculations (using the Gpower program) and obtains the
graph shown below. We can see that to achieve a power of at least .8 she needs to recruit a minimum sample of 15
subjects (i.e. 8 in the treatment group
and 8 controls) furthermore given
this information it is pointless
increasing the sample size greatly as
information
redundancy
would
result.
While this is a fictitious example with
possibly values to suggest a smaller
sample size that would be required in
reality it is important as it
demonstrates that it is not necessary
to have large samples for quantitative
research. In fact one of the main
areas of statistical research in the
20th century was small sample
theory notably by William Gossett who developed the t tests specifically for small sample sizes.
2.4 Tools
The questionnaire is one of the most common tools used in research and the quantitative researcher develops and uses it
in a specific way, to provide numeric data, for example:
Questionnaires with pre-defined responses that could be coded numerically. I.e. Likert scales etc.
Text or 'soft' data would be translated (i.e. coded) to numeric values. For example a sentence might be coded as 1 to 10
on an aggression scale. Similarly diary entries might be coded in particular ways.
Instruments such as questionnaires and other measuring devices would be reliable with the same input value (regardless
of the researcher) producing the same result (code) each time.
Robin Beaumont robin@organplayers.co.uk Document1
page 8
Quantitative Research
3. Quantitative research
When you carry out any quantitative research the you are taking on a particular world viewpoint, often called a "World
view I" this viewpoint asserts the belief that there exists a real objective reality. This idea goes back to Plato (truth by
reflective thought) and Aristotle (knowledge through observation and classification). This is the traditional, taken-forgranted view of the world embodied in the RCT (Randomised Control Trial) design that is the bedrock of all medical
research.
The following table adapted from Littlejohn (p13 5th Ed.) lists the five premises that embodies this World View I
Premise/assumption Description
Synchrony
Stability over time exists in contrast to change (diachrony). It is therefore valid to develop Causal
explanations etc.
Objective measures It is possible to objectively measure an independent reality.
Independent reality There does exists a single reality. The reality is objective and not value-laden. Our experiences
are just reflections/interpretations of it. Because our perceptions are merely reflections of this
reality we should mistrust concepts such as “subjectivity”, “consciousness” and creative
reflection in helping to understand this reality.
Dualism
Objects (i.e. the world) and symbols (i.e. language) are separate. Language is just a tool for
description, and the world would exist without it.
Correspondence
Language corresponds to reality (but provides an imperfect reflection of it).
Most people will think the above aspects represent the truth and are just common sense. The important thing to realise is
that they are basically a list of
Assumptions
These assumptions which you could call beliefs imply commitment, and those of you who subscribe to the above
beliefs/assumptions, will have arguments to validate them and feel threatened when these are attacked, probably
psychologists would call it cognitive dissonance.
Exercise 2.
a) How strong would you say your desire is to defend the above viewpoint?
b) What type of arguments would you use to defend them?
I bet you had strong views and used arguments of a 'scientific' / logical nature to defend them.
What is important to realise that logical / scientific arguments depend upon the above assumptions themselves to work.
Take for example the argument you may have for the above being the true state of affairs, this probably included stating
that it has been proved in some way, however proof in this sense depends upon - Synchrony so your argument requires
the assumptions itself!
It is not my desire to discuss the proof or repudiation of the above assumptions, doing so would distract from the
important task of using this world view. If you would like to consider other world views (such as the qualitative approach)
you can find details on my website.
Robin Beaumont robin@organplayers.co.uk Document1
page 9
Quantitative Research
Exercise 3.
By considering the questions asked, knowledge aspects and the method used decide which of the following represents a
researcher with a World I view:
Research
The
asked
1. A researcher wants to find out about patient
satisfaction at a local hospital outpatients
department. She develops a questionnaire consisting
of 20 questions each of which consists of a number
of predefined responses. The questionnaire is given
to 500 random subjects and the data analysed using
a statistical programme (SPSS). The results are
published along with a set of recommendations for
Hospital Outpatients Departments in general.
2. A researcher wants to find out about patient
satisfaction at a local hospital outpatients
department. She spends a few days in the
department asking treated patients to tell her what
they think of the department and their experience.
She records the interviews and then quotes
(sometimes at length) the parts she feels are most
relevant in a document describing individual patients
experiences. The document is structured into a
number of themes that she feels those interviewed
felt to be most important.
3. A researcher wants to find out about patient
satisfaction at a local hospital outpatients
department. She sits quietly observing what is
happening in the department (unknown to both staff
and patients) and records the number of times
patients return to reception to enquire about their
waiting time and the number of patients that leave
before being seen by a medical member of staff.
4. A researcher wants to find out about patient
satisfaction at a local hospital outpatients
department. She develops a web blog and publicises
it within the department. After 6 months, when she
has over 1,000 entries, she pulls out 'themes' she has
identified in the postings and produces a report.
5. A researcher wants to learn more about patient's
experiences at a local hospital outpatients
department. She has a background in nursing and
has arranged with a senior manager to shadow a
nurse for the week on the assumption that she is
interested in seeing the variety of procedures the
nurse's carry out. During her time observing she
makes notes about how the patients and nurses
interact and how a satisfactory outcome is
negotiated between them. She writes up her
experience after a great deal of reflection.
Robin Beaumont robin@organplayers.co.uk Document1
page 10
questions
The
types
of
knowledge
used
along with the
values ascribed to
them.
The method
use
Quantitative Research
4. Data
Data is everywhere. In fact nearly everything one can think of has a set of characteristics ('data') which enables us to
recognise and remember it. Data can vary from the particular colour of a dandelions flower to the resting heart rate of a
subject undergoing an exercise trial. While it is obvious that the colour and heart rate differ radically, one could not
sensibly compare a colour with a heart rate, it is more difficult to define how exactly they do differ but in this document
we will present a classification which does achieve this, so how does data differ, and why is it inappropriate to compare
colour of dandelions to heart rates.
Often a set of Data is collected to form a Dataset. A dataset consists of values for one or more characteristics over a
number of objects (i.e. patients) A simple dataset may consist of peoples shoe size.
If a particular characteristic can take more than one value it is known as a variable e.g. shoe size, height, IQ etc. A
characteristic that can take only one value is known as a constant. However note that a characteristic may be a variable in
one situation (e.g. species for a group of farm animals or age of patients) and a constant in another (e.g. species for a
group of children, age for a group of children in their first year at primary school in the UK). We frequently say that we
have collected a number of variables rather than a dataset.
Exercise 4.
In
the
following
dataset
which
could
be
classified
A
g
e
G
e
n
d
e
r
as
a
variable
and
which
a
constant?
A persons star sign, shoe size and age can therefore all be classified as variable data for a group of people in which there
might exist more than one different value for each variable.
Instinctively
we feel that each of these variables are different
1
m
as we did with comparing the Heart rate to the Dandelions
flower
a
colour
in the previous paragraph. What exactly is the
8
l consider two different ways data has been classified, one
difference between them? To help answer this question we will
e
from the viewpoint of the statistician and the other from the social
scientists perspective.
4.1 The Statisticians View of Data
qualitative
Data
quantitative
1 Nominal
m
a
9
l
e
Ordinal
Discrete
1Continuous
m
a
9
l
Statisticians classify data into two broad types qualitative and quantitative, each of these is further sub-divided into two
e
further types resulting in four basic types; Nominal, Ordinal, Discrete and Continuous. It is important to give you a
warning concerning the use of the term qualitative:
Qualitative data and Qualitative research methods are completely different things
Eye colour is an example of qualitative data. This is because with qualitative data the 'values' are essentially words used
to specify categories. In contrast, with quantitative data,
2 themvalues are numerical attributes which the data possesses
Robin Beaumont robin@organplayers.co.uk Document1
page 110
a
l
Quantitative Research
itself. Qualitative data are often given numerical codes, but any arithmetic done with the codes will be meaningless, as
will become obvious when one returns to the words behind the codes. For example, type of dwelling ('semi-detached',
'detached', 'terrace', etc.) is qualitative (nominal). The 'values' might be coded '1', '2', '3', etc., but although we can add '1'
and '2', adding 'detached' and 'semi-detached' has no meaning, and certainly does not result in 'terrace'! This scale only
allows us to state which particular category a data value belongs in (e.g. red blue or green for eye colour) and count
(enumerate) how many there may be in each category for a particular data set. Nominal data is therefore also often
called categorical or enumerate data. The count for a particular category is often referred to as the frequency.
The term 'ordinal' is used when it is possible to order the various categories to create a scale. For example, the responses
to a question asking 'How often do you have problems getting to sleep?' might be labelled 'every night' , 'most nights',
'some times', 'rarely' and 'never'. The correspondence between these words, although clearly graded in order, is
questionable in terms of the relative distance between each, for example is 'most nights' a similar distance away from
'some times' as 'rarely' is to it in the opposite direction. Much effort is made to make such ordinal data possess a scale
that approximates equal distances such as the common set of responses ('strongly disagree', 'disagree', 'neutral', 'agree',
'strongly agree') to attitudinal questions such as, 'I find statistics boring?'. It should be noted that even when efforts are
made to make the scale have similar intervals it does not make sense to perform mathematical operations on them such
as 'disagree' + neutral = 'agree' etc.
In medicine there are several systems for classifying the extent or stage of cancer. The two most common are the Stage I,
II, III, IV system (in contrast the American Joint Committee on Cancer (AJCC) uses five stages from 0 to IV) and the TNM
(Tumour, Node, metastases) system. These staging systems provide estimates of the stage of disease and chances of
survival.
For
more
information
see
both
http://www.cancerstaging.org/cstage/index.html
and
http://www.oncologychannel.com/coloncancer/staging.shtml
The only valid type of mathematical operation that can be carried out on ordinal data is to order it and count how many
observations exist at each point in the scale. The process of ordering data is called Ranking which we will discuss latter.
Both Nominal and Ordinal data is sometimes just referred to as qualitative data again:
Qualitative data and Qualitative research methods are completely different things
Exercise 5.
Which of the following variables are Nominal and which are Ordinal?
1. List of patron saints (Broadcaster = St. Gabriel; Invalids= St. Stephan; Bee keepers= St. Ambrose etc.)
2. Hair style
3. Learning style (such as 'deep', and 'superficial')
4. Shoe size (1 to 14)
5. Species of Ant
6. The Seven Dwarves (Bashful, Doc, Dopey, Grumpy, Happy, Sleepy, Sneezy)
7. Eye response (No eye opening, Eye opening to pain, Eye opening to speech, Eyes open spontaneously)
8. Sleeve length of glove( Shoulder length, Above elbow, Elbow, Mid-forearm)
9. Ivy League Universities (Brown, Columbia, Cornell, Dartmouth, Harvard, Pennsylvania, Princeton, Yale)
10. Deadly sins (Pride, Greed, Lust, Envy, Gluttony, Anger, Sloth)
11. Star (Zodiac) sign
12. Pencil Hardness
13. Iceberg size( Growler, Bergy bit, Small, medium, Large, Very large)
The other major subdivision of data used by statisticians is that dividing Quantitative data. Quantitative data, also called
numerical data can be either discrete (for example, the number of children in a family) or continuous ( for example, the
height of an experimental subject in cm). What is the difference?
Discrete data can only have values that are separated by impossible values, e.g. you cannot have half a child. Another
example is Shoe, or for that matter most readymade clothes, sizes.
Robin Beaumont robin@organplayers.co.uk Document1
page 12
Quantitative Research
Continuous data can take any value within a range, e.g. While a particular height might be 217 cm it could easily be
217.34 or even 2.17.345635 depending upon the accuracy of the measuring mechanism.
While often people are told that discrete data consists of whole numbers (integers) this is not always the case. Take for
example, the case of the 'number of questions' answered correctly in a test on spelling' . This will be represented by
'whole' numbers, however you can represent the same data as a 'proportion of correct answers', for it is clear that 21
correct answers out of 30 is a discrete value that may nevertheless be represented as 0.7 when it is the proportion of
correct answers that interests us.
While statisticians spend a great deal of time differentiating between discrete and continuous data, for our purposes
continuous data is often treated as discrete data and it does not really cause too many problems.
In contrast to the above classification social scientists classify data in a slightly different way which will be discussed next.
4.2 The Social Scientists View of Data
While social scientists use the Nominal and Ordinal classification they prefer to adopt another type of classification for the
terms discrete and continuous. The third level of measurement they describe is one which possesses those characteristics
described above for ordinal data but in addition has equal sized intervals e.g.: Fahrenheit, Celsius, bank balance. It is
interval measurement data.
The most complex type of scale involves all of the above characteristics and in addition possesses an absolute zero point
e.g. height, weight, distance, Kelvin's. This, ratio measurement type, is the most complex of the measurement scale types.
Do not worry too much if you find it difficult to differentiate between Interval and ratio data as it does not matter too
much, in contrast the important thing is to be able to differentiate between Nominal, Ordinal, and (Interval/Ratio) data.
Since Stevens 1951, suggested this classification several authors have criticised the degree of prominence it has achieved
concerning which types of statistics are considered appropriate for each type (Gaito 1980).
The chart below provides the above information in summary form. An easy way to remember the data types is the word
'Noir'.
Measurement type (e.g.)
Name
Nominal
X
Order
Equal
Intervals
Absolute
zero
Tip: Memorise this chart
and you can't go wrong
(star sign)
Ordinal
(fitness rating scale)
X
X
Interval
(Fahrenheit)
X
X
X
Ratio
(weight)
X
X
X
X
Exercise 6.
Produce a list of about 20 characteristics about yourself and categorise them into the appropriate measurement
scale type.
Robin Beaumont robin@organplayers.co.uk Document1
page 13
Quantitative Research
Exercise 7.
As the Dean of Students for your Medical School, you must prepare each student’s class ranking in clinical work.
The ranking, which will be used for internship and other recommendations, comes from a combination of
grades for clinical clerkships in five departments. Each grade is given an equal weight in the student’s “class
standing.” The five clinical departments express their individuality by using different scales for giving grades.
The scales are as follows:
Internal Medicine: A,B,C,D,E, (with A = highest and E = lowest)
Obstetrics-Gynecology: A+, A, A−, B+, B, B−, C+, C, C−, D, and E
Pediatrics: Numerical examination grade from 100 (perfect) to 0 (terrible)
Psychiatry: Superior, Satisfactory, Fail
Surgery: High honors, Honors, Pass, Fail
How would you combine these non-commensurate scaling systems to form a composite score for class
standing? Taken from Feinstein, 2002 page. 65
4.3 Concepts
Researchers frequently wish to investigate some
abstract concept (e.g. health, patience etc) which
means they need to develop one or more
measures which they may combine in an attempt
to measure it.
Two such examples are intelligence and physical
exhaustion. A standard method of measuring
intelligence is to use the Wechsler Adult
Intelligence Scale (WAIS) intelligence scale to
obtain an intelligence quotient (IQ). The WAIS
consists of 10 measures, which are themselves
grouped together to form sub-scales before being
combined.
Constructs assessed by the MHP-H
Adult Health History (individual ratings of
general health and the
Intelligence_Scale
Similarly the concept of physical exhaustion can be
measured by considering both the Borg scale, and Vo 2Max
reflecting both the psychological sensation and the
physiological manifestation of physical exhaustion. The
process by which a researcher chooses/devises one or more
(also called constructs) to measure some possibly abstract,
concept is known as operationalisation and how well s/he
this often provides the litmus test for the research. There is
much debate concerning the validity of combining or
alternatively keeping separate the various sub scales. As
can imagine there are numerous statistical techniques to
you make a decision.
Obviously the measurement of ‘health’ is a major area of
concern. Many scales have been developed, a typical
Robin Beaumont robin@organplayers.co.uk Document1
page 14
From:
http://en.w
ikipedia.org
/wiki/Wech
sler_Adult_
presence of chronic illness)
1. Severity of typical illness
2. Presence of a chronic illness
3. Impairment due to chronic illness
scales
4. Overall health (over adult life)
5. Recent health (over past six months)
Health Habits (sums of 24 individual ratings
of positive and
negative health habits)
6. Positive habits
7. Negative habits
Health Care Utilization (frequency of utilizing
does
always
you
help
Quantitative Research
examples are the Duke health profile (17 items!) and the Nottingham health profile. Specialist measures have also been
developed such as the Multidimensional [psychological] Health Profile (Karoly, Ruehlman, & Lanyon, 2005). According to
Lanyon, Maxwell & Karoly 2007 “This instrument was designed to alert health care personnel to potential [psychological]
problems that should be addressed in more detail, and consists of two sections: Psychosocial Functioning (MHP-P), and
Health-related [psychological] Functioning (MHP-H).”
Exercise 8.
Consider the concept of fatigue? Do you think many papers have been published discussing how you might
measure it? Do you think there are different varieties?
Within the healthcare setting which conditions do you think might benefit from such a measure?
Do you think when measuring it you would need to develop one or more sub-scales, that is do you think it might
have different aspects?
Please don’t turn the page until you have carried out the above exercise.
Pathological fatigue
Quoting Kittiwatanapaisan 2003
Pathological fatigue, in contrast to normal fatigue, does not subside with rest and is characterized by a
feeling of tiredness before activity, lack of energy to complete tasks, exhaustion after usual activity, or all of
the above. Fatigue has been found to correlate with physical and psychological parameters in patients with
multiple sclerosis, chronic fatigue syndrome, HIV infection, and AIDS (Breitbart, McDonald, Rosenfeld,
Monkman, & Passik, 1998; Ford, Trigwell, & Johnson, 1998; O'Dell, Meighen, & Riggs, 1996; Packer, Foster,
& Brouwer, 1997; Vercoulen et al., 1997; Walker, McGown, Jantos, & Anson, 1997). . . . .
Fatigue has been measured with various instruments, mainly, visual analog scales and questionnaires with
Likert-scale format, including the Checklist of Individual Strength-Fatigue (van der Werf et al., 1998;
Vercoulen et al., 1996, 1997), the Fatigue Severity Scale (Packer, Sauriol, & Brouwer, 1994), the Chalder
Fatigue Scale (Ford et al., 1998), the Fatigue Assessment Inventory (O'Dell et al., 1996), the Piper Fatigue
Scale (Cupler, Otero, Hench, Luciano, & Dalakas, 1996; O'Dell et al., 1996), and the Multidimensional
Assessment of Fatigue (Schwartz, Coulthard-Morris, & Zeng, 1996). However, these instruments are not
specific for measuring fatigue in the MG [Myasthenia Gravis] patient population.
Grohar-Murray, Sears, Hubsky, and Becker (1994) combined and modified two unpublished questionnaires,
which were used to measure fatigue in multiple sclerosis patients, for use with MG [Myasthenia Gravis]
patients. . . . . .
Abstract references:
Breitbart, W., McDonald, M.V., Rosenfeld, B., Monkman, N.D., & Passik, S. (1998). Fatigue in ambulatory
AIDS patients. Journal of Pain and Symptom Management, 15, 159-167.
Chalder, T., Berelowitz, G., Pawlikowska, T., Watts, L., Wessely, S., Wright, D., et al. (1993). Development of
fatigue scale. Journal of Psychosomatic Research, 37, 147-153.
Cupler, E.J., Otero, C., Hench, K., Luciano, C., & Dalakas, M.C. (1996). Acetylcholine receptor antibodies as a
marker of treatable fatigue in HIV-1 infected individuals. Muscle & Nerve, 19, 1186-1188.
Dzurec, L.C., Hoover, P.M., & Fields, J. (2002). Acknowledging unexplained fatigue of tired women. Journal of
Nursing Scholarship, 31(1), 41-46.
Ford, H., Trigwell, P., & Johnson, M. (1998). The nature of fatigue in multiple sclerosis. Journal of
Psychosomatic Research, 45, 33-38
Grohar-Murray, M.E., Becker, A., Ricci, M., Polak, M., & Danehy, S. (1994). Fatigue characteristics, severity,
and impact on the functional status in myasthenia gravis. Unpublished manuscript.
O'Dell, M., Meighen, M., & Riggs, R.V. (1996). Correlates of fatigue in HIV infection prior to AIDS: A pilot
study. Disability and Rehabilitation, 18, 249-254
Robin Beaumont robin@organplayers.co.uk Document1
page 15
Packer, T.L., Foster, D.M., & Brouwer, B. (1997). Fatigue and activity patterns of people with chronic fatigue
A
thoug
h
literat
ure
review
of
variou
s
measu
res of
fatigue
as
used
in
chroni
c
exercis
e
resear
ch can
be
found
in
Puetz,
O'Con
nor,
Dishm
an,
2006.
Quantitative Research
5. Hierarchy of datatypes
It is importance to realise that the various datatypes (or measurement scales if you prefer the term) represent a hierarchy
of complexity. The most complex data it that which has Interval/Ratio characteristics and because of this hierarchy of
complexity if you convert Interval/Ratio data to Ordinal data you will loose some of the information within it. This is
related to the process of Ranking which is discussed in the next section.
Robin Beaumont robin@organplayers.co.uk Document1
page 16
Quantitative Research
6. Ranking Data
The process of ordering data and assigning a numerical value is called Ranking. Let's take an example by considering the
following numbers: 5, 3, 8, 1, 10
Ranking them from smallest to largest and assigning a value to each ‘the rank’ would produce the following result:
Original data
Original
5
5
3
3
2
8
4
1
1
10
5
3
8
1
10
Ranking
Rankings
3
2
4
1
5
What do we do if we have the situation of tied scores (ties) i.e. two, or more, with the same value?
Score
(ordered)
Ran
k
10
1
8
2
5
3
3
5
3
5
3
5
1
7
Example: Consider the following numbers 5, 3, 8, 3, 1, 3, 10
Placing them in order of magnitude: 10, 8, 5, 3, 3, 3, 1, We note that there are three 3s. These are equivalent to the
ranked scores or the 4th, 5th and 6th score. We therefore allocate the average of these ranks (i.e. 4 + 5 + 6 / 3 = 5) to
each of them.
6.1 Magnitude and Ranking
Now considering the following example instead of one set of data consider the two given below. Notice that increasing
the magnitude of the lowest and highest scores has not affect on their rankings. Therefore by ranking our data we have
lost the importance of magnitude in the original dataset.
Originals
Original data 1
5
3
8
1
10
Original data 2
5
Ranking
5
5
3
3
3
2
8
8
4
1
-10
1
10
25
5
3
8
-10
25
6.2 Rating Scales
Robin Beaumont robin@organplayers.co.uk Document1
page 17
Rankings
3
2
4
1
5
Quantitative Research
Rating scales are just a way of obtaining ordinal data by asking subjects to rate from say 1 to 10 a particular response. For
example it might be to ask them 'how
exhausted they are' (known as the Borg
Please mark on the line below how anxious you feel:
scale), or how much pain they have etc.
Not at all
Extremely Psychologists and statisticians argue if
the data collected can be considered
ordinal or does possess the additional
characteristics to be interval or ratio.
0
10
Rating scales are also sometimes
represented graphically as shown
above.
Robin Beaumont robin@organplayers.co.uk Document1
page 18
Quantitative Research
7. Collecting and organising your data
Robin Beaumont robin@organplayers.co.uk Document1
page 19
Quantitative Research
8. Multiple Choice Questions (MCQs)
1. I suggest two reasons why I feel people fall foul at the first hurdle of learning statistics. Which of the following are
they? (two correct choices)
a. 'user friendly' introductions under emphasising basic concepts
b. 'user friendly' introductions incorrectly explaining basic concepts
c. statistics presented as a poorly defined subjective discipline
d. over emphasis on the use of computers
e. statistics presented as a clear cut subject with clearly defined rules
2. Which of the following is an example of nominal data? (one correct choice)
a. Number of people on a course
b. Cancer staging scale
c. List of different species of bird visiting a garden over the past week
d. Popularity rating of UK top ten television programmes
e. Heart rate
3. Which of the following are examples of Interval/Ratio data? (two correct choices)
a. Number of people on a course
b. Cancer staging scale
c. List of different species of bird visiting a garden over the past week
d. Popularity rating of UK top ten television programmes
e. Heart rate
4. Which of the following are examples of Ordinal data? (two correct choices)
Robin Beaumont robin@organplayers.co.uk Document1
page 20
Quantitative Research
a. Number of people on a course
b. Cancer staging scale
c. List of different species of bird visiting a garden over the past week
d. Popularity rating of UK top ten television programmes
e. Heart rate
5. Which of the following is the correct listing of data from the simplest to the most complex? (one correct choice)
a. Nominal -> Ordinal -> Interval -> Transcendental
b. Nominal -> Ordinal -> Interval -> Ratio
c. Qualitative -> Ordinal -> Interval -> Discrete
d. Qualitative -> Ordinal -> Interval -> Ratio
e. Nominal -> Ordinal -> Interval -> Quantitative
6. Which of the following is an incorrect statement about Ranking a dataset? (one correct choice)
a. You can rank any dataset as long it is not Nominal
b. Each value in a dataset should only occur once
c. The process of ranking a dataset involves ordering it and then assigning a 'rank' value to each score from 1 to the
number of scores in the dataset.
d. When ranking a dataset tied scores receive the average of the rank value given to the ties.
e. The result of ranking a dataset means that you lose the effect of magnitude if the data were Interval/Ratio
9. Summary
In this chapter we have looked at data and how it is classified from two perspectives, that of the statistician and also the
Social scientist. We started by discussing the simplest type of data, Nominal data which can only be categorised and
counted. Next came data where it was possible to order the various categories to produce a scale, although the relative
distance between the points was not specified. A large number of examples of this type of Ordinal data were given from
attitudinal question scoring to Cancer staging scores.
The more complex data, termed by statisticians 'Quantitative' data possessed the additional characteristic of magnitude,
and the social scientists termed such data Interval/Ratio. Such data was what most people would traditionally class as
real data possessing a scale with equal intervals.
The process of ranking data was described and the effect that the process had upon a original dataset of Interval/Ratio
data was discussed. The effects of this process highlighted the importance of bearing in mind the hierarchy of data types
Robin Beaumont robin@organplayers.co.uk Document1
page 21
Quantitative Research
(measurement scales) and how possibly valuable information about data may be lost by converting data from a higher
level scale to that of a lower level one.
The chapter ended with a set of Multiple Choice Questions (MCQs) to help you review what you have learnt. As a final
revision exercise you should return to the learning outcomes at the beginning of the before moving on to the next
chapter.
10.
References
Feinstein RA 2002 Principles of Medical Statistics Chapman Hall [Author died at the age of 75 in 2001]
Gaito J 1980 Measurement Scales and Statistics: Resurgence of an Old Misconception. Psychological Bulletin. Vol 87(7),
564-567
Gigerenzer G Swinjtink Z Porter T Daston L Beatty J Kruger L 1989 The empire of chance. Cambridge University Press
Gonick L Smith W 1993 The cartoon guide to Statistics. Harper Resource
Grant S, Aitchison T, Henderson, E Christie J, Zare S, McMurray J, Dargie H 1999 A comparison of the reproducibility and
the sensitivity to change of visual analogue scales, borg scales, and likert scales in normal subjects during submaximal
exercise. doi:10.1378/chest.116.5.1208
Hawkins A Jolliffe F Glickman L 1992 Teaching statistical concepts. Longman
Howell D 1992 Statistical methods for psychologists Duxbury (chapman & hall in UK)
Karoly, P., Ruehlman, L. S., & Lanyon, R. I. 2005 The assessment of adult health care orientations: Development and
preliminary validation of the Multidimensional Health Profile-Health Functioning (MHP-H) in a national sample. Journal of
Clinical Psychology in Medical Settings, 12, 79–91.
Kittiwatanapaisan W, Gauthier D K, Williams A M, Shin J O. 2003 Fatigue in Myasthenia Gravis patients. Journal of
Neuroscience
Nursing.
April,
2003
Retrieved
from:
http://www.entrepreneur.com/tradejournals/article/102271373_4.html on 15/02/2010 17:04
Lanyon R · Barbara M. Maxwell B M, Karoly P, Ruehlman S L. 2007 Concurrent Validity of the Multidimensional Health
Profile—Health Functioning Scales (MHP-H) in the Pre-operative Assessment of Applicants for Gastric Bypass Surgery.
Journal of Clinical Psychology in Medical Settings, 14:41–49.
Mood A M Graybill F A Boes C D 1974 Introduction to the theory of statistics (3rd ed.) McGraw Hill
Puetz, T W; O'Connor, P J; Dishman, R K. 2006 Effects of chronic exercise on feelings of energy and fatigue: A quantitative
synthesis. Psychological Bulletin. Vol 132(6), 66-876
Stevens S S 1951 Mathematics, measurement, and psycho physics. In S S Stevens (Ed.) Handbook of experimental
psychology (pp. 1 - 49) New York Wiley
Robin Beaumont robin@organplayers.co.uk Document1
page 22
Quantitative /Qualitative research fundamental propositions
12. FAQs (Frequently asked Questions)
12.1 If you repeat the same research what does it mean?
In a previous version of this document exercise 4 repeated the research
"2. A researcher wants to find out about patient satisfaction at a local hospital outpatients department. She spends a
few days in the department asking treated patients to tell her what they think of the department and their
experience. She records the interviews and then quotes (sometimes at length) the parts she feels are most relevant in
a document describing individual patients experiences."
The idea behind the two identical pieces of research was to get you thinking about the consequences of the
assumptions of world view 2.
In littlejohn you will read about phenomenology - basically gaining knowledge through personal experience/
interpretation, for example if you want to understand blindness you wouldn't read about it but cover your eyes and
force yourself to be blind for a time. You would then reflect upon it interpreting your personal experience.
Along with this concept comes the idea that it is not possible to generalise as we all have unique experiences which
are constantly developing. In music the first time you hear a chord has a different impact and significance to a
repetition of it, and even more repetitions have yet other effects which the minimalist composers have investigated.
For a researcher who espouses world 2 beliefs the same research repeated offers unique insights each time. This is in
complete opposition to the world I viewpoint where replication / reproducibility is considered a good thing.
So if you hold world I beliefs both research exercises are the same (replication), but if you hold world II beliefs they are
not.
Notice that the difference relates to the researcher not the research method etc.
12.2 Qualitative and quantitative research ask different
questions - wrong
Consider the following from a past student:
"Statement: Questions such as "How many parents would consult their general practitioner when their child has a
rash?" or "What proportion of common cold suffers take over the counter preparations?" clearly need answering
through quantitative methods. However, questions like "Why do parents seek medical opinion when a child has a
rash?" and "What stops people with common cold seeing their general practitioner?" cannot be answered by these
methods. Hence, qualitative techniques have to be employed."
The above shows many misunderstandings. The writer is confusing method adopted with philosophical approach. A
person with qualitative beliefs would apply those techniques in any situation; similarly a researcher with quantitative
beliefs would apply those in any situation. Think of the different and (opposing approaches to measuring cognition from personal biographies to complex experimental designs).
12.3 Qualitative research does not use hypotheses - wrong
Consider the following from a research methods book:
"Quantitative research begins with an idea (hypothesis), which then, through measurement, generates data and, by
deduction, allows a conclusion to be drawn. Qualitative research, in contrast, begins with an intention to explore a
particular area, collects "data" (observations and interviews), and generates ideas and hypotheses from these data
largely through inductive reasoning."
This is not true - Look at possible shadowing exercise (a qualitative technique) - you may decide to enter it with a clean
slate (e.g. the Glazzer and Strauss grounded theory approach ), alternatively, knowing about the particular
environment your entering you may have researched the literature in that area (i.e. women working in engineering
etc) and have already decided to take a feminist lens to the day. Similarly at other times those with a world view 1 set
of beliefs may well start with a hypothesis (as recommended by Fisher and Neymenn Pearson etc) however there are
techniques which encourage the researcher to interact and explore the data such as Structural equation modelling,
longitudinal designs and some newer, beyond significance testing techniques.
So both qualitative and quantitative research approaches may use hypotheses or not.
Robin
Beaumont
robin@organplayers.co.uk
beaumont.co.uk/virtualclassroom/chap5/s5/comm_theories/qual_quan1.pdf Page 24 of 28
http://www.robin-
Quantitative /Qualitative research fundamental propositions
12.4 Qualitative research is high in validity whereas
quantitative research is high in reliability - wrong
Again this is not true - I do not want to get into a discussion about the many verities of validity but once you look into
the literature you can see this is more to do with the process of 'operationalisation' and how this is handled in a
quantitative research design (i.e. world 1 view assumptions).
Why do people make the above mistakes, I think one of the common causes is the table that is often reproduced
below - most of these dichotomies do not exist.
Qualitative versus quantitative research:
Misleading WRONG table
Qualitative
Quantitative
Social theory
Action
Structure
Methods
Observation, interview
Experiment, survey
Question
What is X
How many Xs?
Reasoning
Inductive
Deductive
Sampling method
Theoretical
Statistical
Strength
Validity
Reliability
Basically - a misleading inappropriate table.
12.5 You can combine the two philosophical approaches - not
really
You can combine both qualitative and quantitative research - no, how can you be a catholic and a muslim at the same
time? Look at the table which lays out the propositions of world view I and II in the fundamental proposition
document - they contradict each other!
When you do combine the two you always make a value judgement - for example you use qualitative interviews to
help inform the development of a quantitative questionnaire - so you are espousing world 1 beliefs really. etc.
12.6 There is one hierarchy of Evidence - no
World I viewpoint
Sackett's hierarchy of evidence has provoked much unresolved controversy about the kind of evidence that is actually
most relevant to practice; for example, controlled trials often restrict the kind of patients recruited, whereas cohort
studies better reflect normal patterns of patient management.
The hierarchy of levels of evidence is briefly:
1.
2.
3.
A
Systematic reviews/ meta-analyses
B
RCTs
C
Experimental designs
A
Cohort control studies
B
Case-control studies
A
Consensus conference
B
Expert opinion
C
Observational study
Robin
Beaumont
robin@organplayers.co.uk
beaumont.co.uk/virtualclassroom/chap5/s5/comm_theories/qual_quan1.pdf Page 25 of 28
http://www.robin-
Quantitative /Qualitative research fundamental propositions
D
Other types of study eg. Interview based, local audit
E
Quasi-experimental, qualitative design
4.
Personal communication
Possible World II viewpoint from a phenomenological perspective?
1.
A
Self reflection - autobiographies, collected journals, life histories etc
B
Letters to friends etc
C
Public interviews etc
2.
A
Shared documents
3.
A
Consensus conference
B
....
C
....
D
.....
E
RCTs, Experimental designs
4.
Systematic reviews/ meta-analyses
Basically one table is a mirror reflection of the other!!
Robin
Beaumont
robin@organplayers.co.uk
beaumont.co.uk/virtualclassroom/chap5/s5/comm_theories/qual_quan1.pdf Page 26 of 28
http://www.robin-
Quantitative /Qualitative research fundamental propositions
12.7 The basis of knowledge is belief or the scientific method!
Much of what you read about qualitative and quantitative methods does not taken into account the philosophical
considerations and is not worth the paper it is written on (!) and you will soon be able to critique such superficial
dribble. Obviously your own viewpoint means that you either adhere to one or the other - one is right because you
have proved (i.e. believe in) it. Religion - a good example of a world II viewpoint states that the basis of
knowledge/wisdom is belief and this in essence reflects the world 3 view of the nature of knowledge. For those with a
world one viewpoint the foundation of knowledge is seen as the scientific method offering a very different approach.
The philosophical study of the nature of knowledge is called epistemology so we can say that those with a world 1
viewpoint posses a very different epistemological viewpoint to those espousing a world 1 viewpoint which is
demonstrated above in the two different hierarchy of evidence tables.
12.8 Quantum theory validates the world 2 viewpoint - no!
Much has been written about various descriptions in quantum mechanics to suggest that there is no one single
external reality. Common arguments present such findings as:
Viewing an experiment changes its outcome
A
thing
can
be
in
two
places/states
at
See Schrödinger's cat http://en.wikipedia.org/wiki/Schr%C3%B6dinger%27s_cat
the
same
time.
This is not the case and no respectable scientist would subscribe to the use of these subatomic mathematical
modelling constructs to large atomic structures such as a cat! Two excellent sources of information are; Fashionable
Nonsense by Sokal & Bricmont 1998, and the road to reality by Penrose 2005.
I think the stumbling block is that people see the mathematical model whether it be Newtonian or Quantum Physics
as reality rather than just a possible imprecise model of it, Models only ever offer a lens through which to analyse it
(note the singular it).
Observer
Model
Reality
Exercise 9.
The following abstract is taken from a confused student; consider an appropriate answer, highlighting his main
misunderstandings
"Quantitative studies are where responses are predefined for analysis purposes I understand, and although it does
have limitations I can see the value of it Qualitative research adds depth. Where I have great difficulty is the analysis
of notes taken by an observer of the perceived interactions of others.
Not having been involved with qualitative studies I find it hard to comprehend how such research is undertaken so
that there is a useful outcome. Surely if subjects are made aware that they being observed they will change their
attitudes/ actions and therefore influence the outcome of the study. This could be good or bad. There are number of
factors that could influence this e.g bad mood, dislike of the observer etc. I know that I am influenced when making a
phone call to a bank/ utility service, and I’m told that the call may be recorded for training purposes; I make a effort to
be pleasant. Surely this invalids the research."
Robin
Beaumont
robin@organplayers.co.uk
beaumont.co.uk/virtualclassroom/chap5/s5/comm_theories/qual_quan1.pdf Page 27 of 28
http://www.robin-
Quantitative /Qualitative research fundamental propositions
13. References
Sandelowski M 1995 Sample size in qualitative research. Res Nurs Health. Apr;18(2):179-83.
Schulz KF,Altman DG, Moher D, 2010 CONSORT 2010 Statement: updated guidelines for reporting parallel group
randomised trials. BMJ 2010; 340 doi: 10.1136/bmj.c332 (Published 23 March 2010)
New ones
EBM triangles: http://cardio.nl/ACS/StudiesRichtlijnenProtocollen.html
Robin
Beaumont
robin@organplayers.co.uk
beaumont.co.uk/virtualclassroom/chap5/s5/comm_theories/qual_quan1.pdf Page 28 of 28
http://www.robin-