Chapter 10
Clarifying Measurement and Data
Collection in Quantitative Research
Copyright © 2011 by Saunders, an imprint of Elsevier Inc.
1
Concepts of Measurement Theory





Directness of measurement
Measurement error
Level of measurement
Reliability
Validity
Copyright © 2011 by Saunders, an imprint of Elsevier Inc.
2
Directness of Measurement

Directness of measurement


Direct measures (concrete things such as oxygen
saturation, temperature, weight)
Indirect measures (abstract concepts such as
pain, depression, coping, self-care, and selfesteem)
Copyright © 2011 by Saunders, an imprint of Elsevier Inc.
3
Measurement Error

Difference between the true measure and
what is actually measured

Systematic error: the variation in measurement is
in the same direction
 Random error: the difference is without pattern
Copyright © 2011 by Saunders, an imprint of Elsevier Inc.
4
Levels of Measurement




Nominal
Ordinal
Interval
Ratio
Copyright © 2011 by Saunders, an imprint of Elsevier Inc.
5
Nominal-Scale Measurement




Lowest of the four levels of measurement
Categories that are not more or less, but are
different from one another in some way
Mutually exclusive and exhaustive categories
Named categories
Copyright © 2011 by Saunders, an imprint of Elsevier Inc.
6
Example of Nominal Data

Gender


1 = Male
2 = Female
Copyright © 2011 by Saunders, an imprint of Elsevier Inc.
7
Ordinal-Scale Measurement


Order/ranking imposed on categories
Numbers must preserve order



1 = Tallest
2 = Next tallest
3 = Third tallest
Copyright © 2011 by Saunders, an imprint of Elsevier Inc.
8
Interval-Scale Measurement



Numerical distances between intervals
Absence of a zero point
Likert scale scores





1 = Strongly disagree
2 = Disagree
3 = Neutral
4 = Agree
5 = Strongly agree
Copyright © 2011 by Saunders, an imprint of Elsevier Inc.
9
Ratio-Scale Measurement



Highest for measurement
Continuum of values
Absolute zero point
Copyright © 2011 by Saunders, an imprint of Elsevier Inc.
10
Example of Ratio Data

Test scores



1 = Lowest third percentile
2 = Middle third percentile
3 = Top third percentile
Copyright © 2011 by Saunders, an imprint of Elsevier Inc.
11
Reference of Measurement


Norm-referenced testing
Tests performance standards that have been
carefully developed over years with large,
representative samples using a standardized
test with extensive reliability and validity
Copyright © 2011 by Saunders, an imprint of Elsevier Inc.
12
Reference of Measurement (cont’d)

Criterion-referenced testing

Comparison of a subject’s score with a criterion of
achievement that includes the definition of target
behaviors
• When behaviors are mastered, the subject is considered
proficient in the behaviors.
Copyright © 2011 by Saunders, an imprint of Elsevier Inc.
13
Reliability Testing




Concerned with how consistently the
measurement technique measures the
concept of interest
Needs dependability, consistency, accuracy
and comparability
If expressed as a correlation coefficient, 1.00
is perfect reliability, whereas 0.00 is no
reliability.
The lowest acceptable coefficient for a welldeveloped measurement tool is 0.80.
Copyright © 2011 by Saunders, an imprint of Elsevier Inc.
14
Types of Reliability



Stability: Concerned with the consistency of
repeated measures or test-retest reliability
Equivalence: Focused on comparing two
versions of the same instrument (alternate
forms reliability) or two observers (interrater
reliability) measuring the same event
Homogeneity: Addresses the correlation of
various items within the instrument or internal
consistency; determined by split-half reliability
or Cronbach’s alpha coefficient
Copyright © 2011 by Saunders, an imprint of Elsevier Inc.
15
Interrater Reliability

Consistency in raters

% = number of behaviors performed/total number
of behaviors
Copyright © 2011 by Saunders, an imprint of Elsevier Inc.
16
Critiquing for Reliability
What reliability information is provided?
Does the author include reports of the
validity of the instrument from previous
studies?
3. Did the author perform pilot studies to
examine the validity of the instrument?
4. Did the researcher report use of data from
the present study to examine instrument
validity in the discussion section of the
report?
1.
2.
Copyright © 2011 by Saunders, an imprint of Elsevier Inc.
17
What Is Validity?

It is the extent to which an instrument reflects
the concept being examined.
Copyright © 2011 by Saunders, an imprint of Elsevier Inc.
18
Physiological Measures





Accuracy
Selectivity
Precision
Sensitivity
Sources of error
Copyright © 2011 by Saunders, an imprint of Elsevier Inc.
19
Critiquing Methods of Measurement





Were the measurement strategies clearly
identified and described?
What level of measurement was achieved by
each instrument?
Was the reliability of each measurement
method adequate?
Was the validity of each measurement
method adequate?
If physiological instruments were used, were
they accurate and precise?
Copyright © 2011 by Saunders, an imprint of Elsevier Inc.
20
Measurement Strategies in Nursing



Physiological measurements
Observational measurements
Interviews
Copyright © 2011 by Saunders, an imprint of Elsevier Inc.
21
Examples of Physiological Measures

Physical measurement methods



ECG, BP, I&O
SVO2, pulse oximetry
Microbiological



Smears
Cultures, number of CFU
Sensitivities
Copyright © 2011 by Saunders, an imprint of Elsevier Inc.
22
Observational Measurements


Unstructured observations
Structured observations



Category systems
Checklists
Rating scales
Copyright © 2011 by Saunders, an imprint of Elsevier Inc.
23
Critiquing Observational
Measurement
Is the object of observation clearly identified
and defined?
2. Is interrater reliability described?
3. Are the techniques for recording
observations described?
1.
Copyright © 2011 by Saunders, an imprint of Elsevier Inc.
24
Types of Interviews: Unstructured

Uses broad questions


Describe for me your experience with…
Role of interviewer is to encourage continued
discussion
Copyright © 2011 by Saunders, an imprint of Elsevier Inc.
25
Examples of Unstructured or OpenEnded Interview Questions



Tell me about...
What has been your experience with...
What was it like to hear you have cancer?
Copyright © 2011 by Saunders, an imprint of Elsevier Inc.
26
Types of Interviews: Structured

Structured interviews






Describing interview questions
Pretesting the interview protocol
Training interviewers
Preparing for an interview
Probing
Recording interview data
Copyright © 2011 by Saunders, an imprint of Elsevier Inc.
27
Closed-Ended Interview Questions



Structured
Response alternatives fixed
Which would you rather do, x or y?
Copyright © 2011 by Saunders, an imprint of Elsevier Inc.
28
Critiquing Interview Methods
Do the interview questions address concerns
expressed in the research problem?
2. Are the interview questions relevant for the
research purpose and objectives, questions,
or hypotheses?
3. Does the design of the questions tend to bias
subjects’ responses?
4. Does the sequence of questions tend to bias
subjects’ responses?
1.
Copyright © 2011 by Saunders, an imprint of Elsevier Inc.
29
Focus Groups





Used to study qualitative issues
Obtain participants’ perceptions of narrow
subject in a group interview session
Give group a feeling of “safety in numbers”
Nonverbal approaches are included.
Discussion helps to provide depth of data.
Copyright © 2011 by Saunders, an imprint of Elsevier Inc.
30
Focus Group Considerations




May sort participants into smaller groups with
common characteristics: segmentation
Need to select an effective moderator to keep
discussion on track
The setting should be relaxed and
comfortable.
High-quality tape recordings should be made.
Copyright © 2011 by Saunders, an imprint of Elsevier Inc.
31
Critiquing the Use of Focus Groups
What was the aim of the focus group?
Was the group size appropriate for the focus
group method?
3. Was group sufficiently homogeneous to
speak candidly?
4. Was moderator successful in keeping
discussion focused?
1.
2.
Copyright © 2011 by Saunders, an imprint of Elsevier Inc.
32
Critiquing the Use of Focus Groups
(cont’d)
Was the aim of the focus group achieved?
Did conclusions appear to be
representative?
7. Were minority positions identified and
explored?
5.
6.
Copyright © 2011 by Saunders, an imprint of Elsevier Inc.
33
Measurement Strategies





Questionnaires
Scales
Q methodology
Diaries
Delphi technique
Copyright © 2011 by Saunders, an imprint of Elsevier Inc.
34
Questionnaires

Administration



In person/on phone
Self-administered
Mail
Copyright © 2011 by Saunders, an imprint of Elsevier Inc.
35
Scales




Rating scales
The Likert scale
Semantic differential scales
Visual analog scales
Copyright © 2011 by Saunders, an imprint of Elsevier Inc.
36
Example of Items in a Likert Scale
Copyright © 2011 by Saunders, an imprint of Elsevier Inc.
37
Example of Visual Analog Scale
Copyright © 2011 by Saunders, an imprint of Elsevier Inc.
38
Critiquing a Scale
Is the instrument clearly described?
Are the techniques that were used to
administer and score the scale provided?
3. Is information about validity and reliability of
the scale described from previous studies?
4. Is information about validity and reliability of
the scale described for the present sample?
5. If the scale was developed for the study, was
the instrument development process
described?
1.
2.
Copyright © 2011 by Saunders, an imprint of Elsevier Inc.
39
Critiquing the Data Collection Process




Was data collection process clearly
described?
Was data collection conducted in a consistent
way?
Were research controls maintained?
If data collectors were used, were they
adequately trained?
Copyright © 2011 by Saunders, an imprint of Elsevier Inc.
40
Serendipity


The accidental discovery of something useful
or valuable.
Can lead to new insights
Copyright © 2011 by Saunders, an imprint of Elsevier Inc.
41