Principles of Measurement
Lunch & Learn Oct 16, 2013
J Tobon & M Boyle
What is Measurement?
•
the process of assigning numerals to variables
to represent quantities of characteristics
according to certain rules.
•
•
Objective – directly observable (e.g., length)
Subjective – inferred from experience
(e.g., sensations, perceptions, behaviour)
What are some examples of
concepts you have/would like
to measure?
Measurement Steps
1. Identify and define the concept
2. Specify the objective and conditions of
measurement
3. Determine if a useful measure exists
4. Decide whether to use existing measure or
develop a new measure
Use
Develop
Measurement Steps
5. Create a question/item pool and evaluate the
relevance/importance of each item using a
formal judgment approach and trim item pool
6. Choose an approach to scaling
7. Pre-test instrument with target respondents
(interviews, focus groups, pilot data)
8. Conduct reliability and validity studies
Measurement Steps
Reliability and Validity studies
Basic Measurement Concepts
• Development
• Evaluation
Basic Concepts…
What is a Concept?
• A mental representation, an abstract idea, the
invented name for an attribute that we cannot
observe directly but infer from the data of
experience (indicators) e.g. SES, I.Q., anger
• Concepts are usually complex and need to be
represented by multiple indicators
Basic Concepts…
What are the Objectives of Measurement?
1. To discriminate between individuals
2. To evaluate change within individuals
3. To classify individuals
What are the Conditions of Measurement?
Practical considerations associated with
assessment and data collection (cost/burden)
Basic Concepts…
What is a Useful Measure?
1. Adequately represents the concept of interest
2. Satisfies the reliability/validity (empirical)
requirements associated with the objectives of
measurement
3. Meets practical requirements
Basic Concepts…
What is Scaling?
• The process of ‘scaling’ is an attempt to
upgrade measurement – the numerical
representation of attributes (e.g., increasing
reliable variance or the meaning of score
intervals)
Basic Concepts…
What are the Levels of Measurement?
• Nominal, ordinal, interval and ratio
• These levels determine the numerical
operations available in an analysis
• Strength and form of association between
variables will depend on capturing their
underlying nature
Levels of Measurement
Nominal
Ordinal
Interval
1
0 NO SCHOOLING
2
3 SOME PRIMARY
0
3
6 COMPLETED PRIMARY
4
10 SOME SECONDARY
5
13 SECONDARY COMPLETED
6
14 SOME COMMUNITY ETC
7
15 COMPLETED COMMUNITY
1
8
16 SOME UNIVERSITY
9
17 COMPLETED UNIVERSITY
Basic Concepts…
What is an Additive Scale?
•multiple items, same response, added up
Rate your agreement with the
following statements
SA
A
N
D
SD
I am knowledgeable about the signs and symptoms
of student mental health issues
1
2
3
4
5
I am knowledgeable about appropriate actions to
take to support student mental health at school
1
2
3
4
5
I am knowledgeable about legislation related to
mental health issues (confidentiality, consent to
treatment, etc.)
1
2
3
4
5
I am knowledgeable about school system services
and resources for helping students with mental
health issues
1
2
3
4
5
Single Item
Signs & Symptoms
Multiple Items
MH Literacy
Basic Concepts…
What is a Good Additive Scale?
• It will be unidimensional
• It will cover the full range of meaningful values
• It will be scaled to the highest level of
measurement
• It will be reliable and valid for its intended use
Basic Concepts…
What is reliability?
• the extent to which measures provide
similar values on replication
What are the types of reliability?
1. Internal consistency – alpha (item sampling)
2. Test-retest (b/w respondent)
3. Inter-observer (b/w observer)
Importance of Reliability
Levels of reliability constrain the magnitude of
associations (Spearman Brown Prophecy)
(Predicted) = (Observed) / (Rel_x)(Rel_y)
0.24
rXY =
0.60 x 0.80
Reliabilities of x & y
Observed
Predicted
= 0.50
Basic Concepts…
What is validity?
• the extent to which measures adequately
represent a concept and are useful for its
intended objective
What are the types of validity?
• Old: face, content, criterion, construct
• New: construct – test hypotheses based on
theoretical linkages with other measured
concepts (nomological net)
Types of Hypotheses
Types of hypotheses should align with the
objective of measurement
•
•
•
•
Differences between groups (discrimination)
Sensitivity to change (evaluation)
Accuracy of classification (prediction)
Correlations with similar measures
Summary
•
•
•
Concepts define the subject matter (boundary)
of different scientific disciplines
Measurement quality (level) defines the
sophistication scientific disciplines
Measurement is a two-step process that works
within practical constraints (objectives and
assessment conditions) and scientific
prerequisites (reliability and validity)