Operationalization
of Variables and
Use of Rating Scales
1
Measurement--Operationalization
Earlier we said:
– Only when we begin to assign numbers to an object do
we begin to learn about that object.
“I often say that when you can measure what you are
speaking about and express it in numbers you know
something about it. But, when you cannot measure it,
when you cannot express it in numbers, your
knowledge is of a meager and unsatisfactory kind; it
may be the beginning of knowledge, but you have
scarcely, in your thoughts, advanced to the stage of
science, whatever the matter may be.”
Lord Kelvin (19th century physicist)
2
Measurement--Operationalization
First step in measurement is operationalization of
theoretical concepts (i.e., variables/constructs).
Operationalization?
– Specification of empirical indicators to represent
theoretical concepts
– While a theoretical/conceptual definition describes a
concept in terms of other concepts, an operational
definition describes the operations that need to be
performed to measure a concept. (Kerlinger, 1986)
– Represents a bridge between theory and data
– Operationalization helps convert theoretical propositions
to empirical (testable) hypotheses--Example
3
Measurement
Theoretical Hypothesis: “Juvenile delinquency is
a function of adult supervision”
• Theoretical Concepts?
 Juvenile
 Delinquency
 Adult
 Supervision
4
Measurement
•
•
THEORIES discuss relations between attributes of objects
MEASUREMENT: Involves assigning numbers to attributes of
objects according to a set of rules
•
•
OBJECTS in social/behavioral sciences are individuals and groups
Some ATTRIBUTES/TRAITS of individual and group are simple,
straight forward, and, directly measurable:
 e.g., height, weight, gender, income, education, etc.
 These represent “observed variables” (or “variables” for
short)
SO:
“VARIABLES” (or observed variables) represent
directly observable entities/attributes capable of
assuming two or more values.
5
Measurement
Other attributes/traits represent ABSTRACT and COMPLEX
concepts that CANNOT be measured directly:

Examples:
– Personality Traits (e.g., aggressiveness, competitiveness, locus of control,
etc.)
– Social/psychological characteristics, and mental/cognitive/emotional
dispositions (e.g., attitudes, values, perceptions)

These entities represent “Latent Constructs” (or “constructs”
for short).
“CONSTRUCTS” are attributes which are not directly
measurable, but may be measured on the basis of their multiple
indicators , where :
 Each indicator is represented by a variable.
 Multiple variables/indicators collectively serve as a proxy for
the construct.
 That is, for each subject, scores on multiple indicators/
variables will be combined (often averaged/summed) into a
single score. The resulting score is called a “summated scale.”
6
Construct
C
Stress
D1
D2
Physical
Tension
Emotional or
Psychological
Tension
Dimension
D
D
Nervousness
Anxiety
Could be measured by
D3
Mental Tension
D
Fear
Could be measured by
Could be measured by
V1
Blood pressure
V1
Extent of sleeplessness
V1
Headaches
V2
Pulse Rate
V2
Sweating
V2
Fatigue
V3
Stomach upsets
V3
Confusion
Construct
Dimension1
DRIVEN
BY WORK
Variable 1
Constantly
working
Achievement
motivation
D2
UNABLE TO
RELAX
V3
Persevering
despite
setbacks
V2
Very reluctant
to take time
off for anything
IMPATIENCE WITH
INEFFECTIVENESS
Does not
like to
work with
slow or
inefficient
people
Does not have
any hobbies
SEEKS
FEEDBACK
V2
Opts to do
a challenging
rather than a
routine job
Opts to take
moderate,
rather than
overwhelming
challenges
V4
Swears under one’s
breath when even
small mistakes
occur
V3
D5
SEEKS MODERATE
CHALLENGE
V1
V1
V2
Thinks of
work even
at home
D4
D3
V1
Asks for
feedback
on how the
job has
been done
V2
Is impatient
for immediate
feedback
8
Measurement—Rating Scales
Many constructs (e.g., personality traits, attitudes, perceptions)
are often operationalized through rating scales, whereby:
• Subjects select standard responses to one or more verbal
statements along some continuum.


Rating scales represent: Choice of a category or relative
position along some spectrum.
(Very Pessimistic=1
2
3
4
5=Very Optimistic)
Advantages of rating scales (vs. open comments)?
Capture responses to many questions:
– quickly,
– in very little space,
– from a relevant/appropriate perspectives
– in a standard, comparable form

Rating scale construction is a critical skill.
9
Measurement
Important considerations when designing rating scales:


Sometimes (ideally) the response scale is implicit in the
item/question.
(EXAMPLE: Do you visit your doctor at least once a year?)
But, sometimes no such common understanding exists between
the researcher and the respondent. Example:
(e.g., How long has it been since you last visited your doctor?)
In such a case, the researcher will need to
explicitly specify the scale in the question.
EXAMPLE?

How many months has it been since …?
When that is NOT possible, scale must be incorporated into the
response options verbally, numerically, or graphically
– Knowledge of scale construction becomes critical.
10
Measurement
A REVIEW OF
SELECTED TYPES OF
RATING SCALES
Commonly Used Rating Scale Formats
Click the following URL for a compendium of all sorts of useful
organizational measures that are provided in the U-Drive:
CLIKCK HERE
11
Rating Scales
The Multiple choice, Single Response scale
What kind of newspaper do you most often read for
business news? (Check only one).
(1)____Local, morning paper
(2)____Local, evening paper
(3)____Local, weekly paper
(4)____Regional, weekly paper
(5)____National, daily paper
(6)____National, weekly paper
(7)____Other (What kind?_____________)
Level of Measurement?
--Nominal/categorical
How many variables represented?
--Represents a single variable
12
Rating Scales
Multiple Choice—Multiple Response Scale:
Please check any type of newspaper you regularly
read for business news (Check all that apply).
____Local morning paper
____Local, evening paper
____Local, weekly paper
____National, daily paper
____National, weekly paper
____Other (What kind?___________)
Level of Measurement?
--Nominal/categorical
How many variables represented?
--Represents Multiple variables
13
Rating Scales
The forced Ranking Scale:
Please rank the soft drinks listed below in their order of
preference to you. Jot the number 1 next to the one you
prefer most, number 2 by your second choice, and so forth.
(NOTE: Do NOT assign the same rank to more than one item)
1.___Pepsi-Cola
2.___Coca-Cola
3.___Royal Crown Cola
4.___Like Cola
Level of Measurement?
--Ordinal
How many variables?
--Multiple variables (i.e., 4)
14
Rating Scales
The Paired Comparison Scale:
For each pair of soft drinks listed below, please
put a check mark by the one you most
prefer, if you had to choose between the
two.
(1) ____Pepsi-Cola
(2) ____Coca-Cola
(1) ____Royal Crown Cola
(2) ____Pepsi-Cola
(1)____Royal Crown Cola
(2) ____Like Cola
(1)____Royal Crown Cola
(2) ____Coca-Cola
(1) ____Coca-Cola
(2) ____Like Cola
(1) ____Like Cola
(2) ____Pepsi-Cola
Number of Comparisons =
K (K – 1) / 2
15
Rating Scales
The Fixed (Constant) Sum Scale:
Of the last 10 times that you ate lunch or
dinner at a casual or fast food restaurant,
how many times did you have each of the
things listed below?
(Please be sure to make the total equal 10)
___Hamburgers
___Hot dogs or sausage
___Chicken
___Pizza
- Level of Measurement?
___Chinese Food
Ordered Metric (Metric)
___Fish or seafood
___Deli sandwiches - Not a summated scale--each
___Hot sandwiches
option is a separate variable
___Mexican Food
___Other (What?_______________)
TOTAL = 10
16
Rating Scales
Another Example for Fixed-sum Scales:
LIFE VALUES: Divide a total of 100 points among
the following aspects of your life to indicate the
relative importance of each in your life at
present time. Please make sure the numbers
add up to 100.
__Your leisure (such as hobbies, recreation and
contacts with friends)
__Your community (such as voluntary organizations,
neighborhood activities)
__Your work
__Your political activities
__Your religion/faith
__Your wealth (such as the things you own)
17
__Your family
Rating Scales
The Ordered Response Scale:
Ordinarily, when do you (or someone in
your family) first turn on a television set
in your home on a weekday? (please check
only one.)
(1)__The first thing in the morning
(2)__A little while after awakening
(3)__Mid-morning
(4)__Just before lunch
(5)__Right after lunch
(6)__Mid-afternoon
(7)__Early evening, before dinner
(8)__Right after dinner
(9)__Late evening
(0)__Usually don’t turn it on
Level of Measurement?
Generates a single metric var.
(response options follow an
ordered sequence; larger
number represent a later time)
18
Rating Scales
The comparative Scale--an evaluative scale:
Compared to the previous management group, the new management team
is…. (circle the number that best reflects your opinion).
Very
Inferior
1
2
About
the same
3
4
Very
Superior
5
NOTES—Two issues:
a. Regarding whether to assign verbal anchors (label) the scale’s all or
only some of the response options (e.g., only the two ends of the
scale):
1.
2.
Both approaches are commonly used
Assigning explicit labels (verbal anchors) to more options
a. increases the likelihood that respondents will interpret each option
in the same manner, and
b. Increases test-retest reliability of the scale
3. Labeling only a few of the options is especially useful when more
variance is desirable, however, it is difficult to come up with
meaningful yet distinctly different labels for all scale options.
4. Some suggest that when working with truly bipolar scales (like the
above) using -2, -1, 0, +1, +2 are preferable because they reinforce
19
the bipolar nature of the requested responses.
Rating Scales
The Likert Scale:
Please pick a number from the scale to show how much you agree or
disagree with each statement and jot it in the space to the right of the item
1=Strongly Disagree 2=Disagree 3=Neutral 4=Agree 5=Strongly Agree
A man should never cry in public (T—if agreed)
_______
Swearing & obscenity is more repulsive in the speech of a
woman than a man (T)
_______
Women should receive equal pay for equal work (P/M)
_______
A man should NOT resent a woman supervising his work (P/M)
_______
There is nothing wrong with a man becoming a stay at home dad (P/M _______
Women should pay their share when dating (P/M)
_______
The husband should handle family legal decisions (T)
_______
Women should never put career before family (T)
_______
The initiative in dating should come from men (T)
______
Economic and social freedom is worth far more to a woman than the
acceptance of the ideal of femininity which has been set by men (P/M)_______
What does it Measure?
-- Note the direction of response options.
-- Number of scale points (see next slide)
Sex Role Attitudes--Traditional vs. Progressive/Modern
A summated metric Scale (a single construct/variable)–
Min scale value = 1x10=10, Max scale value = 5x10=50
20
Rating Scales
Appropriate number of points in the scale (5, 7, etc.) depends on :

Sophistication of the respondents
– Ability to discriminate scale anchors meaningfully
 Consider respondents’ age, education, intelligence, mental capacity, etc.

Complexity of the issue/object
– For views, decisions, intentions, etc. about complex, multi-dimensional issues
that require significant thought and soul searching, more points/anchors
needed

(e.g., attitudes toward welfare benefits, criteria for voting for a presidential
candidate, views on capital punishment, attitude toward unions, views on
deregulation of a particular industry).
– For views, decisions, intentions, etc. about issues that are simple and straight
forward, fewer points/anchors may work OK

(e.g., belief in god, attitudes toward polygamy, views on whether one should attend
college)
– For perceptions, intentions, etc. about a simple and inexpensive products that
require very little effort/thought to purchase, or are habitually purchased,
fewer scale points/anchors may work OK

(e.g., snacks, soft drinks, soaps, toothpaste or soap)
– For products that are complex, multifaceted, and expensive, more scale
points/anchors are required

(e.g., a house, car, TV set, or computer)
(continued)
21
Rating Scales
Number of points/anchors in the scale (3, 5, 7, etc.) :

Salience of the issue/object
– For views, decisions, etc. regarding serious issues, or issues and objects, that
have a high impact on, or play an important role in, one’s life, welfare, career,
peace of mind, health, etc., more scale points/anchors are needed


(e.g., intention to leave one’s employer, what type of college to attend, what type of
person to marry, or support for going to war.
The number of items in the scale
– If few items are used in a multi-item summated scale (fewer than 4 or 5
items), then more scale points/anchors can make up for that deficiency;
otherwise, fewer anchors work OK.
The Bottom Line:
To obtain accurate results, meaningful variance is needed!
So, all else considered, more scale points are preferable (unless
other considerations instruct us us against it).
– More scale points generate more variance, which helps prevent range
restriction and, thus, produce more accurate results and give greater
flexibility in choice of statistical techniques.
22
Rating Scales
The Verbal Frequency Scale--useful for
capturing one’s behavioral tendency/general policy
toward an issue.
Please pick a number from the scale to shows how often
you do each of the things listed below and jot in the
space to the right of the item.
1=Always 2=Often 3=Sometimes 4=Seldom 5=Never
Actually vote during a strictly local election
___
Actually vote during a state and national election ___
Vote along strict party lines **
___
Contribute money to a local political campaign
___
Contribute money to a national political campaign ___
Volunteer to work on a local political campaign
___
Volunteer to work on a state political campaign
___
Volunteer to work on a national political campaign ___
Not all summated scales have to
have reverse items.
What does it
Measure?
Political Activism (A
summated Scale --a
single variable)
Note the direction/values of
response options
23
Rating Scales
Horizontal, Numeric Scale of Importance (An Evaluative Scale):
When choosing a place to buy sports equipment, how important is:
(mean values shown
Not
from highest to lowest) Important
1
Quality of merchandise
Selection of brands
Variety of products
Service after purchase
Store guarantees
Price of the goods
Courtesy of service
Help and advice offered
Speed of service
Store hours
Location of store
Very
Important
2
3
4
5
- Non-Summated
Version (multiple
variables)
- Very useful for
measuring
decision making
criteria
24
Rating Scales
Horizontal, Numerical Importance Scale:
Please indicate below how important to you is
each of the public issues listed below?
Very Unimportant = 1
The
The
The
The
The
The
The
The
2
3
4
5 = Very Important
protection of endangered species of animals
improvement of the quality of the air
discovery of additional petroleum reserves (R)
development of ‘renewable” sources of energy
reduction or elimination of water pollution
development of additional nuclear power (R)
rapid industrial growth of the nation (R)
protection of overall ecological balance
What Does it Measure?
-Environmentalism
-Summated Version
(single variable)
Note: Not all
scales can be
bipolar and
balanced with a
neutral mid-point,
e.g., funniness)!
25
Rating Scales
The Semantic Differential Scale--very useful for measuring
mental images and perceptual profiles:
Please circle a number on each line below to show your opinion
of the pizza served here.
Hot
Bland
Expensive
Moist
Soggy
Good
Unattractive
Fresh
Small
Natural
1
1
1
1
1
1
1
1
1
1
2
2
2
2
2
2
2
2
2
2
3
3
3
3
3
3
3
3
3
3
4
4
4
4
4
4
4
4
4
4
5
5
5
5
5
5
5
5
5
5
6
6
6
6
6
6
6
6
6
6
7
7
7
7
7
7
7
7
7
7
How many
variables?
Cold
Spicy
Inexpensive
Dry
Crisp
Bad
Attractive
Stale
Large
Artificial
26
Rating Scales
Semantic Differential Image Profile:
1
Hot
Bland
Expensive
Moist
Soggy
Good
Unattractive
Fresh
Small
Natural
2
*
3
4
5
6
7
*
*
*
*
*
*
*
*
*
Cold
Spicy
Inexpensive
Dry
Crisp
Bad
Attractive
Stale
Large
Artificial
Mean Ratings are shown with “*”
27
Rating Scales
Positioning Profile Vis-a-Vis the Ideal Pizza:
1
Hot
Bland
Expensive
Moist
Soggy
Good
Unattractive
Fresh
Small
Natural
* Our Pizza
2
* *
3
4
5
6
*
*
7
*
*
**
*
*
*
*
**
* *
**
*
*
Cold
Spicy
Inexpensive
Dry
Crisp
Bad
Attractive
Stale
Large
Artificial
* The Ideal Pizza
28
Rating Scales
An alternative application of the Semantic Differential
Scale— Descriptive-Normative Gap Analysis
Please circle a number on each line below to show your opinion of
the pizza served here, first in terms of the way it currently is,
an then in terms of the way it should be.
The way it is:
The it should be:
The way it is:
The way it should be:
Hot = 1
Hot = 1
2
2
3
3
4
4
5
5
6
6
7 = Cold
7 = Cold
Bland = 1
Bland = 1
2
2
3
3
4
4
5
5
6
6
7 = Spicy
7 = Spicy
The way it is:
Expensive = 1
The way it should be: Expensive = 1
2
2
3
3
4
4
5
5
6
6
7 = Inexpensive
7 = Inexpensive
The way it is:
The way it should be:
2
2
3
3
4
4
5
5
6
6
7 = Dry
7 = Dry
Moist = 1
Moist = 1
29
Rating Scales
The Adjective Check List: Please put a check
mark in the space in front of any word or
phrase that describes your job.
__Easy
__Technical
__Boring
__Interesting
__Low-paying
__Strenuous
__Routine
__Dead-end
__Changing
__Important
__Demanding
__Temporary
__Safe
__Exhausting
__Difficult
__Rewarding
__Secure
__Slow-paced
__Enjoyable
__Rigid
__Pleasant
Multiple
__Satisfying
nominal
__Degrading
variables
__Risky
30
Rating Scales
The Stapel Scale: Please pick a number from the scale to
show how well each word or phrase below describes your job and
jot it in the space in front of each item.
SCALE:
Not at all = 1
__Easy
__Technical
__Boring
__Interesting
__Low-paying
__Strenuous
__Routine
__Dead-end
__Changing
__Important
__Demanding
__Temporary
2
3
4
5
6
7 = Perfectly
__Safe
__Exhausting
__Difficult
__Rewarding
__Secure
__Slow-paced
__Enjoyable
__Rigid
__Pleasant
Multiple
metric
__Satisfying
variables
__Degrading
__Risky
31
Rating Scales
Guttman Scale (a series of items tapping
progressively higher levels of the attribute being
measured):
Please check the option that best reflects how
you feel about your job:
___I like may job a great deal
___I am somewhat satisfied with my job
___I don’t strongly like or dislike my job
___I am somewhat dissatisfied with my job
___I don’t like my job at all
A single metric
variable
32
Graphic Rating Scale, Stressing
Visual Communications
3
VERY
VERY
GOOD
2
1
VERY
VERY
POOR..
33
Measurement

Among the important characteristics that we use rating scales
to measure are people’s social-psychological properties (e.g.,
attitudes, perceptions, values)
– Often illusive and multidimensional--embrace more than a simple
reaction to a simple statement (constructs)
– Often measured through responses to a series of verbal statements
– Among these, measurement of attitudes is very common, but
particularly challenging.
For a sample of organizational measures (rating scales) see:
-
U-Drive: Measurement Instruments
-
http://www.dmidi.net/All+Scales
–
http://division.aomonline.org/rm/
–
http://www.psywww.com/resource/bytopic/testing.html
–
http://www.isworld.org/surveyinstruments/surveyinstruments.htm
–
http://www.personalityresearch.org/tests.html
34
Measurement
ATTITUDES: Fairly enduring psychological
predispositions focused on some target (e.g.,
person, event, idea, issue)
Note that attitudes have 3 components/
Dimensions:
– Affective/Emotional (How do you feel
about it? liking/disliking)
– Cognitive/Beliefs/knowledge (What do
you think/know about it?)
– Behavioral/action (What are you likely
to do about it?)
35
Measurement
Example: Job Attitudes
 Affective/Emotional Components?
– I love my job.


Cognitive/Beliefs/Knowledge
Component?
– My job allows me to use a wide range
of my abilities.
Behavioral/Action Components?
– If I had the choice, I would quit my
job today.
36
MEASUREMENT OF ATTITUDES:
Guidelines for constructing scale items to
measure ATTITUDES:
– Use multiple items (i.e., statements or
questions) to measure the attitude--use
a summated scale
– Each item in the scale should deal with
only one point/issue.
– Likert scales (i.e., AgreementDisagreement scales) are very helpful in
construction of these scale items.
37
MEASUREMENT OF ATTITUDES
Guidelines for constructing scale items to
measure ATTITUDES:
– Attitudes have a target object; keep focus of
each and all items on the target object

That is, keep the focus on the entity whose
favorableness/unfavorableness you are assessing)
Example: Which of the items is problematic?
 I love my job
 I will quit my job the first chance I get.
 I am very competent in my job
 In this economy, good Jobs like mine are
hard to come by.
38
MEASUREMENT OF ATTITUDES:
Guidelines for constructing scale items to measure
ATTITUDES:
– Prepare a list of relevant dimensions (+ & -) of the target object,
and use it as a basis for constructing the scale items (e.g.,
attitudes toward welfare system--what are some of the issues
that make people favor/oppose it? The + or -_Features?):
• Cost: Costly to taxpayers, drain on federal budget
• Protection Against Poverty/Safety Net:
–Provides safety net for the needy
–Helps reduce poverty
–Takes care of those who can’t take care of themselves
• Social-Economic Values:
–Undermines the competitive and free economic system
–Source of social ills (promotes laziness)
–Undermines sense of individual responsibility
–Reduces motivation to search for a job
–Creates a cycle of dependency
–Reflects society’s compassion toward the needy
• Fairness:
–Helps distribute wealth more justly
–Creates a fairer society
–Unfair to those who work hard and pay their taxes
39
MEASUREMENT OF ATTITUDES:
Guidelines for constructing scale items to
measure ATTITUDES:
– Use only items that are clearly indicative of a
favorable or unfavorable viewpoint/disposition
toward the target object.
– When applicable, make some statements favorable
and others unfavorable (Reverse Items).
– Consider asking about reactions to hypothetical
situations when necessary.
– Consider including items capturing the feeling,
belief, and action components of attitudes.
– Select an appropriate range for response options
(e.g., a 3-point vs. 5-point vs. 7-point scale).
40
MEASUREMENT OF ATTITUDES:
Guidelines for constructing scale items to
measure ATTITUDES:
– Consider whether or not to use a “Neutral”
midpoint in the scale. Choose your word
carefully. Note that “Neutral” is not equivalent
to “Don’t Know.”
 Neutral,
Unsure, or Undecided could mean
Indifferent, Ambivalent, Equally Favorable and
Unfavorable, Honest Lack of Opinion, or Not Have
Made up One’s Mind.
– Use “Don’t Know” sparingly as a choice.
 “Don’t
Know” implies lack of sufficient information to
express an opinion.
 If used, “Don’t Know,” like “Not Applicable”, has to
be treated as missing data.
41
EVALUATION OF PRODUCTS, SERVICES,
PROGRAMS, OBJECTS, ETC.
Guidelines for Developing EVALUATIVE
Instruments:
– Develop a comprehensive list of possible
characteristics/attributes to be
evaluated.
– Select the salient attributes to be
evaluated.
– For services and programs, consider
including both summative (service
outcomes) and formative (process of
service delivery) evaluation items.
42
EVALUATION OF PRODUCTS, SERVICES,
PROGRAMS, OBJECTS, ETC.
Guidelines for Developing EVALUATIVE
Instruments:
– Consider also asking respondents to rank
the three/five most important
attributes/criteria to them if they were to
select from among alternative product
brands/service providers.
– Consider also asking them to rank the three
most appealing/satisfactory attribute of the
product/service/program being evaluated.
– Include an item soliciting an overall
evaluation.
43
EVALUATION OF PRODUCTS, SERVICES,
PROGRAMS, OBJECTS, ETC.
Guidelines for Developing EVALUATIVE
Instruments:
– Consider asking if they would recommend the
product, service, or program to others.
– Consider asking how they would characterize
the ideal product/service in terms of salient
attributes.
– Consider including “Don’t Know” as a response
choice.
– Consider asking some open ended questions. 44
QUESTIONS OR
COMMENTS
?
45