Research Design
(1) develop concepts derived from ideas, theories, or prior
research
(2) Translate concepts into measurable variables
(operationalize)
(3) select the most appropriate research method to gather
data (surveys, experiments, field methods, etc.) based on the
goals of the project (to describe, explain, predict, etc.)
(4) choose a sampling strategy
(5) plan how to collect the data and who will do it
(6) decide on the relevant statistical and analytical tools
(7) describe plans for interpreting and analyzing the results
and writing a final report
Concepts & Variables
 Concept: an idea, a general mental formulation
summarizing specific occurrences, such as “Gender”
 Construct: abstract concepts, such as “anomie”
 Variables: concepts made measurable and that vary
across a sample (not a constant)
 Values: the attributes or categories of a variable
 Operationalization: the process specifying how a
variable is to be measured
Concepts & Variables
Example 1: Child Rearing Values in Urban China
Hong Xiao, Sociological Perspectives 43:3, 2002, 457-471
Child socialization theory suggests a link between social class and
child-rearing values. Middle class parents value children’s autonomy,
whereas working class parents endorse conformity. However, the
effect of class and education differ in non-market and capitalist
economies.
Concepts: social class, autonomy, education (autonomy defined by
dictionary as “the condition or quality of being self-governing”)
Variables: autonomy of the job: how much decision-making in the job
(“how much” implies a variable amount, not all the same amount)
Operationalize: ask self-report using values of 1 (category: least job
freedom) to 10 (most job freedom)
Concepts & Variables
Example 2:
Mexican-Origin Women's Employment Instability
Roberto De Anda, Sociological Perspectives 43:3, 2002, 421-437
Compares the causes & earning consequences of employment instability
among Mexican-origin women. Workers who had one or more spells of
unemployment during the year or involuntarily worked part-time during
the entire year are said to have experienced employment instability.
Results show that Mexican-origin women with low levels of schooling,
and are immigrants are highly vulnerable to employment instability.
Concepts & Variables
P. 425: “The operationalization of labor market experience
variables strictly adheres to the official definition of labor
force concepts used by the Bureau of Labor Statistics. Thus
the unstable active category includes workers who
experienced at least one episode of unemployment during
the year or yearlong, involuntary part-time work…. A person
is considered unemployed if he or she is without a job and is
actively looking for work.” Persons are considered part-time
workers if they work less than thirty-five hours per week.
Part-year workers are defined as persons who work less than
fifty weeks during the year. “Respondents are asked directly
if they want and are available for full-time work.”
Concept: labor force
Variables: labor force experience: unstable active: part-time, year-long
Operationalize: ask directly – want work & are available
Hypothesis
Untested statements that specify a relationship between two
or more variables are called hypotheses. A hypothesis
is a hunch derived from an informed reading of the
literature, theory, or personal observations and
experience, and must be capable of being tested.
The outcome we are seeking to understand is called the
dependent variable and we hypothesize that its
variability in our sample depends on particular
explanations or causes. The explanations or causes (or
predictors, if you are trying to predict the variation in the
dependent variable) are called the independent
variables.
Type of Hypotheses
Null two-directional (two-tailed): There is no relationship
between the number of peers teenagers have who engage in
delinquent acts and the amount of delinquency they engage in.
Positive two-directional: There is a relationship between the
number of peers teenagers have who engage in delinquent
acts and the amount of delinquency they engage in.
Positive one-directional: Teenagers who have more peers who
engage in delinquent acts are more likely to engage in
delinquency themselves. More peers
more delinquency
Inverse (negative) one-directional: Teenagers who have high
levels of support from parents are less likely to engage in
delinquent acts. (more parental support
less
delinquency)
Research question:
Is there a relationship between peers and delinquency?
Note: none of these is about one person; hypotheses are about
aggregated groups of people.
Hypothesis Exercise: Write hypotheses about a
relationships between respondents’ ages and
hours viewing TV per week.
Which is the independent and which is the
dependent variables?
Two-directional null hypothesis:
There is no relationship between respondents’ ages and the number
of hours viewing TV per week.
Alternative (positive): There is a relationship between age and hours
of TV viewing.
One-directional positive hypothesis:
Older respondents watch more TV, or younger respondents watch
less TV.
One-directional inverse (negative) hypothesis:
Older respondents watch less TV, or younger respondents watch
more TV.
Hypotheses
De Anda: “Mexican-Origin Women’s Employment”
“Workers employed in core sectors of the economy will
have less employment instability.”
“Workers employed in periphery service sectors of the
economy will receive less than average wages.”
Dependent variables?
Amount of employment stability
Amount of wages (salary)
Independent variable?
Type of employment sector
One-tailed or two?
One directional
Levels of Measurement
A discrete variable has values that do not contain
additional information between those values.
 Number of books in a library
 Rankings of the Top 10 selling CDs
A continuous variable, on the other hand, has values
that have information between those values.
 Weight
 Height
Levels of Measurement
A nominal variable: a discrete measure whose values
represent named categories of classification. For coding
purposes, they can be assigned arbitrary numerals.
 The variable “favorite color” is measured nominally
 City you were born in [1=NYC, 2=LA, 3=Seattle, etc.]
An ordinal variable is a discrete measure with the category
values in sequence from high to low, or low to high. For
coding purposes, they can be assigned arbitrary numerals.
 Ranking (rank order) 3=bad, 2=OK, 1=great
 Social class (lower, middle, upper)
 Strongly Agree (1) to Strongly Disagree (4)
Levels of Measurement
An interval variable is a measure whose numbers are in order
with equal size intervals, but have no absolute or fixed zero as
a starting point.
 Temperature (Zero and below zero are actual measures)
A ratio variable is a measure whose numbers are in order with
equal size intervals and with an absolute zero.
 Height (no such thing as negative height)
 Salary in dollars ($20/hr. is twice as much as $10/hr)
These will be referred to as Interval/Ratio measures.
Levels of Measurement
 The levels of measurement are themselves ordinal:
Nominal
Ordinal
Interval
Ratio
simplest, non-mathematical measure
most complex, mathematical measure
 Variables with two values or categories are often called
dichotomous or dummy variables: These may be treated
as any level of measurement.
1 = Tall
2 = Short
0 = Did not graduate
1 = Graduated
 Ordinal measures with equal-appearing intervals
(Likert) may be treated as interval/ratio measures
1
2
3
4
=
=
=
=
Very Satisfied
Satisfied
Dissatisfied
Very Dissatisfied
Validity
Validity is about accuracy and whether the
operationalization is correctly indicating what it's supposed
to.
 Face
 Content
 Construct
 Criterion
Concurrent
Predictive
Reliability
Validity is about accuracy and whether the
operationalization is correctly indicating what it's supposed
to.
 Test-retest
 Parallel form & inter-item
 Split-half (internal consistency)
 Inter-rater
Reliability & Validity
Example 1:
The High School Personality Questionnaire (HSPQ) is a self-report
inventory for children ages 12-18. It measures 14 personality
characteristics that research has shown to be good predictors of social,
clinical, occupational, and school behavior.
Short-term test-retest reliability is in the .70s and .80s, and long-term is
in the .50s and .60s. Factor analysis was used to establish construct
validity.
Test-retest reliability
Construct validity
Reliability & Validity
Example 2:
The Goodenough-Harris/Draw a Person Test may be used to estimate
developmental status in children from 5 to 17 years of age. Because it is
a nonverbal measure of ability, it can be given to children from diverse
cultures and to deaf children.
Eleven raters with inter-rater reliability in the mid .90s assessed the
sample. Internal consistence was estimated as being in the high .80s
using Cronbach's Alpha. The average test-retest reliability coefficient was
.74. The validity for the construct "developmental change" was
established in the .60s which showed that scores tend to increase with
age, as expected in any test of development.
Inter-rater reliability
Internal consistency (split-half)
Test-retest reliability
Construct validity