Manipulation and
Measurement of Variables
Psych 231: Research
Methods in Psychology

For labs this week you’ll need to download
(and bring to lab):

Class experiment packet
Announcements

Independent variables


Dependent variables


How do we measure these?
Extraneous variables



How do we manipulate these?
Control variables
Random variables
Confound variables
Variables

Independent variables


Dependent variables


How do we measure these?
Extraneous variables



How do we manipulate these?
Control variables
Random variables
Confound variables
Variables

Methods of manipulation

Straightforward
• Stimulus manipulation - different conditions use
different stimuli Abstract vs. concrete words
• Instructional manipulation – different groups are given
different instructions Has an “a” vs. “ISU related”

Staged
• Event manipulation – manipulate characteristics of the
context, setting, etc.

Subject (Participant)– there are (pre-existing mostly)
differences between the subjects in the different conditions
• leads to a quasi-experiment
Manipulating your independent variable

Choosing the right levels of your independent
variable






Review the literature
Do a pilot experiment
Consider the costs, your resources, your limitations
Be realistic
Pick levels found in the “real world”
Pick a large enough range to show the effect
• And in the middle of the range
Choosing your independent variable

These are things that you want to try to
avoid by careful selection of the levels of
your IV (may be issues for your DV as well).




Demand characteristics
Experimenter bias
Reactivity
Floor and ceiling effects
Identifying potential problems


Characteristics of the study that may give away the
purpose of the experiment
May influence how the participants behave in the study
 Examples:
• Experiment title: The effects of horror movies on mood
• Obvious manipulation: Ten psychology students looking
straight up
• Biased or leading questions: Don’t you think it’s bad to
murder unborn children?
Demand characteristics

Experimenter bias (expectancy effects)

The experimenter may influence the results
(intentionally and unintentionally)
• E.g., Clever Hans

One solution is to keep the experimenter (as well as
the participants) “blind” as to what conditions are
being tested
Experimenter Bias

Knowing that you are being measured

Just being in an experimental setting, people don’t
always respond the way that they “normally” would.
• Cooperative
• Defensive
• Non-cooperative
Reactivity

A value below which a response cannot be
made


As a result the effects of your IV (if there are indeed
any) can’t be seen.
Imagine a task that is so difficult, that none of your
participants can do it.
Floor effects

When the dependent variable reaches a level
that cannot be exceeded



So while there may be an effect of the IV, that effect
can’t be seen because everybody has “maxed out”
Imagine a task that is so easy, that everybody scores
a 100%
To avoid floor and ceiling effects you want to
pick levels of your IV that result in middle level
performance in your DV
Ceiling effects

Independent variables


Dependent variables


How do we measure these?
Extraneous variables



How do we manipulate these?
Control variables
Random variables
Confound variables
Variables

The variables that are measured by the
experimenter

They are “dependent” on the independent variables
(if there is a relationship between the IV and DV as
the hypothesis predicts).
Dependent Variables

How to measure your your construct:

Can the participant provide self-report?
• Introspection
• Rating scales

Is the dependent variable directly observable?
• Choice/decision (sometimes timed)

Is the dependent variable indirectly observable?
• Physiological measures (e.g. GSR, heart rate)
• Behavioral measures (e.g. speed, accuracy)
Choosing your dependent variable


Scales of measurement
Errors in measurement
Measuring your dependent variables

Scales of measurement - the correspondence
between the numbers representing the
properties that we’re measuring

The scale that you use will (partially) determine what
kinds of statistical analyses you can perform
Measuring your dependent variables

Categorical variables


Nominal scale
Quantitative variables
Scales of measurement

Nominal Scale: Consists of a set of categories that have
different names.



Label and categorize observations,
Do not make any quantitative distinctions between
observations.
Example:
• Eye color:
blue, green, brown, hazel
Scales of measurement

Categorical variables



Nominal scale
Ordinal scale
Quantitative variables
Scales of measurement

Ordinal Scale: Consists of a set of categories that are
organized in an ordered sequence.

Rank observations in terms of size or magnitude.

Example:
• T-shirt size:
Small,
Med,
Lrg,
XL,
Scales of measurement
XXL

Categorical variables



Nominal scale
Ordinal scale
Quantitative variables

Interval scale
Scales of measurement

Interval Scale: Consists of ordered categories where
all of the categories are intervals of exactly the same
size.

With an interval scale, equal differences between numbers on
the scale reflect equal differences in magnitude.

Ratios of magnitudes are not meaningful.
• Example: Fahrenheit temperature scale
40º
20º
“Not Twice as hot”
Scales of measurement

Categorical variables



Nominal scale
Ordinal scale
Quantitative variables


Interval scale
Ratio scale
Scales of measurement

Ratio scale: An interval scale with the additional feature
of an absolute zero point.

Ratios of numbers DO reflect ratios of magnitude.

It is easy to get ratio and interval scales confused
• Example: Measuring your height with playing cards
Scales of measurement
Ratio scale
8 cards high
Scales of measurement
Interval scale
5 cards high
Scales of measurement
Ratio scale
8 cards high
0 cards high
means ‘no
height’
Interval scale
5 cards high
0 cards high
means
‘as tall as
the table’
Scales of measurement

Categorical variables



Nominal scale
Ordinal scale
Quantitative variables


Interval scale
Ratio scale
“Best” Scale?
• Given a choice, usually prefer highest level of
measurement possible
Scales of measurement


Scales of measurement
Errors in measurement
Measuring your dependent variables
Example: Measuring intelligence?




Reliability & Validity
How do we measure the
construct?
How good is our
measure?
How does it compare to
other measures of the
construct?
Is it a self-consistent
measure?

Reliability


If you measure the same thing twice (or have two
measures of the same thing) do you get the same
values?
Validity

Does your measure really measure what it is
supposed to measure?
• Does our measure really measure the construct?
• Is there bias in our measurement?
Errors in measurement
Reliability = consistency
Validity = measuring what is intended
unreliable
invalid
reliable
invalid
Reliability & Validity
reliable
valid

True score + measurement error


A reliable measure will have a small amount of
error
Multiple “kinds” of reliability
Reliability

Test-restest reliability

Test the same participants more than once
• Measurement from the same person at two
different times
• Should be consistent across different
administrations
Reliable
Reliability
Unreliable

Internal consistency reliability


Multiple items testing the same construct
Extent to which scores on the items of a measure
correlate with each other
• Cronbach’s alpha (α)
• Split-half reliability
• Correlation of score on one half of the measure with
the other half (randomly determined)
Reliability

Inter-rater reliability


At least 2 raters observe behavior
Extent to which raters agree in their observations
• Are the raters consistent?

Requires some training in judgment
Reliability

Does your measure really measure what it is
supposed to measure?

There are many “kinds” of validity
Validity
VALIDITY
CONSTRUCT
INTERNAL
CRITERIONORIENTED
FACE
PREDICTIVE
CONVERGENT
CONCURRENT
DISCRIMINANT
Many kinds of Validity
EXTERNAL
VALIDITY
CONSTRUCT
INTERNAL
CRITERIONORIENTED
FACE
PREDICTIVE
CONVERGENT
CONCURRENT
DISCRIMINANT
Many kinds of Validity
EXTERNAL

Usually requires multiple studies, a large body
of evidence that supports the claim that the
measure really tests the construct
Construct Validity

At the surface level, does it look as if the
measure is testing the construct?
“This guy seems smart to me,
and
he got a high score on my IQ measure.”
Face Validity

Are experiments “real life” behavioral situations,
or does the process of control put too much
limitation on the “way things really work?”
External Validity

Variable representativeness


Subject representativeness


Relevant variables for the behavior studied along
which the sample may vary
Characteristics of sample and target population
along these relevant variables
Setting representativeness

Ecological validity - are the properties of the
research setting similar to those outside the lab
External Validity

The precision of the results

Did the change result from the changes in the DV
or does it come from something else?
Internal Validity

History – an event happens the experiment

Maturation – participants get older (and other
changes)

Selection – nonrandom selection may lead to biases

Mortality – participants drop out or can’t continue

Testing – being in the study actually influences how
the participants respond
Threats to internal validity

Control variables

Holding things constant - Controls for excessive
random variability
Extraneous Variables

Random variables – may freely vary, to spread
variability equally across all experimental conditions

Randomization
• A procedures that assure that each level of an extraneous
variable has an equal chance of occurring in all conditions of
observation.
• On average, the extraneous variable is not confounded with our
manipulated variable.
Extraneous Variables



Can you keep them constant?
Should you make them random variables?
Two things to watch out for:


Experimenter bias
Demand characteristics
Control your extraneous variable(s)

Confound variables

Other variables, that haven’t been accounted for
(manipulated, measured, randomized, controlled)
that can impact changes in the dependent variable(s)
Confound Variables

Pilot studies



A trial run through
Don’t plan to publish these results, just try out the
methods
Manipulation checks


An attempt to directly measure whether the IV
variable really affects the DV.
Look for correlations with other measures of the
desired effects.
“Debugging your study”


Read chapters 8.
Remember: For labs this week you’ll need to
download:

Class experiment packet
Next time