Foundations of
Research
1
1
Lectures 3: Developing Research Questions
Basic experimental designs.
 How do social values affect
science?
 Where do research questions &
hypotheses come from?
 Variables in research.
 Basic experimental designs.
Phenomenon
Theory
Hypothesis
Methods
& data
Results
© Dr. David J. McKirnan, 2015

The University of Illinois Chicago


McKirnanUIC@gmail.com
Do not use or reproduce without
permission

Discussion &
Conclusions
Foundations of
Research
2
2
Lectures 3: Developing Research Questions
Basic experimental designs.

 How do social values
affect science?

Where do research questions &
hypotheses come from?

Variables in research.

Basic experimental designs.
Phenomenon
Theory
Hypothesis
Methods
& data
Results
Discussion &
Conclusions
Foundations of
Research
Social Values…
3
3
Phenomenon
Theory
…affect what we choose
as our research question…
Hypothesis
Methods
& data
Results
Discussion &
Conclusions
Specific methods are
more standard and
objective
…and our conclusions
Foundations of Values, theory and data in the scientific process.
Research
4
4
 Social
Phenomenon
Theory
Hypothesis
Methods
& data
Results
Discussion &
Conclusions
values help define a
scientific “problem” or question.
 Norms,
values (& data) determine
what is credible / fundable.
 Theory
is influenced by norms +
empirical background of field.
 Science
hinges on clear,
objectively stated hypotheses.
 Clear
hypotheses lessen bias in
interpreting results.
 Methods
& analyses are most
objective, but fields vary in
methodological rigor.
 The
“meaning” of a finding is
influenced by cultural & social
values or concerns.
…for science and, particularly,
for society.
Foundations of
Research
Values and science: The internet and sexual risk.
5
5
 Social
Phenomenon
Theory
Hypothesis
values help define a
scientific “problem” or question.
 Norms,
values (& data) determine
what is credible / fundable.
 Until 1974 homosexuality was
classified as mental illness, and was
studied that way.
 Little research was done on the
Methods
& data
Results
Discussion &
Conclusions
topic until the HIV crisis in early
1980s.
 GLBT research is now mainstream.
 Research on Transgendered people
is now emerging as important.
Foundations of Values, theory and data in the scientific process.
Research
Phenomenon
 Theory
is influenced by values &
empirical background of field.
Theory
Hypothesis
Explanations of, e.g., crime or drug use
can take very different perspectives:
Methods
& data
 Theories of individual Ψ factors –
e.g., depression – examine issues
within the person that interfere with
decision making.
Results
Discussion &
Conclusions
 Theories emphasizing Social
structure address the social or
cultural environment.
Taking an individual v. social
perspective can be an important value
choice.
6
6
7
7
Foundations of Values and science: climate change, 1.
Research
Phenomenon
Theory
Hypothesis
Methods
& data
Results
Discussion &
Conclusions
 Science
hinges on clear,
objectively stated hypotheses.
 Clear
hypotheses lessen bias in
interpreting results.
 Studies can show some results just
by chance.
 Without a clear hypothesis we
cannot tell whether they are
meaningful or junk.
Foundations of Values, theory and data in the scientific process.
Research
Phenomenon
Theory
Hypothesis
Methods
& data
Results
Discussion &
Conclusions
Shared conventions for methods can
make this step less biased. However:
 Behavioral sciences vary
considerably in their rigor.
 Choosing, e.g., quantitative vs.
qualitative research can be a value
choice.
 Fields such as literary criticism,
history or feminist studies may use
substantially different methods.
 Methods
& analyses are most
objective, but fields vary in
methodological rigor.
8
8
Foundations of Values, theory and data in the scientific process.
Research
9
9
Phenomenon
Theory
Hypothesis
Methods
& data
Results
Discussion &
Conclusions
 The “meaning” of a finding is
affected by existing theory and
empirical findings.
 Many important findings – e.g., from
economics – have little affect on
social policy if they contradict a
widely shared ideology.
 The
“meaning” of a finding is
influenced by cultural & social
values or concerns.
…for science and, particularly,
for society.
10
10
Foundations of
Research
 How do social values affect
science?

 Where do research
questions & hypotheses
come from.
 Variables in research
 Basic experimental designs
Phenomenon
Theory
Hypothesis
Methods
& data
Results
Discussion &
Conclusions
Foundations of
Research
Research questions
Where do research questions come from?
 Practical questions
 Unanswered questions from previous research
 Testing theories.
11
11
Foundations of
Research
Sources of research questions
Practical / applied questions
Describe or explain an important social process
 Evaluate an intervention or policy change
EXAM PLE

Does college increase critical
thinking, complex reasoning
and written communication?
Longitudinal data show that colleges
generally
on this.
Research
onare
thisfailing
practical
issue;
Predictors:
 Can evaluate performance over time
rigorous
reading
/ of
writing
assignments,
Negative:
Positive:
Low
expectations
students,
 Identify- behaviors
or cultural
variables
that contribute…
- outside
contact with
activities
Instructors.
(work),
- social rather than academic focus.
12
12
Foundations of
Research
Sources of research questions
13
13
Practical / applied questions
Unanswered questions from previous research

Clarify conflicting / unclear findings


Attraction: physical characteristics v. cultural & value similarity?
Do previous findings generalize to…
…different groups


Many Social Psychology studies enroll middle(+) class White
female undergraduates in research labs.
…different research areas


Can interventions to increase healthy behavior generalize to
recycling and energy conservation?
…different research approaches


Do controlled lab studies generalize to less controlled field
research?
Foundations of
Research
Theories
Practical / applied questions
Unanswered questions from previous research
Testing theories

Use existing theory to explain a new phenomenon


Test contrasting theories of a phenomenon


“Sensation seeking” personality is associated with drug use.
Might it also explain unsafe sex in adolescents or gay men?
How much is adolescent problem behavior controlled by
psychological variables (depression) vs. peer influence?
Develop new / expanded theory

We discriminate among very subtle differences in smell.
Might olfactory cues affect who we are attracted to?
14
14
Foundations of
Research
Theories
How do we go from a research question
to an actual study?
 Phenomenon
 Theory
 Hypothesis
 Methods
15
15
Foundations of
Research
16
16
The research process
Phenomenon
Overall issue or question;
What controls emotional states?
Why are some people vulnerable to depression?
Theory
Possible explanation: “How it works” statement
Several theories may help explain the phenomenon
Theory 1
Theory 2
Emotional stability requires
secure emotional
attachments.
Some brains are genetically
disposed to serotonin
depletion during stress
Foundations of
Research
17
17
Research process, 2
Theory 1
Emotional attachment  emotional stability.
A theory can lead to several hypotheses
Hypothesis 1
Fewer parent – child interactions
 vulnerability to depression.
Hypothesis 2
Emotional support during stress
 less depression
A given hypotheses can be tested in several ways
Methods 1
Survey measurement: assess # of
“family meals” per week,
correlate it with self-reported
depression.
Methods 2
Experiment: ½ have structured
parent / child interactions, ½ do
not, induce stress to both
groups, assess depression
Foundations of
Research
18
18
Research process, 3
Theory 1
Emotional attachment  emotional stability.
Hypothesis 1
Family interactions  depression.
Hypothesis 2
(Non)Support + stress  depression
Some hypotheses are best tested in a
measurement approach, and some with
experimental designs
Best tested by a measurement
study
 Family interactions are
difficult to bring into the lab,
 Possible ethical problems.
Can be tested in an experiment:
Both support and stress can be
controlled or manipulated in
the lab.
Foundations of
Research
Research process:
The Big Picture
Phenomenon
Big picture question.
Theory 1
Possible explanation, invoking
one set of hypothetical
constructs.
Hypothesis 1
A prediction that logically flows
from – and tests – the theory.
Methods 1
Operationally define the variables
& test the hypothesis.
Theory 2
Alternate explanation, invoking
other hypothetical constructs.
Hypothesis 2
Another prediction that tests the
same theory.
Methods 2
An alternate operational definition
& way of testing the hypothesis.
19
19
Foundations of
Research
Question 1
A hypothetical
construct is…
A = A specific prediction about the
outcome of an experiment
B = A little known band from
Muncie Indiana
C = A general ψ process that
underlies our observations
D = A central element in a theory
20
20
Foundations of
Research
Question 2
To be testable, a
hypothesis…
Must rest on operational
definitions.
A = true
B = False
C = I don’t know
21
21
Foundations of
Research
Question 2
An operational
definition is
A = The procedure(s) we use to
measure a study variable
B = The way we define our theory
C = The procedure(s) we use to
manipulate a study variable
D = What we use to derive our
hypothesis
22
22
Foundations of
Research
Question 3
To be testable, a
hypothesis…
Must potentially be found to
be false.
A = true
B = false
C = I don’t know
23
23
Foundations of
Research
Question 1
A theory…
A = Leads to one specific
hypothesis
B = May be one of several ways to
explain something
C = Is not as important as simply
collecting data
D = Is what you make up to
explain why you forgot your
boy/girl friend’s birthday
E = Is not really affected by social
or personal values
24
24
25
25
Foundations of
Research
 How do social values affect
science?
 Where do research questions &
hypotheses come from.
Phenomenon
Theory

 Variables in research
Hypothesis
 Basic experimental designs
Methods
& data
Results
Discussion &
Conclusions
26
26
Foundations of
Research
Variables in research:
 Types of variables


Independent v. Dependent / Predictor v. Criterion

Random variables

Confounds
Control variables
 Forming variables:


Direct manipulation

Measurement
Indirect manipulation
Foundations of
Research
Variables in Research: Independent v. Dependent
27
27
Experiments
Independent Variable
Dependent Variable
Imposed / manipulated by
Measured as the outcome
researcher
 Defines the “contrast space”  Models the phenomenon
 What is compared to what
e.g., drug v. placebo
 What is being explained;
e.g., task performance
 Hypothetical “cause”
 “Effect”
 Categorical
 Continuous
Foundations of
Research
Variables in Research: Predictor v. Criterion
28
28
Measurement / Correlational Studies
Predictor Variable
Measured
Criterion Variable
Measured as the outcome
 Defines the “contrast space”  Models the phenomenon
 What predicts the outcome
e.g., age, ethnic group…
 What is being explained;
e.g., political attitudes
 Hypothetical “cause”
 “Effect”
 Continuous or
Categorical
 Continuous or
categorical
29
29
Foundations of
Research
Variables in research:
 Types of variables


Independent v. Dependent / Predictor v. Criterion

Random variables

Confounds
Control variables
 Forming variables:


Direct manipulation

Measurement
Indirect manipulation
Foundations of
Research
Control Variables
30
30
 Experiments:
 Things we keep constant between experimental & control groups
environment. Room, equipment, time of day, researchers…
• Physical environment;
• Procedures.
Procedures; Recruitment & enrollment, consent, instructions,
assessments.
• Basically, everything except the Independent Variable.
 Measurement / observational:
 Constant procedures across different measurement groups:
• All participants get the same questions, addressed in same way…
 Statistical controls:
• Individuals or groups always differ on variables such as SES, age…
• Statistical controls can adjust data for those differences.
31
31
Foundations of
Research
Variables in research:
 Types of variables


Independent v. Dependent / Predictor v. Criterion

Random variables

Confounds
Control variables
 Forming variables:


Direct manipulation

Measurement
Indirect manipulation
Foundations of
Research
Random Variables
32
32
 Variables that vary randomly within and between groups;
• Demographics; age, ethnicity, education…
• Attitudes & beliefs, psychological states…
 We consider these irrelevant to our experiment.
 If we come to consider a variable relevant – e.g., prior
experience with experimental settings – we will need to control
it.
 Variables we cannot control in the experiment / measurement
we use statistical procedures to adjust for.
33
33
Foundations of
Research
Variables in research:
 Types of variables


Independent v. Dependent / Predictor v. Criterion

Random variables

Confounds
Control variables
 Forming variables:


Direct manipulation

Measurement
Indirect manipulation
Foundations of
Research
Confounds
34
34
 Variable other than the IV that affected the results.
I am •studying
the variable
effect of inadequately
visual media on
learning.
A control
controlled…
Mary Lou is the “instructor” for the high media (experimental) group.
• Unanticipated / unmeasured random variable:
Joe instructs the low visual (Control) group.
o
Differed between groups…
The groups differ on the outcome measure of performance: the high media
group did
o better.
Actually made a difference to the results…
Clearly the
use of visual
media
o
...rather
than the
IV. enhances performance…..?
 Confounds make results difficult (impossible?) to interpret.
• Known confound may be quasi-controlled via statistical
analyses.
• Confounds create the illusion that results supported the
hypothesis…
• …when the results were due to something else entirely
(e.g., mistake in measurement / experimental design).
35
35
Foundations of
Research
Variables in research:
 Types of variables


Independent v. Dependent / Predictor v. Criterion

Random variables

Confounds
Control variables
 Forming variables:


Direct manipulation

Measurement
Indirect manipulation
Foundations of
Research
Creating independent variables [IVs]
1. Direct experimental manipulation

Most typical of “true” experiments

Maximum control over IV
2. Indirect manipulation via experimental or
research conditions

Less direct control over IV
3. Quasi-Independent variables: forming groups
using a measured variable.

Experiments without complete control over variables

Used in measurement studies
36
36
Foundations of
Research
Forming Variables
1. Direct experimental manipulations
 Drug or biomedical intervention,
 Behavioral intervention,
 Focused experimental study,
 System-wide “treatment” (e.g., policy change, school-based…),
 Structure the IV
vis-à-vis:
 Simple presence v. absence of the
treatment or stimulus
 Single v. multiple treatment doses
 Type of treatment or stimulus
37
37
Foundations of
Research
38
38
Example: Direct experimental manipulation
 Hypothesis: words presented in a semantic context are
recalled better than when presented randomly.
Experimental
group
Control
group
Independent
Variable
Dependent
Variable
Target words
presented
within complete
sentences
Word
recognition task
Target words
presented
randomly
Word
recognition task
Completely controlled
by the experimenter
Experimental
manipulation same as
Independent Var.
Foundations of
Research
Forming Variables
39
39
2. Indirect experimental manipulations


“Stage manage” a social event

Induce mood via description of the experiment
 requirecheck
presentation
inyou
frontactually
of peers
Do aStress
manipulation
to see if
manipulated
your
Independent
Variable
Depression
 Write
about worst
mistake you ever made
EXAM PLE

Experimental “induction” of a mood or state…
Stereotype standard
threat  “This
test reflects
onstress)
your group”
 Self-report,
assessment
(e.g., of
Anxiety rating
Stage a robbery or fight
 Observer
Happiness  Lottery winnings?
Relaxation  Meditation
Foundations of
Research
Indirect experimental manipulation
40
40
 Hypothesis: happiness enhances pain resistance.
Experimental
Manipulation
Independent
Variable
Experimental
group
Imagine you
won the lottery
– what will you
buy first?
Happy
state
Control
group
…what will you
need to buy this
month?
Normal /
baseline
state
Directly
controlled by
experimenter
Not directly
controlled by
experimenter
Our induction of the Independent
Variable (happiness) is indirect.
Dependent
Variable
Cold Presser
Task (ice bucket)
Cold Presser
Task
Foundations of
Research
Indirect experimental manipulation
41
41
 Hypothesis: happiness enhances pain resistance.
Experimental
Manipulation
Independent
Variable
Experimental
group
Imagine you
won the lottery
– what will you
buy first?
Happy
state
Control
group
…what will you
need to buy this
month?
Normal /
baseline
state
?
?
Dependent
Variable
Cold Presser
Task (ice bucket)
Cold Presser
Task
A Manipulation Check tests whether the
experimental manipulation actually induced the
Independent Variable
42
42
Foundations of Quasi-independent variables
Research
3. Create a quasi-Independent variable using a
measured variable.

Categorize participants by measuring (not manipulating)
something:

Scores over / under an established “cut point”,



Scores based on a frequency a distribution:

Median split: top v. bottom half.

Extreme scores: top v. bottom 10% of scores.
Simple self-identification:


e.g., over 4 depression symptoms on a standard scale.
e.g., “Republican” v. “Democrat”.
Behavioral index:

Used any drug in previous year v. not.

Voted in 2012 v. not.
Not a “True” IV:
Participants not
randomly
assigned to
groups.
Using a measured variable to create groups
Foundations of
Research



43
43
Administer depression scale, count the # symptoms rated 2 or 3.
Form groups based on a cut point; e.g., > 4 symptoms = quasi-clinical
depression.
Participants are assigned to groups based on their ratings, not
random assignment.
Below is a list of different feelings. Circle the number that shows how many days you felt each
of these over the PAST WEEK.
I was bothered by things that usually do
not bother me.
I felt I could not shake off the blues even
with help from my friends or family.
I had trouble keeping my mind on what
I was doing.
I felt depressed.
I felt that everything I did was an effort.
My sleep was restless.
I was happy.
I enjoyed life.
I felt sad.
Rarely or
none of
the time
A Little
of the Time
A moderate
amount of
the time
Most or all of
the time
(less than 1 day)
(1 or 2 days)
(3 - 4 days)
(5 - 7 days)
0
1
2
3
0
1
2
3
0
1
2
3
2
2
2
2
2
2
3
3
3
3
3
3
# of symptoms
rated0 2 or 3 1
0
0
0
0
0
1
1
1
1
1
Foundations of
Research
Question 4
An independent
variable…
A = Is measured on a continuous
scale
B = Is manipulated by the
researcher
C = Is the outcome of the
experiment
D = Is the “phenomenon” you
are trying to explain.
E = Does not care about other
people
44
44
Foundations of
Research
Question 5
An dependent
variable…
A = Is typically measured on a
binary scale
B = Is manipulated by the
researcher
C = Is the putative cause in the
theory
D = Is the “phenomenon” you
are trying to explain.
E = Is over-concerned about
other people
45
45
46
46
Foundations of
Research
 How do social values affect
science?
 Where do research questions &
hypotheses come from.
 Variables in research

 Basic experimental
designs.
Phenomenon
Theory
Hypothesis
Methods
& data
Results
Discussion &
Conclusions
Foundations of
Research

Overview: Basic
47
47
Designs
“Pre-experimental” designs: no control group
Post-Test Only Design
Pre- Post- Test Design
Group
assignment
Pre-test
Experimental
manipulation
Outcome
Experimental
Observe1
Treatment
Observe2
Foundations of
Research

48
48
Basic Designs
“Pre-experimental” designs: no control group
Post-Test Only Design
Pre- Post- Test Design

True (or Quasi-)experimental designs with a
control group
“After only” Control group design
Pre- Post- Group Comparisons
Group
assignment
Pre-test
Experimental
manipulation
Outcome
Experimental
Observe1
Treatment
Observe2
Control
Observe1
Control
Observe2
Foundations of
Research

49
49
Basic Designs
“Pre-experimental” designs: no control group
Post-Test Only Design
Pre- Post- Test Design

True (or Quasi-)experimental designs with a
control group
“After only” Control group design
Pre- Post- Group Comparisons
Multiple group comparison
Group
assignment
Pre-test
Experimental
manipulation
Outcome
Experimental
Observe1
Treatment 1
Observe2
Experimental
Observe1
Treatment 2
Observe2
Control
Observe1
Control
Observe2
“Pre-experimental” designs
Foundations of
Research
Post-Test Only Design
Group
Typically existing
group.
Treatment
Measure
Experimental intervention
(“Treatment”) often not
controlled by the researcher:
Naturally occurring or systemwide events.
Measurement may
or may not be
controlled by the
researcher.
Pre- Post- Test Design
Group
• Only 1 group
available?
• Naturally
occurring
intervention?
Measure1
Treatment
Measurements at
baseline.
Measure1
All participants get the
same treatment.
Measurement at Followup.
50
50
Foundations of
Research
“Pre-experimental” Designs (2)
Advantage of “Post-” & “Pre- Post-” Designs:
Allow us to study naturally occurring interventions.
 e.g., test scores before and after some school change,
 Crime rates after a policy change, etc.
 Having both Pre- and Post measures allows us to examine
change.
51
51
Foundations of
Research
“Pre-experimental” Designs (2)
Disadvantage of “Post-” & “Pre- Post-” Designs:
No control group = many threats to internal validity.

Maturation: Participants may be older / wiser by the post-test

History; Cultural or historical events may occur between preand post-test that change the participants

Mortality: Participants may non-randomly drop out of the study

Regression to baseline: Participants who are more extreme at
baseline look less extreme over time as a statistical confound.

Reactive Measurement: Scores may change simply due to
being measured twice, not the experimental manipulation.
52
52
Foundations of
Research
53
53
Experiments
“After only” Control group design
Experimental
Control
Treatment 2
Observe2
Control
Observe2
Adds a control group. Either…
Observed Groups:
Measure Dependent
Variable(s) only at follow-up.

Naturally occurring (e.g., Class 1. v. Class 2) or

Self-selected (sought therapy v. did not…).
Use experimental or standard
measures (e.g., grades, census
data, crime reports).
Assigned Groups:
 Randomly assign participants to experimental v.
control group, or
 Match participants to create equivalent groups.
Foundations of
Research
54
54
Advantages of experimental design
“After only” Control group design
Experimental
Control
Advantage:
Treatment 2
Observe2
Control
Observe2
Lessens the likelihood of confounds (threats to internal
validity).
Control group
Random assignment
Disadvantage: Existing or self-selected groups may have confounds.
No baseline or pre- measure available:
 Cannot assess change over time.
 …or if the groups were equivalent at baseline.
Foundations of
Research
Basic Designs:
True experiments (2)
Pre- Post- Group Comparisons
Group 1
Measure 1
Group 2
Measure 1
Observed
(quasi-experiment)
or
Assigned
(true experiment).
(most common study design)
Baseline (“pre-test”)
measure of study
variables and
possible confounds.
55
55
Foundations of
Research
Basic Designs:
56
56
True experiments (2)
Pre- Post- Group Comparisons
(most common study design)
Group 1
Measure 1
Treatment
Measure 2
Group 2
Measure 1
Control
Measure2
The group getting the
experimental condition is
contrasted with a control
group..
“Post-test”;
 Simple outcome
 Change from
baseline.
Foundations of
Research
Basic Designs:
57
57
True experiments (2)
Pre- Post- Group Comparisons
(most common study design)
Group 1
Measure 1
Treatment
Measure 2
Group 2
Measure 1
Control
Measure2
Advantages:
Pre-measure assesses baseline level of Dependent
Variable
 Allows researcher to assess change
 Can find matched pairs of participants and assign each
to different groups (rather than random assignment).
 Can assess whether groups are equivalent at baseline.
Disadvantage: Highly susceptible to confounds if using observed or
self-selected groups.
Foundations of
Research
More Complex Experimental Designs
Multiple group comparison
Group 1
Measure1
Treatment #1
Group 2
Measure1
Treatment #2
Group 3
Measure1
Control
 3 (or more)
groups
 Typically formed
by Random
assignment.
Multiple experimental groups, e.g.
 Low drug dose,
 High drug dose,
 Placebo.
or
 Male therapist,
 Female therapist,
 Wait list control.
58
58
Foundations of
Research
59
59
More Complex Experimental Designs
Multiple group comparison
Group 1
Measure1
Treatment #1
Measure2
Group 2
Measure1
Treatment #2
Measure2
Group 3
Measure1
Control
Measure2
Compare:
 Level 1 of independent
variable from Level 2
 Either / both experimental
groups from control grp.
Foundations of
Research
60
60
More Complex Experimental Designs
Multiple group comparison
Group 1
Measure1
Treatment #1
Measure2
Group 2
Measure1
Treatment #2
Measure2
Group 3
Measure1
Control
Measure2
Advantages: Test dose or context effects:
 Drug doses, amounts of psychotherapy, levels of anxiety, etc.
Disadvantage:
 More costly and complex.
 Potential ethical problem with a “no dose” (or very high dose) condition.
Foundations of
Research
Participant
Selection
Sample
Recruit a
sample of
participants
from the
larger
population
61
61
Experimental design overview
Participant
Assignment
Experimental
Procedures
Experimental
Treatment or
Manipulation
Results
Group A 
Procedure 
Treatment 
Outcome
Group B 
Procedure 
Control 
Outcome
(Group C) 
(Procedure ) 
(Alternate
Treatment?)
…randomly
assign
participants
to groups.
Same
procedures
for all
groups…

…except the
experimental
manipulation,
(Independent
variable).
(Outcome)
Hypothesis:
Dependent
Variable
varies by
group only.
Foundations of
Research
Participant
Participant Experimental
Recruitment Assignment Procedures
Sample
Does the
sample
represent the
population?
External
validity
Random
selection
62
62
Overview: experimental designs
Experimental
Treatment or
Manipulation
Results
Group A 
Procedure A  Treatment 
Outcome
Group B 
Procedure A  Control 
Outcome
Group C 
Alternate
 Outcome
Procedure A  Treatment (?)
• Biased recruitment?
• Large enough sample?
Foundations of
Research
Participant
Participant Experimental
Recruitment Assignment Procedures
Sample
Does the
sample
represent the
population?
Experimental
Treatment or
Manipulation
Results
Group A 
Procedure A  Treatment 
Outcome
Group B 
Procedure A  Control 
Outcome
Group C 
Alternate
 Outcome
Procedure A  Treatment (?)
Groups = at
baseline?
External
Internal
Random
selection
Random
Assignment
validity
63
63
Overview: experimental designs
validity
• Self-selection (in or out)
• Existing groups?
Foundations of
Research
64
64
Overview: experimental designs
Participant
Participant Experimental
Recruitment Assignment Procedures
Experimental
Treatment or
Manipulation
Results
Group A 
Procedure A  Treatment 
Outcome
Group B 
Procedure A  Control 
Outcome
Group C 
Alternate
 Outcome
Procedure A  Treatment (?)
Groups = at
baseline?
Procedures
= for all
groups?
External
Internal
Internal
Random
selection
Random
Assignment
Sample
Does the
sample
represent the
population?
validity
validity
validity:
Lack of
confounds
• Groups have ≅
expectations?
• Participants &
researchers blind?
Foundations of
Research
65
65
Overview: experimental designs
Participant
Participant Experimental
Recruitment Assignment Procedures
Experimental
Treatment or
Manipulation
Results
Group A 
Procedure A  Treatment 
Outcome
Group B 
Procedure A  Control 
Outcome
Group C 
Alternate
 Outcome
Procedure A  Treatment (?)
Groups = at
baseline?
Procedures
= for all
groups?
External
Internal
Internal
Random
selection
Random
Assignment
Sample
Does the
sample
represent the
population?
validity
validity
validity:
Lack of
confounds
Core assumption:
• Groups equivalent
• …except for the
experimental
manipulation (IV)
Foundations of
Research
Participant
Participant Experimental
Recruitment Assignment Procedures
Sample
Experimental
Treatment or
Manipulation
Results
Group A 
Procedure A  Treatment 
Outcome
Group B 
Procedure A  Control 
Outcome
Group C 
Alternate
 Outcome
Procedure A  Treatment (?)
Procedures
Does
• Does
the the operational
Are the definition
the same for
sample
express
well the
groups
construct
equal of all
interest?
groups?
represent
the dose
at baseline?
• Correct
of the IV?
population?
External
Internal
Random
selection
Random
Assignment
validity
66
66
Overview: experimental designs
validity
Internal
validity:
Lack of
confounds
Independent
variable
reflects the
construct?
External
Validity
Correct IV?
Foundations of
Research
Participant
Participant Experimental
Recruitment Assignment Procedures
Sample
Does the
sample
represent the
population?
Experimental
Treatment or
Manipulation
Results
Group A 
Procedure A  Treatment 
Outcome
Group B 
Procedure A  Control 
Outcome
Group C 
Alternate
 Outcome
Procedure A  Treatment (?)
Procedures
= Independent
Groups
• Are
=at group
differences
for all
variable
baseline?
statistically
significant
(reliable
groups?
reflects the
& meaningful)?
construct?
Groups
really
different at
outcome?
Internal
External
Internal
Correct IV?
Likelihood of
chance results
External
Internal
Random
selection
Random
Assignment
validity
67
67
Overview: experimental designs
validity
validity:
Lack of
confounds
Validity
Validity:
Foundations of
Research
68
68
Overview: experimental designs
Participant
Participant Experimental
Recruitment Assignment Procedures
Experimental
Treatment or
Manipulation
Results
Group A 
Procedure A  Treatment 
Outcome
Group B 
Procedure A  Control 
Outcome
Group C 
Alternate
 Outcome
Procedure A  Treatment (?)
Groups = at
baseline?
Procedures =
for all
groups?
External
Internal
Internal
Random
selection
Random
Assignment
Sample
Does the
sample
represent the
population?
validity
validity
validity:
Lack of
confounds
Independent Groups really
variable
different at
reflects the
outcome?
construct?
External
Internal
Correct IV?
Likelihood of
chance results
Validity
Validity:
Foundations of
Research
Overview

How do social values affect
science?

Where do research questions &
hypotheses come from.

Variables in research

Basic experimental designs
69
69
Foundations of
Research
Overview: key terms

Theory

Hypothetical construct

Hypothesis

Variable

Operational definition

Internal & external validity

Independent v. Dependent variables

Measurement v. experimental studies
70
70
Foundations of
Research
Research flow
71
71
Foundations of
Research
Observation or Measurement
Simple Description
Qualitative
Explore the actual
process of a
behavior.
72
72
Basics of major forms of research.
Quantitative
Describe a
behavioral or
social trend.
External validity
Experiments
Correlational
Studies
Quasiexperiments
“True”
experiments
Relate measured
variables to each
other to test
hypotheses.
Test hypotheses
in naturally
occurring events
or field studies.
Test specific
hypotheses via
controlled “lab”
conditions.
Internal validity
Foundations of
Research
Key terms & concepts

Role of values & social judgments in the
research process

Basic elements of science

Hypothetical constructs

Operational definitions

Statement of testable hypothesis



Predictive, potentially refutable

Specify Variables in functional relationship
Replication
The hierarchy of
phenomena, theory,
hypotheses, & methods:
73
73
Foundations of
Research


Key terms & concepts, 2
Measurement v. experimental methods

Types of variables used

Cause & effect assumptions
Creating variables

Direct treatment dose or manipulation

Indirect use of context (manipulation
check)

Using a measured variable (self-reports or
“status” variable”) to assign to groups
74
74
Foundations of
Research

Overview, 3
Experimental design key elements

Control group v. non-controlled designs

Threats to internal validity:





Maturation
History
Mortality
Regression to baseline
Reactive Measurement

“Pre-experimental” designs

Pre-post designs

Multiple group comparisons.
75
75