Research Methods
Lecture 2
Non-experimental and Experimental
Research Approaches
Chapters 2 & 3
Research
Designs/Approaches
Type
Purpose
Time
frame
Experimental
Test for cause/ current
effect
relationships
Quasiexperimental
Test for cause/ Current or
past
effect
relationships
without full
control
Degree
of
control
High
Examples
Comparing
two types of
treatments for
anxiety.
Moderate Gender
to high
differences in
visual/spatial
abilities
Research
Designs/Approaches
Type
Purpose
Time
frame
Degree
of
control
Examples
Nonexperime
ntal correlational
Ex post
facto
Examine
relationship
between two
variables
Current
(crosssectional)
or past
Low to
medium
Examine the
effect of past
event on
current
functioning.
Past &
current
Low to
medium
Relationship
between
studying style
and grade
point average.
Relationship
between
history of
child abuse &
depression.
Research
Designs/Approaches
Type
Purpose
Time
frame
Nonexperime
ntal correlational
Cohortsequential
Examine relat. Future betw. 2 var.
predictive
where 1 is
measured
later.
Examine
Future
change in a
var. over time
in overlapping
groups.
Degree
of
control
Examples
Low to
moderate
Relat. betw.
history of
depression &
development
of cancer.
How motherchild
negativity
changed over
adolescence.
Low to
moderate
Research
Designs/Approaches
Type
Purpose
Time
frame
Degree
of
control
Examples
Survey
Assess
opinions or
characteristics
that exist at a
given time.
Discover
potential
relationships;
descriptive.
Current
None or
low
Voting
preferences
before an
election.
Past or
current
None or
Low
People’s
experiences of
quitting
smoking.
Qualitative
Non-experimental Research
Designs

Describes a particular situation or
phenomenon.
 Hypothesis generating
 Can describe effect of implementing actions
based on experimental research and help
refine the implementation of these actions.
Correlational Design
Measure two variables
– Study methods and grade-point average
Determine degree of relationship between them
– Correlation coefficient (e.g., r = 0.50)
Allows description and prediction of the
relationship
Correlational Studies

Type of descriptive research design
– Advantage is that it can examine variables that
cannot be experimentally manipulated (e.g., IQ
and occupational status).
– Disadvantage is that it cannot determine
causality.
– Third variable may account for the association.
– Directionality unclear
Origins of the Correlation
Coefficient
Children’s height
64”
65”
70”
66”
67”
68”
69”
2
4
5
5
69”
2
3
5
8
9
9
68”
3
6
10
12
12
2
67”
7
11
13
14
13
10
66”
6
8
11
11
8
6
65”
3
4
6
4
3
2
Correlation between parent’s height and children’s height
Correlation Scatterplot
Strong Positive Relationship
Correlation Scatterplot
Strong Negative Relationship
Correlational Designs

What are some correlational studies that
you can do?
Ex Post Facto Study

Variable of interest is not subject to direct
manipulation but must be chosen after the
fact.
 E.g., Define two groups of people according
to a certain characteristic (e.g., history of
trauma) and measure how they respond in
terms of anxiety to a certain stimulus (e.g.,
watching violent film).
 Limitation – self-selection bias, cohort
effects may explain the effect.
Personality and Hypertension,
Effect of Hypertension Awareness
Hypertension Study - Screened 10,500 Employees
Matched
Normotensive
1st BP Screen
Hypertensive DBP
2nd BP
Screen
2nd BP Screen
2-3 weeks later
Personality
Study
Personality
Study
3rd BP Screen
3 months
4th BP Screen
4 months
5th BP Screen
5 months
Mean DBP >= 90 mmHg
Personality and Hypertension: Effect
of Hypertension Awareness
Variable
Group 1
Aware
Hypertensive
Group 2
Normotensive
Group 3
Unaware
Hypertensive
Group 4
Normotensive
% Male
75
75
89
89
Age
Mean*
(SD)
46.2
(9.2)
46.2
(8.2)
46.4
(8.3)
45.8
(8.0)
135.1/
93.9
(9.2/5.1)
118.7/
76.3
(11.5/5.5)
135.8/
93.8
(8.2/3.4)
118.5/
75.7
(10.3/4.8)
SBP/DBP
Mean*
(SD)
Personality and Hypertension: Effect
of Hypertension Awareness
Variable
Neuroticism
Mean*
(SD)
Type A
Mean*
(SD)
Group 1
Aware
Hypertensive
Group 2
Group 4
Normotensive
Group 3
Unaware
Hypertensive
12.0
(5.3)
9.3
(5.3)
9.7
(4.8)
9.5
(4.6)
0.79
(8.5)
-3.0
(9.4)
-2.0
(9.4)
-2.6
(8.2)
* Group 1 > Group 2 & Group 3 (p < 0.01)
Normotensive
Personality and Hypertension:
Effect of Hypertension
Awareness
14
13
12
11
10
9
8
7
6
5
4
3
2
1
0
Aware Hyper
Normot
Unaware Hyper
Normot
Neuroticism
Aware hypertensive > normotensive & unaware hypertensive,
P < 0.001
Personality and Hypertension:
Conclusion
Do hypertensives have a different personality
than those with normal blood pressure?
– No, because the unaware hypertensives did not
differ from the normotensives.
Why did the aware and unaware hypertensives
differ?
– Possible explanations?
Personality and Hypertension:
Conclusion

Awareness of hypertension status confounds
assessment of the association between
personality characteristics and hypertension.
– Due to hypertension labeling effect; or
– Due to self-selection bias
Cross-Sectional Study
Designs

Compares groups at one point in time
– E.g., age groups, ethnic groups, disease groups.

Advantage is that it is an efficient way to
identify possible group differences because
you can study them at one point in time.
 Disadvantage is that you cannot rule out
cohort effects.
Longitudinal Design

Gathers data on a factor (e.,g. confidence) over time.
 Advantage is that you can see the time course of the
development or change in the variables
– Confidence increasing with age.
– Confidence increasing at a faster rate in the 30’s than the
40’s.
– Confidence decreasing in the 50’s and 60’s.
 Disadvantage is it is costly and still subject to bias
Cohort-Sequential Design

Combines a bit of the cross-sectional design and
longitudinal design
– E.g., Different age groups are compared on a variable over
time.
Advantage – very efficient and reduces some of the
biases in the cross-sectional design since you can see
the evolution of change over time.
 Disadvantage – cannot rule out cohort bias or the
problem of the ‘unidentified’ third variable
accounting for the change.

Naturalistic Observation

Aims to unobtrusively observe behaviour in
the natural setting.
 Observing in the natural setting enables one
to minimize or eliminate the problem of
artificial behaviour in response to being
studied (i.e., reactivity effects).
 One variation is being a participant observer
(e.g., undercover agent).
Naturalistic Observation

Advantages
– Observe the natural phenomena (not artificial)

Disadvantages
– Observer bias
– Reactivity in subjects
– Ethics
Meta Analysis
(Glass 1976)

Quantitative approach to integrate and
describe results across a range of
independent studies.
 Enables you to combine the probability (p)
value for statistical tests over a number of
studies.
 Enables you to determine the effect size of
the independent variable (e.g., treatment
group) across studies.
Survey Research

Collecting standarized information from
people using an interview or self-report
format.
 Typically survey knowledge or opinions.
 To standarized the information one uses a
questionnaire with set questions.
 Ideally the questionnaire has been validated.
 Representativeness of the sample is very
important.
Survey Methods

Interviews
– Advantage - Comprehensive, ensure participant
understands the question, minimizes missing
data, enables clarification of unclear responses
– Disadvantage – expensive, people more like to
refuse participation, can be risky for interviewer,
interviewer may bias the responses.
Types of Survey Methods

Face-to-face interviews
– Expensive and time-consuming

Telephone interviews
– Need to use random-digit dialing to reach both
listed and unlisted numbers.

Mail
– Return rate is usually low (20-30%).
Types of Questions

Open-ended
– E.g., Can you tell me about your typical
experience with dating?

Close-ended
– E.g., How do you typically meet someone to date?
 Introduced by someone
 Social event
 In university class or place of work
 At a bar
 Through sports or other athletic events
Sampling

Population is everyone in your population
of interest.
 Sample is some proportion of the
population.
 Haphazard sampling – convenience sample
 Random sampling
– There is always some degree of sampling error.
Qualitative Methods

Multimethod approach to studying people in
their natural environment
– It is interpretive – researcher has to make sense
of the data
– Multimethod – can use interviews,
photographs, natural observation, archives, etc.
– It is typically conducted in person’s natural
environment.

Valuable to use when phenomenon not fully
defined.
Qualitative Methods
Limitations
Participant’s observations and accounts can
be biased. For example, filtered by his/her
style of expression, gender, social class,
race, age, ethnicity, etc.
 People are seldom able to provide a true and
full account of their experience.

– Defensive
– Lack insight
– Unaware
Qualitative Methods
Transcripts
Experimental Designs

Examines differences between experimentally
manipulated groups or variables (e.g., one
group gets a certain drug and the other gets a
placebo).
 At minimum, experimental (independent)
variable has two levels (e.g., drug vs.
placebo).
– Advantage is that you can determine causality.
– Disadvantage is cost and many variables cannot
be experimentally manipulated (e.g., smoke
exposure over time).

Experimental Designs
Four Canons for Identifying
Causality
Method of Agreement –
– Observe the element common to several
instances of the event
– Problem is you may inadvertently overlook a
significant variable.

Method of Difference –
– Identify the different effects produced by two
situations that are alike in all ways but one.
– Fairly robust and strong method.

Experimental Designs
Four Canons for Identifying
Causality
Joint methods of agreement and difference
– Observe the element common to several
instances of the event
– Form hypothesis based on observations
– Test hypothesis using method of difference

Method of Concomitant Variation –
– Identify the different effects produced by more
than two situations that are alike in all ways but
one.
– E.g., Compare two active drugs to a placebo
Experimental Design

Because it is so difficult with human behaviour
to demonstrate causation unequivocally, some
argue that a theory or prediction can only
achieve the status of “not yet disconfirmed”
(Popper, 1968).
 Our scientific efforts are directed at finding the
causal factors rather than ‘the cause’ per se.
Psychological Experiment:
Is Objective

Researcher strives for freedom from bias.
 Recognize that:
– Mistakes can occur
– Carefully scrutinize all steps of the experiment to
identify where mistakes are likely.
– Take the steps necessary to minimize error.
Psychological Experiment:
Focuses on a Phenomenon

This is a publicly observable behaviour.
– Actions
– Appearances
– Verbal statements
– Responses to questionnaires
– Physiological responses.
Psychological Experiment:
Is Done Under Strictly Controlled
Conditions

Eliminate all factors that could influence the
outcome other than the factor being
manipulated.
 Control is needed to infer causation.
 All conditions are kept constant except one;
the manipulated variable.
 The variable of interest is varied in order to
test its effect.
Experimental Method

Advantages
– Strength with which causal relationships can be
inferred.
– Ability to manipulate one or more variables.
– Proven to be a very useful and robust scientific
method (i.e., withstood the test of time).
Experimental Method

Disadvantages
– Tight controls often produce artificial
conditions that could limit the generalizability
of the findings (i.e., internal vs. external
validity trade-off).
– Time consuming.
– Expensive.
– Human behaviour is very complex and cannot
be fully studied using experimental methods.
Experimental Method:
Threats to Internal Validity

Learning or practice effects
– Scores on a measure change on repeat testing
because participant has more familiarity with
the measure and so answers more truthfully.

Natural history effects
– Something happens in the social background
(e.g., society because more affluent generally)
and this influences the participant’s responses.

Maturation
– Natural developments in the participant account
for the changes (e.g., getting older).
Experimental Method:
Threats to Internal Validity

Regression to the mean
– High scores generally move down toward the
mean and low scores move up.

Instrumentation
– If pre and post tests are not equivalent in all
ways (e.g., difficulty, readability) then
differences observed may be due to
‘instrumentation’ differences rather than due to
your experimental manipulation.
Experimental Method:
Threats to Internal Validity

Subject problems
– Selection bias (e.g., participation rate).
– Attrition (e.g., only motivated subjects stay in
the experiment).
Experimental Method:
Threats to External Validity

Subject variables
– Selection bias.
– Attrition bias

Artificial conditions
– E.g., In order to measure a subject’s blood
pressure in response to a well-fined stressor you
bring him/her into the laboratory but his/her
response in the laboratory may not reflect how
his/her blood pressure would really respond
under stress in his natural environment.
Let me know…
If there are any topics from today’s lecture that
need fuller explanations.
 Anything you particularly liked about the
lecture (today’s or others as we go along).
 Anything you particularly disliked about the
lecture (today’s or others as we go along).
