RESEARCH METHODS
Modules 2 and 3
Class Experiment
This is a study of some factors that may affect the solution
of anagrams or scrambled words. A list of letter
combinations that can be unscrambled to create common
English words will be presented. No proper (capitalized)
nouns, abbreviations, or foreign words appear.
Solve each anagram in the spaces below. If you are
having difficulty, raise your hand. I will give you a hint.
Example: DORW
Answer __________________
NORC
NOONI
MATOOT
PREPPE
TEBE
EAP
Class Experiment
This is a study of some factors that may affect the solution
of anagrams or scrambled words. A list of letter
combinations that can be unscrambled to create common
English words will be presented. No proper (capitalized)
nouns, abbreviations, or foreign words appear.
Solve each anagram in the spaces below. If you are
having difficulty, raise your hand. I will give you a hint.
Example: DORW
Answer __________________
LUBL
CALEM
NUKKS
SEUMO
BAZER
EAP
Experiment—Review
General Topic of study: effect of perceptual
set on solving of anagrams.
 What was the research question?
 What was the hypothesis?
 How was the hypothesis tested?
 What were the results?
 Alternative explanation of results and errors?

NEW SUBTOPIC:
INTRODUCTION
Modules 2 and 3: Research Methods
Introduction

How do we gain knowledge?
 Sensory
experience … but can be
undependable and incomplete
 Agreement with others … but majority does not
always = truth
 Expert opinion … but no one knows everything
 Logic … but rational explanations not always
best/true
 Scientific method … that’s the one
Introduction

How do we gain knowledge (cont’d)
 Scientific method



Includes rigorous testing of hypothesis and public nature of
procedures and conclusions
Systematic testing of hypothesis, gathering of data, and analysis
Steps:





Form a question (usually from theory, event, daily experience)
State hypothesis (educated guess of answer to question)
Test hypothesis (systematic way of getting answer)
Report results (replication – if repeated and same results, more likely
true)
Example of why use scientific method?



Question: If someone confessed to a murder in an online newsgroup
would you notify the police?
When simply asked, 56.4% of subjects said they would
When experimentally tested (had it occur), only 3 of 200 newsgroup
members notified police
The Scientific Method
The scientific method is the process of
testing our ideas about the world by:
setting up
situations that
test our ideas.
making careful,
organized
observations.
analyzing
whether the data
fit with our ideas.
If the data don’t fit our ideas, then we modify our
ideas, and test again.
Introduction

Goals of research
 Describe
 Explain
 Predict
 Control
(causation)
Research design (how study is set up) influences
what type of conclusions you can reach.
Introduction

Important general terminology
Hypothesis:
specific statement of
expected results
Testable prediction
Subjects: persons/animals on whom
study is conducted
Introduction

Important terms (cont’d)
 Sample and population
 Population:
all possible members of group
 Sample: representative subgroup of pop.
Want sample to be representative.
 If representative, then can generalize to pop.
 How to make sample representative?



Random sample: every member of pop has = chance of
being selected
Stratified sample: take representative subgroups in
proportions as they exist in pop
 Sampling:
process by which Ss are selected
 Random assignment: when members of random sample
are assigned randomly to experimental or control group
Introduction

Imp. terms (cont’d)

Variable: characteristic or factor that can assume different
values (changes)
 Examples – stress, alcohol use, gender, etc.
 What we study in psychological research
 Not the subjects (people in the study)—what you study in them
 Confounding variables: uncontrolled variables that represent
potential error in research (alternative explanations for results)
aka extraneous variables


Difference btn experimental and control groups other than
independent variable
Operational definition: term defined by how it is measured in
research


How would you define…intelligence? dancing? happiness?
aggression? etc.
Diagnostic criteria for Major Depression
Introduction

Imp. terms (cont’d)
 Validity
and reliability
 Validity
= how accurate are results (true)
 Reliability = how consistent are results
(consistency—would you get them again)
 Test-retest
(similar results from same test)
 Parallel forms (do alternate versions agree)
 Inter-rater (do diff judges/raters agree)
 Split-half (do items referencing same issue score
similarly)
NEW SUBTOPIC:
OTHER ISSUES IN
RESEARCH
Modules 2 and 3: Research Methods
New major subtopic:
Other issues in Research

Within this topic, we will discuss…
 Research settings and their advantages and
disadvantages
 Types of measurements used in research
Other issues

Research Settings: where research occurs
 Laboratory
research:
 Controlled
setting, unnatural
 People would not normally be there engaging in
beh that study is examining
 Advantages
 Offers
more control of extraneous variables
 Can standardize procedures
 Disadvantages
 Artificial
 May
elicit atypical beh
Other issues (cont’d)

Research settings (cont’d)
 Field
research:
 Naturally
occurring setting; Ss already there
engaging in beh
 Advantages
 Offers
more realistic view of beh in natural
settings
 Disadvantages
 Less
control of extraneous variables
 Less able to standardize procedures
Other issues (cont’d)

Research measurements: how data is quantified
 Self-reports
 Ind
reports on past beh, thoughts, beliefs, etc.
 Obtained through interviews, questionnaires, etc.
 Advantages: able to study what you cannot
directly observe
 Disadvantages:
 Distortion
of responses (people lie)
 Ability to be accurate can be limited (defense
mechanisms, recall problems, diff perspectives)
Other issues (cont’d)

Research measurements (cont’d)
 Behavioral
observations
 Any
activity that can be observed
 Examples →raising hand, blood pressure, etc.
 Advantage →
 Beh
objectively measured
 Disadvantage
 Inds
→
may act differently when observed
Other issues (cont’d)

Research measurements (cont’d)
 Archival
Records
 Data
that already exists (collected by someone other
than researcher) – data would exist if study did not
occur
 Examples → medical records, crime rates, newspaper
stories, etc.
 Advantage →
 Observation
 Disadvantage
 Incomplete
does not influence data
→
records
 Not enough detail
Other issues (cont’d)

Research measurements – applied
 Counting
the number of cigarette butts on the
ground in a smoking area.
 Subjects rate their level of anger on scale of 1 –
10
 Reviewing correlation between GPA and SAT over
the last 20 years
 Counting the number of times a student leaves his
seat
 Scores on test used to see if teaching method works
NEW SUBTOPIC:
RESEARCH DESIGNS
Modules 2 and 3: Research Methods
New major subtopic:
Research Designs

Introduction



Research design → How study is set up
Influences procedures and conclusions—what can you say
about what you are studying when you get the results
 Describe
 Explain
 Predict
 Control
Four types
 Descriptive research (case study, survey research, and
naturalistic observation)
 Correlational research
 Experimental research
 Literature review (your term paper)
Research Designs (cont’d)

Descriptive
 Case
Study:
 In-depth
examination of one ind (or few)
 Incl. use of interviews, observations, letters,
diaries, reports from others, etc.
 Uses
 Source
of insight/ideas (Piaget, Freud)
 Describe rare cases (Dissociative Identity D/O)
 Psychobiography (Erikson’s Young Man Luther)
 Used to illustrate anecdotes
Research Designs (cont’d)

Descriptive (cont’d)
 Case
Study (cont’d)
 Advantage
 Able
to study rare cases
 Initial exploration of new cases
 Disadvantages
 Not
as systematic—less control
 No comparison
Research Designs (cont’d)

Descriptive (cont’d)
 Survey
research
 Interview
or questionnaire to lg grp of people
 Attempt to estimate attitudes or beh’s of lg grp
 Examples → political polls, epidemiology studies
 Not simply because a survey is used
 No
manipulation of variables
 Advantages
 Easy
to do
 Lg amt of data quickly
Research Designs (cont’d)

Descriptive (cont’d)
 Survey
research (cont’d)
 Disadvantages
 People
do not respond (either some items or
completely)
 Effect of wording
Ss may not know meaning of words
 Ss may give answers acceptable rather than
accurate
 Questions may be poorly written
 Ss more likely to agree to “not allowing” versus
“forbidding”

Research Designs (cont’d)

Descriptive (cont’d)
 Survey
research (cont’d)
 Disadvantages
 Effect
(cont’d)
of wording (cont’d)
Actual wording influences → When asked about
“assisting poor” 23 % of Ss said too much money
was spent, BUT when asked about “welfare” 52%
said too much spent
 Order of questions →

 “People should have freedom to express their opinions publicly.”
 Different responses depending on whether prior question dealt
with Catholic Church or Nazi Party.
Research Designs (cont’d)

Descriptive (cont’d)
 Naturalistic-observation
 Observe
beh as it occurs in natural settings
 Jane Goodall and gorillas
 Advantage →
 Able
to see beh as it happens
 Disadvantage
 Decreased
→
control over conditions
Research Designs (cont’d)

Correlational research design
 Research
that attempts to find links or
connections btn var’s so that if you know one var
you can predict other
 Attempt to find rel’s btn variables
 How? Collect 2/more scores from ea S
 Examples
 TV
violence and aggression
 College grades and salary
Research Designs (cont’d)

Correlational research (cont’d)
 Correlation results
 Positive
correlation = variables increase or decrease
together (TV violence and aggression)
 Negative correlation = increase in one variable assoc
with decrease in other (optimism & illness)
 Caution
 Correlation
btn psych var’s rarely perfect
 Correlation does NOT equal causation
Can only say var’s are related
 Cannot say causation → may be third variable

 Illusory
correlation: when connection appears to exist, but
is actually random (when we believe rel exists, we tend to
notice instances that confirm that belief)
Research Designs (cont’d)

Experimental research design
 Research
in which 1/more var’s is manipulated
(controlled by researcher) to see if it causes
changes in another var
 To est cause and effect relationships btn
variables
Research Designs (cont’d)

Experimental research (cont’d)
 Variables
in experimental research
 Independent
variable (IV): variable that is
manipulated/controlled by researcher
 Dependent variable (DV): variable that is
affected by IV
 Examples
– “effect of ____ (IV) on ____ (DV)”
 Increases
in anxiety cause improvement in
performance
 Observing live models perform aggressive acts
led to those children engaging in aggressive acts
Research Designs (cont’d)

Experimental research (cont’d)
 Two
groups of subjects
 Experimental
group: grp that gets IV
 Control group: does not get IV
 Both groups measured on DV
 Important → only diff btn grps should be IV
(random sampling and random assignment)
 This is what allows you to say the IV causes a
change in DV (i.e., make cause and effect
conclusions) → Why?
Research Designs (cont’d)

Experimental research (cont’d)
 Essential
features of this design
 Comparison
of at least 2 groups (control and
experimental grp)
 Manipulation of IV by researcher
 Random assignment of Ss to groups
 Summary
→ want to see if there is diff in scores
btn grps so you can say it was IV that caused
diff (everything else same => only diff is IV)
Research Designs (cont’d)

Literature review
 Summary
of broad range of research on subject
 Most published research studies have literature
review within introduction—supports direction of
study and eventual hypothesis
 Term paper for this class
Summary of the types of Research
Comparing Research Methods
Research
Basic Purpose
Method
Descriptive
To observe and
record behavior
Correlational
Experimental
How
Conducted
Perform case
studies,
surveys, or
naturalistic
observations
To detect naturally Compute
occurring
statistical
relationships; to
association,
assess how well one sometimes
variable predicts
among survey
another
responses
To explore causeeffect
What is
Weaknesses
Manipulated
Nothing
No control of
variables; single
cases may be
misleading
Nothing
Manipulate
The
one or more
independent
factors;
variable(s)
randomly
assign some to
control group
Does not specify
cause-effect; one
variable predicts
another but this
does not mean one
causes the other
Sometimes not
possible for
practical or ethical
reasons; results
may not
generalize to other
contexts
NEW SUBTOPIC:
VALIDITY
Modules 2 and 3: Research Methods
New major subtopic: Validity

Validity: degree to which correct inferences or
conclusions can be made (i.e., how true)
 External
validity: extent to which results can be
generalized to population
 Ensure
sample is representative of pop
 Random sampling is best
 Want to know that sample is like pop
Validity (cont’d)

Internal validity: extent to which diffs in DV are
due to diffs in IV and not some other confounding
variable
 What
do psych’ists hope to achieve in experiment?
 Results are due to IV and not something else
 When it is not the IV and is something else 
Threats to internal validity of study
 Subject
 There
characteristics
are diffs btn exper. and control grps that could
cause diffs in DV (instead of IV)
 Problem in selection and assignment of Ss
 Grps differ in unintended ways
Validity—Threats (cont’d)

Mortality
 Parts
of full results missing → incomplete data
 Ind drops out of study or does not answer all ?’s
 Limits ability to generalize → are Ss who dropped
out systematically diff from those who stay in study
(Hite’s women and sexuality study)

Location
 Where
study occurs influences results
 Situation-relevant confounding variable: situations
in which groups are placed should be equivalent.
Validity—Threats (cont’d)

Instrumentation
 Ways
measurements or procedures were done are
inaccurate
 Examples
 Is
method of measurement accurate? (flawed test)
 Does person collecting data interfere with accuracy?
(presence of observer)
 Does bias of collector influence data? (seeing results
as you would want them)
 Solutions
 Double-blind
procedure: neither participants nor
researcher knows who is in what group
Validity—Threats (cont’d)

Testing
 Experience
of taking pretest affects
performance on posttest
 Control grp helps to prevent this

History
 Something
occurs during course of experiment
that affected results
 More likely in longitudinal studies (longer period
of time)
Validity—Threats (cont’d)

Maturation
 Changes
in DV may be influenced in development
that occurs in Ss over time
 Usually with longitudinal studies

Attitude of Ss
 Subjects’
view of participation in study influences
results
 Two examples
 Hawthorne
effect: Ss’ knowledge or feeling special
causes positive results
 Placebo effect: respond b/c belief that med will
work
Validity—Threats (cont’d)

Regression threat
 Results
are due to statistical regression toward
average
 Statistical phenomenon more likely to occur with
extreme scores

Minimizing threats
 Standardize
conditions
 Obtain more info on Ss
 Obtain more info on details of study
 Use appropriate design
NEW SUBTOPIC:
ETHICS IN RESEARCH
Modules 2 and 3: Research Methods
New major subtopic: Ethics in research

Ethical guidelines in psychological research
 Major
consideration = DO NO HARM
 How is this assured?
 Informed
consent
 Confidentiality of data and results
 Minimize potential for harm in procedures
 Debrief after study
NEW SUBTOPIC:
STATISTICS
Modules 2 and 3: Research Methods
New major subtopic: Statistics

Statistics used in research for two
major reasons
Describing data – organize it in
meaningful way
Making inferences – how confidently
can we infer that observed difference
(results) accurately estimate true
difference
Statistics—Describing Data

Descriptive statistics: provide info
about distribution of scores
Frequency
distribution: how frequently
ea score occurs
N = number of scores
Statistics:
Descriptive Statistics (cont’d)

Measures of central tendency: attempt to
describe grp of scores / single score that
represents set of scores
Mean = mathematical average
(sensitive to extreme scores)
Median = score at which 50% of
scores fall above and 50% fall below
Mode = most frequently occurring
score
Statistics:
Descriptive Statistics (cont’d)

Measures of variability: measure of how
much scores are spread out among
distribution (how similar or diverse scores are)
 Range:
diff btn highest and lowest score
 Standard deviation: how scores vary around
mean / ave distance of ea score from mean
 Used in many other inferential stats
 Averages of data with lower variability more
reliable than averages of data with high
variablity
Statistics:
Descriptive Statistics (cont’d)

Standard deviation (same mean diff variability)
Statistics:
Descriptive Statistics (cont’d)

Normal distribution: distribution of scores
where mean, median, and mode are all
same score
 All
variables are assumed to distribute
themselves along normal dist.
 Percentiles: indicate % of scores that fall
below particular score
 AKA normal curve, bell curve
 Skewed distribution: when distribution is not
normally distributed
Statistics:
Descriptive Statistics (cont’d)
Skewed vs. Normal Distribution


Income distribution is skewed by the very rich.
Intelligence test distribution tends to form a
symmetric “bell” shape that is so typical that it is
called the normal curve.
Skewed distribution
Normal
curve
Statistics:
Descriptive Statistics (cont’d)

Correlation Coefficient: indicates strength and
direction of rel btn two variables
 Score
btn + or – 1.0
 Strength = how close number is to 1.0
 Direction = + or –
 Positive
correlation => both variables increase or
decrease together
 Negative correlation => when one increases,
other decreases
Correlation Coefficient



The correlation coefficient is a number representing the strength and direction of correlation.
The strength of the relationship refers to how close the dots are to a straight line, which means
one variable changes exactly as the other one does; this number varies from 0.00 to +/- 1.00.
The direction of the correlation can be positive (both variables increase together) or negative
(as one goes up, the other goes down).
Perfect
positive
correlation
+ 1.00
Guess the Correlation Coefficients
Perfect
negative
correlation
- 1.00
No
relationship,
no correlation
0.00
Statistics:
Descriptive Statistics (cont’d)

Correlation coefficient example
When scatterplots reveal correlations:
Height relates to shoe size, but does it also
correlate to “temperamental reactivity score”? A
table doesn’t show this, but the scatterplot does.
Correlation is not Causation!
“People who floss
more regularly have
less risk of heart
disease.”
If these data are from
a survey, can we
conclude that flossing
might prevent heart
disease? Or that
people with hearthealthy habits also
floss regularly?
Thinking critically:
If a low self-esteem test score “predicts” a
high depression score, what have we
confirmed?
 that low self-esteem causes or worsens
depression?
 that depression is bad for self-esteem?
 that low self-esteem may be part of the
definition of depression, and that we’re
not really connecting two different
variables at all?
If self-esteem correlates with depression,
there are still numerous possible causal links:
Statistics:
Inferential Statistics (new subtopic)



Inferential statistics: allow researcher to draw
conclusions from sample to pop / whether diffs btn
groups are meaningful
 If means of groups from sample reliable, then
differences are more likely reliable.
Why used?
 Is diff in DV btn control and exper. grp large enough to
mean something?
 Is diff in DV due to chance or IV?
When is observed difference considered more reliable?
 Representative sample (not extreme/unusual cases)
 Lower variability (versus > spread out)
 Greater number of subjects
Statistics:
Inferential Statistics (cont’d)


Examples
 t-test: when two variables are studied
 ANOVA: > two variables
Statistical significance: measure of probability that
results (diffs btn experimental and control) were due to
chance and not IV
 Sample averages reliable and when difference btn
them is large, then diff is statistically significant
 What is chance that difference btn groups is due to
chance?
 Read as likelihood results were due to chance
 Expressed in reports as…
(p < .01)
Statistics

Video
(http://www.learner.org/resources/series65.html?pop
=yes&pid=139)