The research process
Psych 231: Research
Methods in Psychology

This is the final lecture


Wednesday (optional) – Review Q&A, help
with posters, etc.
Labs this week



Poster presentations
Turn in group ratings sheets
Turn in the GP: results and discussion
sections
Announcements

Sometimes you just can’t perform a fully controlled
experiment


Because of the issue of interest
Limited resources (not enough subjects, observations are too
costly, etc).
• Surveys
• Correlational
• Quasi-Experiments
• Developmental designs
• Small-N designs

This does NOT imply that they are bad designs

Just remember the advantages and disadvantages of each
Non-Experimental designs

What are they?

In contrast to Large N-designs (comparing
aggregated performance of large groups of
participants)
• One or a few participants
• Data are typically not analyzed statistically; rather rely on
visual interpretation of the data

Historically, these were the typical kind of design
used until 1920’s when there was a shift to using
larger sample sizes
• Even today, in some sub-areas, using small N designs is
common place
• e.g., psychophysics, clinical settings, animal studies, expertise, etc.
Small N designs

Small vs. Large N debate

Some researchers have argued that Small N designs
are the best way to go.
• The goal of psychology is to describe behavior of an
individual
• Looking at data collapsed over groups “looks” in the wrong
place
• Need to look at the data at the level of the individual
• For more see: Poteete & Ostrom (2008)
Small N designs
= observation
Steady state (baseline)
Treatment
introduced
Baseline experiments – the basic idea is to show:


Observations begin in the absence of treatment (BASELINE)
• Essentially our control/comparison level

Then treatment is implemented and changes in frequency,
magnitude, or intensity of behavior are recorded
Small N designs
= observation
Transition
steady state
Reversibility
Steady state (baseline)
Treatment
introduced
Treatment
removed
Baseline experiments – the basic idea is to show:



When the IV occurs, you get the effect
When the IV doesn’t occur, you don’t get the effect
(reversibility)
 This allows other comparisons, to the original baseline as
well as to the transition steady state
Small N designs
Unstable

Stable
Before introducing treatment (IV), baseline needs
to be stable
Measure level and trend


Level – how frequent (how intense) is behavior?
• Are all the data points high or low?

Trend – does behavior seem to increase (or decrease)
• Are data points “flat” or on a slope?
Small N designs
Steady state (baseline) Transition steady state

Reversibility
ABA design (baseline, treatment, baseline)
– The reversibility is necessary, otherwise
something else may have caused the effect
other than the IV (e.g., history, maturation, etc.)

There are other designs as well (e.g., ABAB see
figure13.6 in your textbook)
ABA design

Advantages





Focus on individual performance, not fooled by
group averaging effects
Focus is on big effects (small effects typically
can’t be seen without using large groups)
Avoid some ethical problems – e.g., with nontreatments
Allows to look at unusual (and rare) types of
subjects (e.g., case studies of amnesics, experts
vs. novices)
Often used to supplement large N studies, with
more observations on fewer subjects
Small N designs

Disadvantages



Difficult to determine how generalizable the effects
are
Effects may be small relative to variability of situation
so NEED more observation
Some effects are by definition between subjects
• Treatment leads to a lasting change, so you don’t get
reversals
Small N designs
From Day 1
Presenting your work
Get an idea



A set of skills leading to knowledge & understanding
A way of thinking (beware small samples, correlation is not causation, etc.)
A way of life?
Stats.org: Stats in the news
Course Review: The Research Process

Get an idea


Often the hardest part
No firm rules for how to do this
• Observations
• Past research
• Review the literature
The Research Process

Review the literature

What has already been done?
• What variables have people looked at
• What hasn’t been looked at


How are other experiments in the
area done?
What methods are used?
• To measure the dependent variable
• To manipulate the independent
variable
• To control extraneous variables
The Research Process

Formulate a testable hypothesis

What is a hypothesis?
• A predicted relationship between variables

What does it mean to be testable?
• Must be falsifiable
• Can it be replicated
• Must be able to observe/measure (and manipulate for
experiments) the variables
• Directly
• Indirectly
• Operational definitions
The Research Process
Design the research

What method?

•
•
•
•
What are your variables?

•
•

Experiment, Survey, Developmental designs, …
What kind of comparisons are used
Control groups
Baseline conditions
How many levels of your Independent variable(s)
How do you measure your dependent variable(s)
What can be done to control for biases and
confounds?
The Research Process

Collect Data


Importance of pilot research
Who do you test?
• What is your population?
• Your sample?
• Your sampling method?
The Research Process

Analyze the data


Design drives the statistics
Understanding Variables and variability
• Descriptive statistics (summarizing)
• Means, standard deviations
• Graphs, tables
• Correlation
• Inferential statistics (drawing conclusions)
•
•
•
•
What kind of analysis is appropriate for your design
T-tests
ANOVA
Between or within versions
The Research Process

Interpret the results





Correlation versus causation
Reject or fail to reject null hypotheses
Statistical vs. theoretical significance
Support or refute the theory (or revise)
Generalizability of the results
The Research Process

Present the results

Getting the research “out there”
• Conference presentations
• Posters
• Talks
• Written reports
• APA style
• Supports clarity
The Research Process

Repeat

Each set of results leads to more research
questions
• Refine the theory
• Test a refined theory
• Test alternative explanations
The Research Process

Monday Dec. 7 @ 7:50A-9:50 DEG 206

It is cumulative, covers the entire course. The
majority is on new material (roughly 65%), the
rest is material covered on Exams 1 & 2.
All multiple choice/scantron for the final

Reviewing for the final exam




Final 1/3 of the course
Non experimental methods
 Survey, correlational, & developmental
Statistics
 Descriptive
 Inferential
Presentations
 Papers, Posters, & Talks
Reviewing for the final exam

First 2/3 of the course
 Scientific method









Getting ideas
Developing (good) theories
Reviewing the literature
Psychological Science
Ethics
Basic methodologies
APA style
 Underlying reasons for the
organization
 Parts of a manuscript


Variables
Sampling
Control
Experimental
Designs




Vocabulary
Single factor designs
Between & Within
Factorial designs
Reviewing for the final exam