Goals
3. Developing Ideas for
Research in Psychology
Chapter 3 in Goldstein
• Understand how to come up with research
ideas and assess their novelty:
• Sources of ideas for research
• Literature searching
• Types of research to consider
1
3.1 Ideas Old & New
2
Coming up with
Research Questions
• First, forget what you think you know
about “what psychologists study”
• Think of things that intrigue you, things that
frustrate you, things you wonder about
• Brainstorm: Let it all out.
3
4
Sources of
Research Ideas
•
•
•
•
•
Idea Generation
More Specific
General
Ideas
Ideas
Theories (“How do we test this theory?”)
Specific
Testable Hypothesis
Previous research (“What next?”)
Everyday observations (“Why do they do that?”)
Read Literature
Discuss with
colleagues & others
Applied problems (“How to solve this problem?”)
Serendipity (“Whoa! Why did that just happen?”)
Literature Search
6
5
Literature Search
•
• A useful skill in many
Spend some time on this
Literature Search
• What are some questions people are asking
about my topic?
• What do people already know about my
many areas
question?
• Essential for assessing
how novel your ideas are
7
• What methods have been used to explore
this question?
8
RE
DATABASE
SEARCH
Steps in Literature
Search
Decide on Basic
Search Question
OBTAIN
ARTICLES
Download
PDF
Decide on Basic
Concepts
Find synonyms
of concept terms
(psychinfo
thesaurus)
Find+Photocopy
Library
Hard-copy
Search for
individual concepts by “ORing”
Synonyms
See librarian
for hard-to-find
stuff
Skim, Read,
Take Notes
Search for
concepts combos
by “ANDing” Synonym groups
Search Articles’
Reference
Lists
Apply Limits
(subject group, age,
date, etc.)
Create
bibliography
(Ref Manager)
•
Decide on basic question:
“Does vaccination cause autism?”
•
Decide on concepts & determine related terms
(PsycInfo Thesaurus is useful here)
•
•
Autism: Asperger’s, Autism spectrum, etc.
Vaccination: Immunization, vaccine, etc.
Select articles
via titles and
abstracts
9
10
Steps in Literature
Search
Boolean Logic
•
Search for each individual concept by
combining related terms using “OR”
•
Search for combinations of concepts by using
“AND”
•
Apply limits and use “NOT” statements to
eliminate irrelevant sources
•
Manually search articles by titles and abstracts
for relevancy
11
• OR: Combine two groups to create a larger
group
• AND: Find overlapping area of two groups
• NOT: Find area of one group that does not
overlap another group
12
Boolean Logic
• Who in the class is blond OR brunette?
• Who in the class is blond AND female?
• Who in the class is blond AND female,
All articles
Autism
Articles about autism
OR immunization
Immunization
NOT wearing glasses?
13
14
All articles
All articles
Articles about autism
AND immunization
Autism
Immunization
Autism
Immunization
Empirical
Studies
15
Articles about autism
& immunization that
are also empirical
16
Explode & Focus
All articles
Autism
Immunization
Asperger’s
Syndrome
•
In psychinfo, “Explode” will make for a more
general search.
•
“Focus” does the opposite, only returning articles
where X is the main topic
•
“Search for X as Keyword” can be useful for
finding articles containing specific words within the
title or abstract
•
To cast a broad net, use both explode and
keyword
17
18
Obtaining Articles
Mining Articles For
Articles
Downloading PDFs: PsycInfo has a “Get It” button.
Try this first.
•
•
Articles about autism
& immunization that
are NOT about
Asperger Syndrome
•
Next check “eJournals” (on library main page). They
sometimes have access to an article via this route
even when “Get It” says no.
If all else fails, go to reference librarian and get help.
They can find anything! (given time)
19
• Look into the Past: Search the references of
the articles you find in order to find other
relevant articles
• Look into the Future: Use a citation search
to find later articles that cite the ones you
find
• PsycInfo’s “Find Citing Articles”
20
Literature Searching
Is Iterative
•
Found articles may lead to new searches, or
may even change your question
•
Be prepared to “try try again”.
• Too few articles? Consider that you may
have missed some related terms.
How to Read Journal
Articles
• Slowly. This is not a novel. Information is
dense and specific
• Skim first, get the gist.
• Then, take notes while you read.
Otherwise, you’re wasting your time.
• Keep a bibliography as you go.
21
22
Reference Manager
Software
Searching for
Questionnaires/Instruments
•
Very useful stuff! Keeps a list of references for
easy use later. Also keep your notes here.
•
•
Use PsychInfo, apply the limit “Tests and Measures”
•
EndNote: Works well with MS Word. Cite-whileyou-write feature is great.
•
•
•
Many instruments are proprietary and cost money
to get copies (e.g. BDI, WAIS, WISC, etc.).
RefWorks: Free and supported by the library.
•
You need not have a copy of an instrument to
include in your research proposal.You do need a
reference to the original article presenting it and a
detailed description of the instrument.
Zotero: Free, works really well, but not
compatible with library’s Ovid pages.
23
May have to dig to find original article presenting
the measure (many more articles using it...)
24
Searching for
Questionnaires/Instruments
Questions
• What does it mean to say that literature
•
•
searching is iterative?
Example 1: Search for specific instrument, the CES-D
Example 2: Search for instrument measuring a given
characteristic: Narcissism
• Can you describe the effect of using AND,
OR, and NOT in searches?
• What is typically the main source of ideas
for research questions?
25
3.2 Types of Research
27
26
Types of Research
• Consider all of these for your proposal:
• Basic vs. Applied
• Lab vs. Field
• Quantitative vs. Qualitative
• Note that these are false dichotomies
• Multiple approaches can be beneficial
28
Basic vs. Applied
•
Basic Research
Basic: Aims to figure out fundamental
principles underlying behaviour
• Applied: Aims to solve some existing welldefined practical problem
•
Not immediately practical,
looks at fundamentals.
•
Criticism of basic research:
“...still no cure for cancer”
•
Applied solutions are often
impossible without
fundamental knowledge
29
Interactions of
Basic & Applied
Applied Research
•
•
• Not mutually exclusive
• Researchers are often led from basic to
Aimed directly at practical problems.
•
“What’s a good way of treating depression?”
applied work or vice versa.
Criticism of applied research come mainly from
researchers, not the public:
•
What does this really tell us about what’s
going on “under the hood”?
•
“Do you really understand why it works?”
30
• Basic work “pushes” applied work, applied
work “pulls” basic.
• Basic is long-term, applied short-term.
31
32
Lab vs. Field Work
Lab vs. Field Work
• Lab work usually (not always) basic
• Field work usually (not always) applied
• Is lab research too artificial?
• Does field work lack rigour? How do you
•
•
Trade-off between external & internal validity
•
Internal validity: Degree to which variables are
isolated so that firm conclusions can be drawn
from your research.
External validity: The degree to which your
research applies to the real world.
•
isolate your variables?
Consider degree of internal and external validity
in the studies you read & propose.
33
34
External vs. Internal Validity:
Related Terms
Lab vs. Field Work:
Why Choose Just One?
•
• External validity: Ecological validity,
generalizability, realism.
Dutton & Aron study on misattribution of physiological
arousal
•
•
• Internal validity: Experimental control,
interpretability, scientific rigour.
•
35
Field work on bridges
Lab work at UBC
Complementary strengths of
two approaches lead to better
evidence
36
Quantitative Research
•
Qualitative Research
•
Quantitative: “Having to do with numbers”.
•
•
•
Typical of natural sciences research
Tends to provide better internal validity
Cannot address certain questions adequately
Findings described not in numbers but in analytical text.
•
•
Typical of social sciences & humanities research
•
Commonly done in case studies, ethnographies,
naturalistic observations, grounded theory studies, etc.
Data may be in the form of interviews, texts, images,
films, etc.
37
Examples of
Operational Definitions
Operational Definition
•
•
Construct
Measure
Behaviours
Anger
Blood pressure
Expressed responses
Anger
Self-rating on 1-10 scale
Math phobia
Tendency to choose non-math tasks,
even when harder than math ones
Prejudice Against
African Americans
Change in % of callbacks when name on
CV is changed to AA name
Love of Ice Cream
$ participant is willing to pay
Linking of a concrete measure
to an abstract construct
•
•
•
•
38
Physiological activity
etc., etc., etc...
Creativity is required to come
up with good ones
Beauty-A
Karin Kuhlmann, Germany
39
40
What Makes a good
Operational Definition?
Validity vs. Reliability:
Should you look where the light is?
• Lends itself to systematic, replicable
observation (i.e., data gathering)
• Has good validity: Genuinely measures
what you want it to measure
• Has good reliability: Gives the same
measurement value on different occasions
41
Validity vs. Reliability:
Should you look where the light is?
•
“Because we cannot measure what we value, we
begin to value what we measure.”
•
Validity is more important than reliability! If
you’re not measuring what you want to
measure, what’s the point?
•
Avoid the tendency to measure what’s easily
(reliably/objectively/impartially/systematically)
measurable rather than what’s important to
measure.
42
Construct Definition
In order to create good OD, must first
have well-defined construct.
What kind of depression do you want to measure?
What aspects of prejudice are relevant to your theory?
What do we mean by anger, anyway?
Some people confuse construct definition
with operational definition, but they are not
the same
43
Antonio Bassi
http://tinyurl.com/22ml2nl
44
3.3 Theory & Evidence
(not proof)
Questions
• Why is validity more important than
reliability? Can you give an example?
• What are the three pairs of types of
research we discussed?
• What does it mean to say these pairs
represent “false dichotomies”? Can you give
an example?
No amount of experimentation can ever prove me right;
a single experiment can prove me wrong. -Einstein
45
Theories
Theories
Theories
Theories
Theories
Theories
Theories
Theories
Theories
Theories
Theories
Theories
Theories
Theories
are
are
are
are
are
are
are
are
are
are
are
are
are
are
never
never
never
never
never
never
never
never
never
never
never
never
never
never
46
Theories
Theories
Theories
Theories
Theories
Theories
Theories
Theories
Theories
Theories
Theories
Theories
Theories
Theor
proven.
proven.
proven.
proven.
proven.
proven.
proven.
proven.
proven.
proven.
proven.
proven.
proven.
proven.
47
never
never
never
never
never
never
never
never
never
never
never
never
never
become
become
become
become
become
become
become
become
become
become
become
become
become
facts
facts
facts
facts
facts
facts
facts
facts
facts
facts
facts
facts
facts
What A Theory Does
• “A working truth”
• Summarizes existing knowledge
• Describes relationship among observations
• Tentatively explains why these relationships
exist
• Enables hypotheses/predictions to be
generated and tested
48
Predictive Power and
Productivity
•
There is no known systematic way to generate
theory from data, it is a creative process.
•
•
Large-scale theories tend to have low
predictive power for complex systems (e.g.,
people), but high productivity.
Small-scale theories can have higher predictive
power, but lower productivity
49
Erikson's stages of psychosocial development
•
•
Generated via a great deal of careful and creative thought
•
•
Difficult to make predictions from such a large-scale theory.
Based on personal and clinical experience, discussions,
introspection, etc.
However, many specific hypotheses and “sub theories” have
emerged from it
50
STAGE
PSYCHOSOCIAL
CRISIS
Infancy
(0-12 months)
Trust vs. Mistrust
Toddler Stage
(1-3 Years)
Autonomy vs. Doubt
Early Childhood
(3-5 Years)
Initiative vs. Guilt
Mid Childhood
(6-10 Years)
Industry vs. Inferiority
Adolescence
(11-18 years)
Identity vs. Confusion
Early Adulthood
(18-34 Years)
Intimacy vs. Isolation
Middle Adulthood
(35-60 Years)
Generativity vs.
Stagnation
Later Adulthood
(60+ Years)
Integration vs. Despair
51
Large-Scale Theory:
Smaller-Scale Theory:
Seligmanʼs Learned Helplessness Theory
• Theory: When there is no contingency between
responses and outcomes, organisms stop trying
to control their situation
• Serendipity: Seligman generated theory due to
another research team’s surprising results
regarding dogs trained to avoid lights
52
Learned Helplessness
Theory
Stephen’s Power Law
• Summarizes existing data: Dogs trained to fear shock in a
How bright will a light of given
physical intensity look to people?
“Pavlovian Hammock” don’t avoid shock.
• Describes relationship between observations:
• Contingency = yes ∴ Control attempts = yes
• Contingency = no ∴ Control attempts = no
• Provides explanation of why the relationship exists: Dogs
B = K * IN
B is subjective brightness rating
K is a free parameter constant.
N is an exponent that depends on
sensory modality. (.5 for brightness.
form cognitive representation of contingency that guides later
behaviour.
• Allows Predictions: People may respond the same way,
Very specific theory with high
predictive power, but limited in
scope.
possibly explaining depression
54
53
Theory and Data
Proof and Disproof
•
Data can precede theory: LHT was developed from data
from an unrelated experiment.
• Theories can never be proven they can only be
•
Data can follow theory to test if its predictions are accurate:
Later experiments showed a relationship between
uncontrollable trauma and depression.
•
•
If data fits hypothesis, theory is supported inductively
“not disproven so far”
• Fallacy: Affirming the consequent.
• Correct: “modus tolens” (denying mode)
• In practice, aggregation of data leads to support or
abandonment/modification of theories.
If data do not fit hypothesis, they fail to support theory
55
56
If I water My Lawn,
Then My Lawn Will Be Wet
Proposition
If Theory X is True,
Then Y will be observed
Valid?
Proposition
Valid?
I watered my lawn, therefore it must be wet
Yes
“X is true, therefore Y will be observed”
Yes
My lawn is wet, therefore I watered it
No
“Y was observed, therefore X must be true”
No
I didn’t water my lawn, therefore it’s not wet
No
“X is not true, therefore Y won’t be observed”
No
My lawn isn’t wet, therefore I didn’t water it
Yes
“Y was not observed, therefore X isn’t true”
Yes
57
58
Falsifiability
Summary:
Creativity in Science
•
Many think scientists seek “perfect” theories that
explain every possible outcome.
•
But in fact, a theory that explains every conceivable
outcome is useless. It predicts anything, therefore
predicts nothing.
•
Theories must, at least in principle, be falsifiable
•
59
Creativity is needed in psych research to come up with
(among other things):
•
•
Theories: Abstract / Lateral / Syncretic thinking
•
Research methods: Measuring abstract constructs
(pragmatically and ethically) is difficult
Research questions: How to go from broad theory to
tractable hypothesis?
60
Questions
• Greg believes that all swans are white. He
observes a white swan and decides that he
is correct. Is that valid?
• Which is better, a large-scale theory or a
small-scale one?
61