Science and Psychology
Psych 395 - DeShon
Card Selection Task (Wason,1966)


4 Cards (letter/number)
Rule:
–
If a card has a vowel on one side, then it has an
even number on the other side
E

K
4
7
Which card or cards do you have to flip to
determine whether the rule is violated?
Answer

Pick These Cards:
E
7
Let’s get a little symbolic



If a card has a vowel on one side, then it has
an even number on the other side
If P then Q
Becomes a problem in deductive logic…
Answers





Flip E card: If P then Q means…if P then Q.
(Modus Ponens)
Flip 7 card: If P then Q means…if not Q then
not P. (Modus Tollens)
Flip K card: Error – Irrelevant to Argument
Flip 4 card: Error – Rule doesn’t say that Q
cannot occur in the absence of P.
Only About 10% of participants opt for
both E and 7.
Implication




Humans are not naturally good at all aspects
of formal logic. (Scientists included)
Scientific methods exist to prevent scientists
from deceiving themselves (e.g., Levine &
Parkinson, 1994).
The nature of Science is the focus of today’s
lecture
But first an aside on the subjectivity of
science…
Popper (1968, p. 51)

“There is no more rational procedure than
the method of trial and error – of conjecture
and refutation: of boldly proposing theories;
of trying our best to show that these are
erroneous; and of accepting them tentatively
if our critical efforts are unsuccessful.”
Deductive Reasoning and
Scientific Reasoning
Deductively Valid Arguments


Definition: Arguments where the truth of the
premises guarantees the truth of the
conclusion.
Consider Two Forms:
–
–
Modus Ponens (Mode that Affirms)
Modus Tollens (Mode that Denies)
Modus Ponens

Formally:
–
–
–

Premise 1: If P then Q.
Premise 2: P
Conclusion: Therefore, Q
In words:
–
–
–
If an animal is pig then the animal oinks.
Wilber is a pig.
Therefore, Wilber oinks.
Modus Tollens

Formally:
–
–
–

Premise 1: If P then Q.
Premise 2: Not Q
Conclusion: Therefore, not P
In words:
–
–
–
If an animal is pig then the animal oinks.
Wilber does not oink.
Therefore, Wilber is not a pig.
Is This How It Works in Science?




Theory: All pigs oink.
Make some observations about a newly
discovered species.
Observe Q - These animals oink.
Conclude that P is True. These new animals
are pigs.
Fallacy of Affirming the Consequent




In reality, other animals besides pigs may
also oink!
So this method of reasoning is not a path to
scientific progress.
Basic Idea: We CANNOT prove that scientific
theories are TRUE. (Sir Karl Popper)
Scientific Reasoning is Not About
Confirmation. Science is about Refutation.
Scientific Reasoning - Refutation




We can show that the consequence of a theory are
not empirically supported. Modus Tollens.
According to Popper, falsification is the line of
demarcation between science and non-science.
If we observe “Not Q” then we can conclude that P is
False.
If the new animals don’t oink then they are not pigs.
Sounds Easy Enough, But…



No theory is tested in Isolation (Duhem-Quine
Thesis)
There are lots of auxiliary theories involved in
conducting a scientific study.
– For example, we assume that our instruments are
good indicators of the constructs under
investigation.
– This is why we spend so much time on
measurement in 395.
We don’t know what we have precisely falsified
when we observe “NOT Q” – the Big P theory or one
of the auxiliary theories.
Fundamental Inferences
Indicator of IV
(Observed Relation)
IV Construct
Indicator of DV
DV Construct
(Real Relation)
Example




Strain Theory in Sociology (Merton, 1938) –
Social Strain Produces Crime.
Possible Prediction: Blocked Opportunities
should Correlate with Crime.
Observation: Perceptions of blocked
Opportunities are not correlated with arrest
records.
What do we conclude?
Fundamental Inferences
Self-Reports of
Blocked Opps
(Observed)
Strain
Official Arrests
Crime
(Positive Relation)
Deduction
Prediction
from Theory
Error!
Findings
Revision of
Theory
Induction
Feedback Loop Modified from Figure 2A in Box (1976, p. 791).
Separate Justification from Criticism
(e.g., Bartley, 1984)



Searching for complete justification is not
going to get us very far.
Instead, let’s acknowledge that there is a
continuum from speculation to wellestablished knowledge.
Argue about the reasonableness and degree
of empirical support for particular
propositions.
For Psychological Researchers…



Specify models, test models, revise models…
Researchers must be transparent so that their work
can be subject to maximum criticism.
Full disclose of what was done and what was found.
Accuracy and clarity are paramount virtues in the
reporting of psychological research.
– Hence, good writing is important.
Further Thoughts on Science and
Models
Just a Few Reasons Why Psychology
is Hard (e.g., Lykken, 1991; Meehl,
1978)



Everybody is an armchair psychologist. This
provides resistance to simplification.
Experimental control is difficult.
Psychology seeks to understand one of the
most complicated “machines” in the known
universe – human brains which give rise to
thoughts, feelings, and behaviors.
Pragmatic Viewpoint

“Every scientist in the back of [her] mind takes it for
granted that even the best theory is likely to be an
approximation to the true state of affairs”
–

Paul Meehl (1990, p. 113)
“Models, of course, are never true, but fortunately it
is only necessary that they be useful. For this it is
usually needful only that they not be grossly wrong.”
–
George E. P. Box (1979, p. 2)
More Thoughts on Models…


“The great advantage of the model-based
approach over the ad hoc approach, it seems
to me, is that at any given time we know what
we are doing.” (Box, 1979, p.2)
Since all models are wrong the scientist must
be alert to what is importantly wrong. It is
inappropriate to be concerned with mice
when there are tigers [around]. (Box, 1976,
p. 792).
What to Do…



Appreciate how science really operates.
Favor theories that make risky predictions
and survive rigorous test of those theories
(e.g., Meehl, 1978, 1990).
Think Beyond One Study to Multiple Studies
-- Focus on Replication (Lykken, 1968)
Importance of Replication

Scientists have known for centuries that a
single study will not resolve a major issue.
Indeed, a small sample study will not even
resolve a minor issue. Thus, the foundation
of science is the culmination of knowledge
from the results of many studies.

Source: Hunter, Schmidt, & Jackson (1982, p. 10)
Causality
Science vs. Societal Values


The history of science reflects a clash between what
we currently believe to be true and the new “truths”
we discover about the world.
Virtually everything we once “knew to be true” has
been shown to be false.
–
–
–
–
–
Magic
Geocentric models of the universe
Flat earth
Medicine (blood letting, arsenic, etc.)
Newtonian Physics?
Example: Rind et al (1998)



Societal Belief: Child Sexual Abuse is wrong
and causes substantial harm and results in
negative outcomes in adults.
Rind et al (1998) performed a meta-analysis
(a statistical summary) of the existing
literature and did not find support for the
claim that CSA results in substantial negative
long-term outcomes in adults
< 1% of variance in outcomes
Huge public and political response






Lilienfeld (2002) documents the response
Nearly caused the downfall of the APA
Dr. Laura condemned the research
Religious groups united against the research
Politicians with no scientific training began
criticizing the study
APA caved to political and public pressure
and renounced the study
Aftermath





Public policy review initiated at APA
Congress voted 355 to 0 to condemn and denounce
the findings
Senate passed the resolution the same month
Scientific review panels found no serious fault with
the study
Highlights a huge gap between popular press and
academic science
Some Terminology




Variables: Any attribute that changes values across
things that are being studied.
Hypothesis: A testable statement describing the
relation between two or more constructs.
Constructs: Abstract qualities that we attempt to
measure.
Operational Definition: Statement of the process we
use to measure constructs.
–
Constructs  Actual Measures in a given study
Operational Definitions

Going from the Unobserved to the
Observed…
Measured
Variables
x
y
Unobservable
Constructs
X
Y
Scientific Approach to Causation

“Development of Western science is based
on two great achievements: the invention of
the formal logical system (in Euclidean
geometry) by the Greek philosophers, and
the discovery of the possibility to find out
causal relationships by systematic
experiment (during the Renaissance).”
–
Albert Einstein (1953)
Causal Relations


John Stuart Mill (1806-1873)
Three Conditions for Cause and Effect
Relations.
–
–
–
Temporal Ordering (Cause Preceded the Effect)
Covariation (Cause and Effect are Associated)
Rule Out Third Variables (No Plausible
Explanation for the Effect Other than the Cause)
Newton’s 4 Rules of Scientific
Reasoning (Principia, 1680)
Admit no more causes of things than are both true
and sufficient to explain their appearance.
1.
–
–
–
–
Occam’s razor
Rule of Parsimony
Sherlock Holmes
The simplest theory that fits the facts of a problem is the
one that should be selected.
Example: Geocentric view of solar system
Example: Crop circles
Newton’s 4 rules
To the same natural effects we must, as far as
possible, assign the same causes.
2.
–
–
Generality
Example: Gravity and falling objects
The qualities of bodies which are found to belong
to all bodies within the reach of experimentation
should be assumed to be universal qualities of all
bodies
3.
–
–
Generalization
Example: Gravity on other planets
Newton’s 4 rules
Theories obtained from experiments should be
considered true until new experimental evidence
shows the old to be incorrect.
4.
–
Values should not lead to changes in theories
–
The history of science represents a continuous
struggle between existing values and empirical
data (new and old beliefs)
Experimental Approach to Studying
Causal Relations




Intervention, Observation, and Control
Random assignment of participants to two or
more conditions of an experiment.
The Independent variable is the variable that
is manipulated or the experimental conditions
(e.g., treatment or no-treatment).
The Dependent variable is the outcome of
interest in the particular study.
Where do research ideas
come from?

Observe your world!
–
–
–
–
–
–
–

Why do some people work so hard?
Do social systems function like ant colonies?
Why is karaoke so popular?
Why do people sacrifice long term gain for short term
pleasure (delay of gratification)?
What is boredom?
What do individuals get out of belonging to a counterculture?
Why do people collect stuff?
Ask questions about your own behavior!
–
Why did I do that?
Bathroom Behavior
Personal Space Invasion in
the Lavatory: Suggestive
Evidence for Arousal Middlemist et al. (1976)
Invasions of Privacy are Bad
Do Space Violations Cause Arousal?
 Theory: Space Violations  Arousal 
Responses to Personal Space Violation
 Where can we study personal space
violations?

The Men’s Room….
Let’s Put this in Scientific Terms…



“Urinals are open and placed side by side so that,
under crowded conditions, men stand shoulder to
shoulder, coactively engaging in private elimination”
(Middlemist et al., 1976, p. 542)
Conducted research on micturation – “a process
sensitive to arousal”
“Fright and embarrassment inhibited relaxation of the
external sphincter of the urethra” (Middlemist et al.,
1976, p. 542)
Hypothesis (Middlemist, et al., p. 542)


“If personal space invasions produce
arousal, then subjects standing closest to
others … would show increases in the delay
of onset of micturation and decreases in the
persistence of micturation”
Translation:
Design

60 Men randomly assigned to one of three
conditions:
–
–
–


Close Distance (16 to 18 inches)
Moderate Distance (52 to 54 inches)
Control (Peeing Solo)
An observer hides in toilet stall and makes a “direct
visual sighting of the stream of urine”
“Subjects were not informed that they had
participated in an experiment”
Close Distance Condition
Moderate Distance
Results – Delay of Micturation
10
8
6
Control
Moderate
Close
4
2
0
Delay
Results – Persistence of Micturation
25
20
15
Control
Moderate
Close
10
5
0
Persistence
Rephrase these Results…
Conclusions (Middlemist et al., p. 545)





Micturation delay and persistence were sensitive to
situational differences.
The pattern was consistent with arousal theory
Space Violations  Arousal
Q: How was arousal measured?
“The present study implies that [delay and
persistence] have some construct validity as
indicators of arousal” (p. 545)
Let’s Answer Some Key Questions






What was the hypothesis?
What was the Independent Variable?
What were the Dependent Variables?
What were the Operational Definitions?
What were the Results?
What was their Interpretation of the Results?
A Measurement Issue with
Experiments



It seems straightforward enough –
manipulate an IV and measure the effects on
a DV.
But what exactly did we manipulate? What
exactly causes the observed responses?
We know what we manipulated at the
operational level. How do we know what
psychological construct was actually
manipulated?
Scientific Research Process
1
8
Develop a Research Idea
9
Evaluate the Data
- Identify an interesting question
- Review the relevant literature
2
- Statistical Analysis
7
Convert idea into a
hypothesis
- modify or expand idea
10
Conduct the study
- Collect the data!
- Input and organize data
- Specific, testable proposition
3
Refine Idea
Report the Results
- APA Format
- Accurate picture
6
Define & Measure Variables
Select a Research Design
- Determine measurement process
- Evaluate reliability of measures
- Evaluate validity of measures
- Between Subjects vs. within-subjects
- Factorial
- Covariates
5
4
Identify Participants
- How many?
- What population?
- Sampling design
- Ethical treatment issues
Select a Research Strategy
- Internal vs. external validity
- Experimental vs. Quasi-experimental vs. Observational