Variations on Scientific
Methodology
1. Observe some regularity
2. Hypothesize an explanation (cause?)
3. Test that hypothesis via experiment &
observation
4. Refine hypothesis & start over with (1)
Hypotheses?
Observation as tool for establishing
regularities & delineating phenomena
What is Correlation?
Simpsons
How do we establishing a correlation:
Intervention (Experimentation)
Non-Intervention (Observation)
Delineating Phenomena w/in
‘Memory’
Raise your hand if you had:
•
•
•
•
•
•
•
•
BAG
DOG
FAN
GAS
HAT
KID
LOG
PAD
•
•
•
•
SOD
VEX
WIN
ZIP
Characteristics of
Observational Research
• Make some sort of record and analyze
data obtained from it
• Does NOT manipulate or ‘intervene’ in
the scenario.
Naturalistic Observation
• Observations made in the ‘natural’ setting of
the organism.
• Researchers must immerse themselves in the
setting.
• Task: to describe the setting, events,
individuals observed w/out influencing the
situation
• Often Qualitative, not Quantitative
• Often NOT a matter of testing a hypothesis,
but rather gathering data to develop a
testable hypothesis.
Data
• Field notes, journal entries, interviews,
recording ‘artifacts’
Famous Naturalistic
Observations
• Jane Goodall
• Charles Darwin
• Survivor?
Goodall
• ‘Termite fishing’ and tool-use
• Click Here
Darwin
•
Sexual dimorphism is caused by three
possible mechanisms:
1. mechanisms of sexual selection,
2. fecundity selection
3. ecological causation, e.g., resourcepartitioning
Darwin confirmed
Naturalistic Study 1
•
•
•
•
•
•
Marmots
Cows
Horses
Hare
Squirrel
Deer
Problems for Systematic
Observation
•
•
•
•
•
Equipment (Nielson ratings)
Reactivity of subjects
Reliability (Mate selection in Blue Tit)
Sampling
Confirmation Bias
Complexity of Observation
• Expectations & Perception
– Anomalous playing cards
– Multi-modal feedback (Data from lyric
study)
– Underdetermined Perception
• Influence of early hypotheses
– Ratman study data
• Extending Perception w/ Instruments
– Galileo / Golgi (carl) / Hale-Bopp
Anomalous Playing Cards
• Link
Lyrics Study
• Wash U
• UCSD
Ramones Correct
Ramones Incorrect
Systematic Observation
• Careful, often quantitative, observations of
one or more specific behaviors.
• Observations made in ‘quasi-natural’ setting
• Researchers often do NOT immerse
themselves in the setting
• Quantitative, not qualitative (using coding
systems)
• Often a matter of testing a hypothesis
Coding Systems
• Marmots 2
Sampling
• Continuous
• Time Sampling
• Event Sampling
Correlation
Final Score & Post Test v Reading Report
35
30
Final Score
25
20
Post Test Score
Final Score
15
10
5
0
0
50
100
150
Number of modules read
200
250
300
Final Grade v. Questions
Final Score & Post Test v Question Report
35
30
Final Score
25
20
Post Test Score
Final Score
15
10
5
0
0
200
400
600
800
Number of Question Attempts
1000
1200
1400
Final Grade v. Post Test
PostTest v. Final
35
30
25
Final
20
15
10
5
0
0
2
4
6
8
10
Post Test
12
14
16
18
Sleep Study @ UCSD
Establishing Correlation
1. Intervention (experimentation)
2. Non-Intervention (observation)
Vitamin E!
Types of Variables (review)
• Independent (manipulated)
a variable of interest that is suspected to have a causal
impact on the dependent variable, and is manipulated in
an experiment to put this to the test.
• Dependent
a variable of interest that is suspected to be affected by
the independent variable, and is measured after (and
possibly before) the independent variable is manipulated
to put this to the test.
Types of Variables (2)
• Controlled
Confound variables that have been dealt with: extraneous
variables in the same system that may be causally related
to the IV or DV and therefore must be controlled.
– Procedural
most often controlled by constancy
– Subject
most often controlled by randomization
Setting up a study
• Consider the following:
Research at the University of Pennsylvania and the Children’s Hospital of
Philadelphia indicates that children who sleep in a dimly lighted room
until age two may be up to five times more likely to develop myopia
(nearsightedness) when they grow up.
The researchers asked the parents of children who had been patients at
the researcher’s eye clinic to recall the lighting conditions in the
children’s bedroom from birth to age two.
Of a total of 172 children who slept in darkness, 10 percent were
nearsighted. Of a total of 232 children who slept with a night light, 34
percent where nearsighted. Of a total of 75 who slept with a lamp on,
55 percent were nearsighted.
The lead ophthalmologist, D. Graham E. Quinn, said that, “just as the body
needs to rest, this suggests that the eyes need a period of darkness”
Operationalize the Variables
Operationalization Part 2
Assign possible values to the variables:
Explanation 1:
A Direct Causal Link
Internal Validity and Confounding
Variables.
• Internal Validity is the extent to which the study’s
design ensures that its results correspond to
reality.
– In other words, the experiment is internally valid if
there are no confounding variables that may explain
the correlation of the IV and DV.
A Confounding Variable is one that covaries with the IV or
DV.
Experimental design controls
confounding variables.
• The goal of an experiment is to show that the
only possible cause of the values of the DV is
the IV:
New studies reported in the Journal of the American Medical Association indicate
that vasectomy is safe. A group headed by Frank Massey of UCLA paired
10,500 vasectomized me with a like number of men who had not had the
operation. The average follow-up time was 7.9 years, and 2,300 pairs were
followed for more than a decade. The researchers reported that, aside from
inflammation of the testes, the incidence of diseases for vasectomized men was
similar to that in their paired controls.
A second study done under federal sponsorship at the Battelle Human Affairs
Research Centers in Seattle compared heard disease in 1,400 vasectomized
me and 3,600 men who had not had the operation. Over an average follow-up
time of fifteen years, the incidence of heart diseases was the same among men
in both groups.
A Confounding Variable?
Controls: Setting up the groups
• First, we must create a control group:
– there must be some kind of comparison condition
that will enable us to say that the DV depends
solely on the IV.
Poor Design as a Result of Poor
Control Group
• Fallacy 1: No Control Group
• Fallacy 2: Nonequivalent Control Groups.
• Pre-Test Post-Test Pitfalls:
History
Maturation
Testing
Instrument
Decay
Statistical Regression
Independent Groups Design
• Simple Random Assignment
Just what it sounds like: randomly assign each subject to one of
the groups (a coin flip is sufficient for two groups).
• Matched Pairs Assignment
First match subjects into pairs based on some
characteristic related to the IV. Then randomly assign the
members of each pair to one of the two experimental
groups.
Repeated Measures Design
• The same individuals participate in both
experimental conditions, after which the
dependent variable is measured.
– Advantages: Fewer individuals are required.
– If training (or acclimatizing) is required, this design
can save valuable resources.
– As the individuals in the experimental conditions
are identical, more confounds are controlled.
Problems with Repeated
Measures Design
• Order effects
The order in which the measures are administered affects
the DV.
– Practice effect
– Fatigue effect
– Contrast effect
Problems for Many Sciences.
• How do we observe / experiment on the
internal workings of something (I.e.
cognition)?
Sternberg’s Experiment
Sternberg’s Results
Response Time = 398+38(S)
Gravitational Force =
(A constant called G) x (mass of first object) x
(mass of second object)
(the square of the distance between them)
Mechanism
Mechanism
Mechanism
Mechanism
Mechanism
Models & Mechanisms:
• Mechanism: entities and activities
organized to produce a phenomenon
(teleological?)
• Entities and activities organized in such
a way as to realize a functional role.
‘Model’?
• A Model is a description of some
phenomena / on
A model is verdical insofar as corresponds to
the actual phenomena it seeks to model.
A model, just like a ‘law’ or a ‘theory’ explains
phenomena / on and can be used to make
predictions about novel / unobserved aspects
of the phenomena it seeks to model.
Therefore, it is plays the same roll as ‘law’ or
‘theory’ in the H-D method or D-N model of
explanation.
Models
Categorization of different Models
/ Systems:
Scientific Reasoning
Conclusion
If I’m right that the main structure of explanation
in scientific inquiry is the investigation of
underlying mechanisms, then…
1. Correlational / observational studies are
primarily used for establishing the parameters of
the mechanism’s behavior.
2. Modeling is a fundamental, essential part of
scientific activity.
3. Models serve the same roll in scientific inquiry
as Popper’s ‘laws’ – they entail falsifiable
predictions.
4. The line between science & pseudoscience is
more clear:
Psychology v. Astrology
Phenomenon explained /
predicted: human behavior
and personality.
Mechanism: beliefs and
desires interact to
determine human
behavior, which beliefs
and desires get
precedence in any one
choice is influenced by the
hodge-podge of previous
experiences and genetic
dispositions we call
‘personality’.
Phenomenon
explained / predicted:
human behavior and
personality.
Mechanism: the forces
of the planets at time
of birth.
Biology v. Creation Science
Phenomenon to be
explained: Variation of
species over time and
space.
Mechanism: Natural
Selection (random
mutations are replicated if
they help the creature
reproduce by (a)
increasing survival in the
environment (b) changing
the number of offspring
the creature has or (c)
increasing the chances
that that will creature
Phenomenon to be
explained: Variation of
species over time and
space.
Mechanism: ?
Evaluating Competing
Mechanism:
Evaluating Competing
Mechanisms
Ptolemaic Astronomy
Copernican Astronomy
Phenomenon:
Phenomenon:
Parameters:
Fit the location of the planets & stars in
the sky
(They’re equal on this one)
“Other” Values:
The Copernican system is far simpler and
more elegant.
Venus
Venus
Galileo deduced that:
If the Ptolemaic system is correct, then
Venus should not show phases. And
If the Copernican system is correct, the
Venus should show phases.
Venus shows phases.
Therefore, the Ptolemaic system is not
correct.
Scientific Revolutions
The Ptolemaic system dominated western
and eastern science from 388BC until
the 16th century. So why the change?
Chemistry: Lavoisier
Biology: Darwin
Physics: Newton -> Einstein -> Quantum
(Bohr / Heisenberg) -> String Theory
REVOLUTION!
‘Real’ Revolutions as
metaphor.
• Scientific Revolutions are those ‘noncumulative developmental episodes in
which an older paradigm is replaced in
whole or in part by an incompatible one’
Thomas Kuhn The Structure of
Scientific Revolutions
Analogical points:
1. Revolutions are inaugurated by a
‘growing sense, often restricted to a
segment of the political community,
that existing institutions have ceased
to adequately meet the problems
posed by an environment that they
have in part created’
2. Revolutions often seem revolutionary
only to those whose paradigms are
affected to them.
3. Success of a revolution necessitates, in
part, the ‘relinquishment of one set of
institutions in favor of another, an in the
interim, society is not governed by
institutions at all.’
Conclusion:
• Well, that’s the point:
– During revolutions, society is divided into
competing camps or parties – one seeking
to defend the old, others seeking to replace
it with new.
– (There may be competing new camps as
well)
– Once that kind of polarization occurs,
political recourse fails.
• The parties are fighting over the legitimacy of
institutions by which political decisions can be
made – for that very reason, there is no
political mechanism for adjudicating between
the parties.
• So, the parties must ‘take to the streets’ –
appeal to something other than political will
(such as God, history, etc) or resort to force.
• The success of the winner is
determined not by political institutions,
but by extrapolitical institutions – by the
very fact that they replace those
institutions by which they legitimize
themselves.
Therefore, by analogy…
• Scientific revolutions gain legitimacy not
by factors internal to science, but by
extra-scientific methods, such as social
factors. And this is precisely because
the issue at stake is the legitimacy of
factors internal to science.
Modeling
Formulae
relating
observables
‘Mathematical
Models’ in Psych
V = d/t
Investigation of
underlying
structure
Discovered Models
‘Experimental
Systems’
Invented Models
Mathematical
Symbolic
Neural Network
F=ma
1st use: positing unobservables
Performed by Jameson and Hurvich in
1957. A test light is shown to a subject.
If the light appears greenish, a redappearing light is added until the test light
no longer appears at all greenish.
Jameson and Hurvich Results
Cone Sensitivity Curves
Mathematical Transformation of
Cone Sensitivity Functions
• We decorrelate the responses of the L, M and S
cones by weighting each signal with a constant,
and combining those results:
C1(l) = 1.0L(l) + 0.0M(l) + 0.0S(l)
C2(l) = -0.59L(l) + 0.80M(l) + -0.12S(l)
C3(l) = -0.34L(l) + -0.11M(l) + 0.93S(l)
Opponent Processing Model
Modeling
Formulae
relating
observables
‘Mathematical
Models’ in Psych
V = d/t
Investigation of
underlying
structure
Discovered Models
‘Experimental
Systems’
Invented Models
Mathematical
Symbolic
Neural Network
F=ma
2nd use: relating observables
• The most simple use of a mathematical
model is to fit a mathematical function to
some data collected in an experiment. That
function can then be used to make
predictions about novel or unobserved
behavior.
• Sternberg’s Memory Scanning Model
– Response Time = 398 + 38(Memory Set Size)
• De Castro and Brewer
– Intake of food = s(Number of People Present)0.22
Sternberg’s Results
Response Time = 398+38(S)
Gravitational Force =
(A constant called G) x (mass of first object) x
(mass of second object)
(the square of the distance between them)
The importance of Mathematical
Models:
Quick: what is the most
famous mathematical model
in the US right now?
The BCS Formula
• ‘Fit’?
• Data: team record, opponent’s record
(‘strength of schedule’), poll rankings
over the season, team losses & ‘quality
wins’.
Example: Oklahoma 2000?
• AP & Coaches poll end of season rank = 1.
• Average rank over the course of the season=
1.86.
• Average of AP & Coaches poll + average over
season = 2.86.
• (Thanks to Richard Billingsley at ESPN for
the explanation).
Strength of schedule
• Add the opponent’s records together =
73 Wins, 62 losses.
• Drop wins against teams that were not
1-A, and you have 70W.
• Drop losses from opponent’s schedule
that were against OK, and you get 50
losses.
• Total: 70 Wins, 50 losses.
Opponent’s winning %.
• The winning percentage is 70/120 = 58.3% or
0.583.
• 0.583 * 2/3 = 0.3889
• Do the same ‘opponent’ calculation for each
of the opponent’s opponents and weight it by
1/3 = 0.1749
• Add these 2 together and you get 0.5638
Now…
• Rank all the teams according to this ‘strength
of schedule’. OK is 11th
• Finally, take that rank / 25 = 0.44.
• Add ‘Team losses’ (0 for OK) and ‘Quality
wins’ (0 for OK).
• Add that to ‘Poll average’ and you get 3.30.
‘Mathematical’?
– Obvious: algebra / calculus
– Recursive functions
– Game Theory
• Other kinds of models
– Physical (geology)
– Virtual
• Neural Network
• Symbolic
– Animal
• In Vitro
• In vivo