Naturalized Approaches to Theory
Ladenness: Evidence from
Cognitive Psychology, History,
and the Ecological Validity
Argument
William F. Brewer
University of Illinois at Urbana-Champaign
Talk Presented at the Conference on the Theory-Ladenness
of Experience, Heinrich-Heine University, Düsseldorf,
Germany, March 10-11, 2011
Overview of Talk
I. The Problem of Theory Ladenness
II. Ecological Validity Argument
A. A Conceptual argument
III. Naturalized Phil of Sci
A. Framework from Cog Psy
B. Data from Cog Psy
C. Evidence from History of Sci
D. Conclusion
The Problem--Theory Ladenness
A. Norwood Hanson
1. Tycho Brahe/Kepler thought experiment
2. “seeing is a ‘theory-laden’ undertaking”
(1958, p. 19)
3. “we usually see through spectacles made of
our past experience [and] our knowledge”
(1969, p. 149).
B. Thomas Kuhn
1. "Lavoisier...saw oxygen where Priestly had seen
dephlogisticated air and where others had seen
nothing at all” (Kuhn, 1962, p. 117).
2. "After a revolution scientists are responding to a
different world" (Kuhn, 1962, p. 110).
C. Very strong psychological claims
1. clear case in Phil of Sci of need for a naturalized
approach
D Led some folk to relativism
1. Thus the epistemological message of [this] work could
be said to be..relativistic...It is relativistic because it
suggests that belief systems cannot be objectively ranked
in terms of their proximity to reality or their rationality."
(Barnes, 1974, pp. 154)
E. Fodor counterattacks with modularity (1983, 1984)
1. early perceptual processing occurs in modules that are
not cognitively penetrable by the higher order cognition
required by the theory laden claims
2. ‘objectivity’ of science saved!
F. Supported by:
1. Raftopoulos (2001a, 2001b)
2. Gilman (1990, 1992)
The Ecological Validity Argument
A. Neisser (1978) “Memory: What are the important
questions?”
1. many laboratory memory tasks not relevant to
phenomena of memory in real world
B. Brewer (2001) Ecological Validity in Psy Research
1. there needs to be a clear link between the laboratory
tasks studied by the scientist and the phenomena in the
world the scientist is trying to understand
2. example: nonsense syllable learning literature in
experimental psychology did not seem to be relevant to
understanding memory for syntax, word meaning,
schemata, etc.
C. Ecological Validity as Applied to Theory Ladenness
1. the naturalistic data of interest to us are data relevant to
the tasks the scientist carries out
2. we are interested in knowing if theory ladenness occurs
for the the scientist’s perceptions of the objects the
scientist is studying
3. we are interested in knowing if theory ladenness occurs
for the scientist’s attention to the objects the scientist is
studying
D. The Data Discussed by the Modularity Mafia do Not Have
the Required Ecological Validity
1. Fodor (1983, 1984) & Raftopoulos (2001a, 2001b) give
us interesting data from studies of the very earliest stages
of visual and neural processing
2. while the issues of the timing of the processing path for
vision are of interest to vision scientists they are not the
data we need in the Phil of Sci to address the theory
ladenness issue
3. for the problems of interest to us we simply do not
need to know if there is an early stage in vision (e.g.,
Marr’s 2 1/2 D) that is not cognitively penetrable.
4. for the issues of the Phil of Sci we need to know the
outcome of the processing
5. to resolve the issues related to theory ladenness we
need to know what the scientist perceived and what the
scientist attended to in carrying out the process of science
E. Apply the Ecological Validity Criterion to the Lab Data from
Cog Psy in this Talk
1. I also have to play by these tough rules
2. You should compare the laboratory data I review to the
episodes from the history of science and decide for
yourself if the laboratory findings seem directed at the
issues of interest (i.e. are ecologically valid)
F. So What do the Relevant Naturalized Data Tell Us?
1. I claim the ecological argument undercuts the data that
the modularity approach has attempted to use to eliminate
the epistemological problems raised by theory ladenness
2. The core of this talk is a review of the appropriate
evidence from current Cog Psy on the issue of theory
ladenness.
A Framework for Understanding
Theory Ladenness
A. Overemphasis of the Role of Visual Perception
1. There is a long history in Western philosophy of
focusing on visual perception in epistemology
2. British Empiricists
a. Hume, Berkeley, Lock
3. Logical Positivists
4. Kuhn & Hanson postPositivists
5. Fodor’s modularity approach
B. Problems with this Overemphasis
1. “It is data rather than perceptual beliefs that play a
central evidential role in science and data are typically not
descriptions of perceptual appearances or reports of
perceptual belief at all" (Bogen & Woodward, 1992 p,
599).
2. in modern science with computer-based data
acquisition and analysis the actual data used is rarely the
direct perceptual observations of the individual scientist
C. Brewer & Lambert 2001
1. In Psychology of Science we want to study the scientific
process from the initial designing of experiments to the
final writing of a journal article
2. this approach reduces the role of visual perception but
expands the potential role of top down influences to cover
a much wider range
3. potentially relevant mental processes:
a. perception
b. attention
c. thinking
d. experimenting
e. memory
f. communication of findings
D. Term “Theory will be Used Broadly
1. in this talk will ignore the literature on different forms of
mental representation (images, concepts, schemas,
mental models, etc)
2. in this talk theory = any form of top-down knowledge
E. Laboratory Data Derived from Nonscientist Participants
1. in an ideal world the data from Cog Psy would be based
on studies of actual scientists
2. in practice almost all of the data will be from children
and undergraduate participants (actual data from
scientists will be discussed along with the historical cases)
3. while there are many differences between scientists
and ordinary folk we hope that the underlying mental
processes are the same
F. Top Down--Bottom Up Synthesis
1. modern framework (e.g., Lindsay & Norman, 1977)
a. perception is conceived as the product of both
bottom-up and top-down factors
b. bottom-up: physical information coming into the eye
or other sensory organ
c. top-down: beliefs or theories about what the
individual is perceiving
2. connectionist approaches (e.g., McClelland &
Rumelhart, 1981)
a. give a natural account of the interaction of the two
forms of information
3. neuroscience
a. recent work suggests there are actually more topdown descending neural pathways than ascending paths
in the visual system
4. epistemological consequences
a. allows evidence for top-down theory-driven effects
b. bottom-up information from the world
can help avoid the slippery slope to relativism
5. Fodor’s modularity view
a. rejects the top/down-bottom-up view which is the
modal view in current cognitive psychology
b. given Fodor’s visibility in philosophy I think it might
be easy for philosophers not to realize that his view is not
the standard view
G. Strongest Top-down Effects Occur with Degraded Input
1. Brewer & Lambert, (1993, 2001); Brewer & Loschky,
(2005) review of the Cog Psy literature suggests that the
amount of information that comes from top-down
processes various across perceptual tasks
a. hallucinations & dreams = ~100% top-down
a. vivid, unexpected object being attended to would
involve much bottom up information
2. the examples where theory has a very strong effect on
perception tend to be cases of degraded stimuli, stimuli
being presented for a very short time, or ambiguous
stimuli.
H. Facilitatory & Inhibitory Role of Theory on Understanding
the World (the Two Edged Sword)
1. Brewer & Lambert, (1993, 2001) focused too strongly
on the negative role of top-down processes
2. we wanted to show the effects of the top-down
processes and so emphasized laboratory studies and
historical cases that typically showed a negative impact on
the performance of the participant in the lab or the
scientist in the historical episode
3. in laboratory studies we typically know what the stimuli
was so we can easily determine if the impact of theory
was facilitatory or inhibitory
4. the data show that having a correct theory about the
stimulus can reduce perceptual thresholds, direct
attention, etc.
5. (old fashioned view about to come!) I want to argue in
this talk that the historical episodes from the history of
science can be treated the same way. In science we often
know which view of the world was correct (no scare
quotes!). So practicing Whig History in the history of
science allows us to better understand the psychological
processes that were occurring at the time (of course, the
scientist working in real time never knows if the theory
they are working with maps the world successfully)
6. so I claim that the scientist who is lucky enough to have
a theory that reflects the nature of the world (most of us
working scientists are un-reconstructed realists) will show
the positive benefits of improved perception of the objects
in the scientific domain, will be able to direct attention to
the appropriate objects, will better be able to interpret
information, etc.
7. and for the scientist who is unlucky enough to hold an
incorrect view of the world there will be inhibition in all the
same psychological processes in this scientist’s attempt to
understand the world
8. this revived Whig History allows me to understand the
psychological processes the were at work in many
historical episodes
9. the Neptune/Vulcan example-(a) when Galle received the theoretical information
from Le Verrier about the likely location of Neptune it
directed his attention to the correct region of the sky and
allowed him to discover Neptune--a clear case of
facilitation
(b) however, when similar theoretical calculation were
done for Mercury they led to many false sightings of the
hypothetical planet Vulcan (a planet we now know does
not exist)--a clear case of inhibition
10. so this (almost counterbalanced design!) supports my
argument that when a scientist has a theory that correctly
maps the world the mental processes will facilitate the
ability of the scientist to understand the world, while the
reverse in true for scientists with a theory that does not
map the world
I. Strategy in the Next Part of the Talk
1. take a naturalized approach to the philosophical
discussions of the theory-ladenness of perception
2. study perception & the other processes
3. for each mental process--look for evidence from:
a. cognitive psychology
b. history of science
Evidence on the Issue of Theory Ladennes
from Cog Psy and History of Sci
I. Perception--Psychological Evidence
A. Overview
--> 1. perception
2. attention
3. thinking
4. experimenting
5. memory
6. communication
A. What do you see here?
B. What do you see here?
C. What do you see here?
D. What do you see here?
E. What do you see here?
F. What do you see here?
G. Ambiguous Figures--Perceptual Info (e.g., Leeper, 1935)
1. stimuli & procedure
a. unambiguous version of an ambiguous figure
b. note that this ambiguous figure gives rise to two
different perceptual experiences
c. later show ambiguous version
d. ask what the participants see
2. results
a. if 1st see unambiguous old woman then almost all
participants see the ambiguous figure also as old woman
b. if 1st see unambiguous young woman then almost
all participants see the ambiguous figure as young woman
3. analogue in science:
a. scientist looking at an idealized textbook drawing &
then showing facilitation when looking a real slides
4. Bill Bechtel anecdote--anti interpretation
H. Ambiguous Figures--Rat/Man (Bugelski & Alampay, 1961)
I. Ambiguous Figures--Perc Class Info (e.g., Bugelski &
Alampay, 1961)
1. stimuli & procedure
a. show participants set of animal pictures (no rat)
b. later show ambiguous rat/man picture
c. ask what the participants see
2. results
a. up to 80% of participants saw rat if had previously
seen the set of animal pictures (
J. Ambiguous Figures--Conceptual Info (e.g., Liu, 1976)
1. stimuli & procedure
a. participants hear passage about rats
b. later show ambiguous rat/man picture
c. ask what the participants see
2. results
a. doubled the rate of seeing the ambiguous figure as
as rat
3. conclusion
a. seems like a clear example of high-level conceptual
theory information influencing the perceptual experience
K. Vague Stimuli--Reynolds, 1985
1. stimuli: fragmented pictures
a. initially seem to be random shapes (bottom-up info)
then shift to being perceived as a meaningful obj
2. design: 3 conditions
a. no info [9%]
b. group told figs were meaningful picts [55%]
c. group given more specific top down conceptual info
(e.g. “animal”) [74%]
3. these stimuli give a Gestalt like experience (the random
fragments are suddenly perceived as a meaningful object)
so likely to be a true perceptual effect not a conceptual
inference
L. Reynolds 1985
M. Vague Scientific Stimuli (Chinn & Malhotra, 2002)
1. it might be possible to use the ecological validity
argument against the previous laboratory experiments
2. is there any theory laden evidence for an experimental
situation even more like those in science?
3. children watching an actual scientific experiment
4. procedure
a. experimenters show 4th grade children Galileo’s
experiment of dropping heavy & light objects
b. children must report if they saw:
(1) heavy & light rocks hit at same time
(2) heavy hit first or light hit first
c. this a very difficult perceptual judgment
5. children have different naive theories:
a. some believe heavy objects fall faster
b. some believe both fall at same speed
4. results
a. theory hit same 72% report both hit at same time
b. heavy faster 25% report both hit at same time
5. this experiment does not provide ironclad evidence that
the children’s theories influenced their perceptions, but
they certainly show that the children’s theories influenced
their observations and that is the crucial issues (for the
ecological argument)
N. Conclusions: Perception--Psychological Evidence
1. the experiments appear to give strong support to the
top-down bottom-up synthesis.
2. note--in all the strong cases of theory ladenness the
stimuli are ambiguous, degraded, or required a difficult
perceptual judgment (so top-down factors can override the
bottom up factors)
3. but strong bottom-up information will override top-down
beliefs
a. show Galileo a nice Voyager color photograph of
Saturn and he would clearly see the rings
4. so no need to slide down the slippery slope to relativism
5. now shift to evidence from the history of science to see
if it is consistent with the laboratory studies from
psychology
6. the data from history cannot be, by its nature, as clear
as the experiments where we can control and manipulate
variables, but it also provides convincing evidence for the
interplay of top-down and bottom-up processes in the
perception of actual episodes in the history of science
II. Perception--History of Science
A. Overview
--> 1. perception
2. attention
3. thinking
4. experimenting
5. memory
6. communication
A. Rings of Saturn
1. astronomers were expecting to see moons circling
Saturn since that is what they had seen with Jupiter
2. for 40 yrs they drew Saturn with
a. moons or
b. handles coming out of the poles
3. see drawings
4. Drawings of Saturn (Van Helden, 1974)
B. Astronomer’s Psychological Conclusions
1. several astronomers have reviewed this and other
examples of top-down factors in observational astronomy
and have come to conclusions very similar to mine:
2. Sheehan (1988) stated “Once a definite expectation is
established, it is inevitable that one will see something of
what one expects; this reinforces and refines one’s
expectations in a continuing process until finally one is
seeing an exact and detailed--but ultimately fictitious-picture” (p. 85).
3. Sheehan also noted the role of theory in facilitating
astronomical observation. He stated, “It is often
wondered, after an object is once discovered, that it was
not found long before. Faint objects are often reported as
detectable in telescopes two or even three times smaller
than those used for their discovery. Following the
discovery of Saturn’s inner ring (the crépe ring)...for
instance, the new ring was seen by a number of observers
with small instruments” (p. 63).
4. Hetherington (1983) drew similar conclusions from the
history of astronomy and concluded that having a theory is
a two-edged sword. He pointed out that “scientists are
guided by theory in selecting those observations that may
be important” (p. 729) and that “the warping of judgments
by knowledge, the influence on observational reports of
preconceived opinion, is inevitable” (p. 729).
C. N-rays
1. after discovery of X-rays physicists went looking for
other new forms of radiation
2. 1903 famous French physicist, Blondlot, announced
discovery of N-rays
3. N-rays (see slide)
a. produced by Nernst lamp
b. blocked by wet cardboard
c. refracted by aluminum prism
d. detection--a hard perceptual task
(1) increase in spark gap or
(2) by faint phosphorescent glow of paint in dark
4. 300 papers on properties of n-rays were published by
100 different scientists [ouch!] [one said discovered first!]
5. N-ray Apparatus
6. Wood, sceptical American physicist, went to Blondlot’s
lab--in dark removed aluminum prism--experimenters
could still see the n-rays--so by manipulating the physical
factors while leaving the psychological ones constant he
showed the effect was essentially all due to top-down
theory
7. Conclusion
(a) scientists with strong theoretical beliefs who have
to make very difficult perceptual judgments show strong
theory-laden perception
D. Conclusions: Perception--Historical Evidence
1. the historical record also appear to give strong support
to the top-down bottom-up synthesis.
2. when a scientist’s theories map the world the
psychological processes facilitate understanding the
world, but when the scientist’s theory does not map the
world the psychological processes inhibit the scientist’s
ability to understand the world
3. there are clear examples of theory ladenness, but as
with the evidence from cognitive psychology experiments
the top-down effects almost always occur when the stimuli
are ambiguous, degraded, or required a difficult
perceptual judgment
4. but strong bottom-up information will override the topdown beliefs:
a. Tycho Brahe seeing a new star (supernova)
b. if the data in Blondlot’s experiment had been
observations of a 10 on a meter with a 1 to 10 scale it is
likely that the N-ray affair would not have occurred)
5. so, once again, we are not driven to relativism (but
instead to the need for controls against the possibility of
top-down theory-laden processes influencing
experiments)
III. Attention--Psychological Evidence
A. Overview
1. perception
--> 2. attention
3. thinking
4. experimenting
5. memory
6. communication
A. Laboratory Inattentional Blindness (Mack & Rock, 1998)
1. stimuli--cross presented very rapidly (200ms) faster
than a shift in eye fixation possible
2. participants asked which arm of cross was longer
3. on a critical trial a totally unexpected objects (e.g., a
small black square) was presented within one of the
quadrants of the cross
4. depending on details of the experimental procedures
from 25% to 80% report not having seen the unexpected
object
5. control procedure of not asking for cross judgments and
asking participants to report anything they saw showed
essentially 100% reports of the unexpected object
B. Top-down --> Attention (Shinoda, Hayhoe & Shrivastava,
2001)
1. procedure
a. participants asked to carry out virtual driving task
while obeying the usual traffic laws
2. stimuli
a. stop signs in usual locations at intersections
b. stop signs in middle of a block
3. results
a. usual locations ~100% responded to
b. unexpected locations less than 33% responded to
4. conclusions
a. top -down beliefs about typical stop sign location
was directing attention so signs with identical bottom-up
sensory information in atypical locations were less likely to
be attended to and responded to
C. Gorilla Study (Simons & Chabris, 1999)
1. goal--can one get inattentional blindness with very vivid
stimul
2. stimuli--two teams of payers passing basketball back
and forth.
3. top-down attention--participans attention participants
asked to mentally count the number of ball passes made
by one team
4. Gorilla Walks Across Scene (Simons & Chabris, 1999)
5. results
a. only about 50% of the participants report seeing the
gorilla
6. conclusions
a. can get inattentional blindness for very vivid stimuli
b. theory-directed focusing of attention facilitates
theory-directed observation at the expense of non-theory
dirrected observations
IV. Attention--Historical Evidence
A. Overview
1. perception
--> 2. attention
3. thinking
4. experimenting
5. memory
6. communication
A. “Pre-discovery” Observations
1. after some object is “officially” discovered astronomers
look back at earlier data from locations that they now
know the object should have been visible (to extend the
database to calculate better orbits)
2. Forbes (1982) found 22 pre-discovery observations of
Uranus.
3. so with no theory-based attention the earlier
observations were not attended to and reported, but with
theory-based attention Forbes found the 22 pre-discovery
observations of Uranus
B. Brahe’s Bottom-Up Discovery of the 1572 Supernova
1. In addition to the evidence for top-down processes
operating in attention, the history of science also provides
good examples where salient bottom-up information
attracted attention in spite of strong top-down beliefs.
2. Tycho Brahe’s account of his discovery of the
Supernova of 1572 makes the point well: “I was
contemplating the stars in a clear sky, I noticed that a new
and unusual star, surpassing the other stars in brilliancy,
was shining almost directly above my head; and since I
had, almost from boyhood, known all the stars of the
heavens perfectly... it was quite evident to me that there
had never been any star in that place in the sky, even the
smallest, to say nothing of a star so conspicuously bright
as this. I was so astonished at this sight that I was not
ashamed to doubt the trustworthiness of my own eyes”
(Brahe, 1929).
C. James Christy’s Discovery of Charon
1. Christy was refining the orbit of Pluto by measuring it’s
location against background stars
2. every few months he was sent a new photo of Pluto
and he carefully measured its center against the
background stars [see picture]
3. Discovery Picture of Pluto’s Moon, Charon
4. one day in 1978 he noticed a fuzzy bump on the picture
he was measuring--he took out the last one he measured
and saw a fuzzy bump in a different location--he the then
examined all of the old pictures of Pluto in the
Observatory and found the fuzzy bump on most
5. he calculated the changes in location of fuzz as if Pluto
was rotating & found a precise orbital rotation period
D. Conclusions: Attention
1. both the laboratory studies from psychology and the
historical accounts show that top-down processes can
direct attention
2. but strong unexpected bottom-up information can
attract attention
a. Brahe’s discovery of the 1572 supernova
b. Rontgen’s observation of a glow in a darkened room
3. so once again we have both top-down & bottom-up
factors at work
4. no need for relativism
V. Thinking--Psychological Evidence
A. Overview
1. perception
2. attention
--> 3. thinking
4. experimenting
5. memory
6. communication
A. Theory Laden Interpretation of Text
(Bransford &Johnson, 1972)
1. stimuli--obtuse passage [see next slide]
2. Bransford & Johnson, 1972 Obtuse Passage
The procedure is actually quite simple. First you arrange
things into different groups depending on their makeup.
Of course, one pile may be sufficient depending on how
much there is to do. If you have to go somewhere else
due to lack of facilities that is the next step, otherwise you
are pretty well set. It is important not to overdo any
particular endeavor. That is, it is better to do too few
things at once than too many. In the short run this may
not seem important, but complications from doing too
many can easily arise. A mistake can be expensive as
well. The manipulation of the appropriate mechanisms
should be self-explanatory, and we need not dwell on it
here. At first the whole procedure will seem complicated.
Soon, however, it will become just another facet of life. It
is difficult to foresee any end to the necessity for this task
in the immediate future, but then one never can tell.
3. procedure
a. participants hear passage
b. three conditions: (washing clothes)
(1) no topic
(2) topic after hearing passage
(3) topic before hearing passage
4. results (recall--number of ideas right)
a. no topic
2.8
b. topic after 2.7
c. topic before 5.8
5. schema/theory allows interpretation/comprehension
which then facilitates memory
B. Top-down Interpretation of Scientific Data (nonscientists)
(Brewer & Chinn, 1994)
1. participants:
a. undergrads
2. design
a. one group reads passages supporting Theory A
(dinosaurs were warm blooded)
b. second group reads passages supporting Theory B
(dinosaurs were cold blooded)
c. so now have groups that believe different theories
about the dinosaurs
`
3. each group asked to evaluate data that was:
a. consistent with the theory they read and inconsistent
with with other theory or
b. inconsistent with the theory they read and consistent
with the other theory
e.g., (consistent with cold) cold blooded animals have
small pineal systems & impressions in dinosaur skulls
show small pineal systems
4. results (1 to 10 scale on belief in quality of the data)
a. theory consistent data 8.0
b. theory inconsistent data 6.0
5. conclusions
a. the exact same data is interpreted/evaluated
differently (in a theory consistent fashion)
VI. Thinking--Historical Evidence
A. Overview
1. perception
2. attention
--> 3. thinking
4. experimenting
5. memory
6. communication
A. Many Classic Paradigm Shifts in the History of Science
Show Theory-Based Reinterpretation
1. flat earth vs. round earth
2. geocentric vs. heliocentric views
(Ptolemy vs. Copernicus)
3. phlogiston vs. oxygen in chemistry
4. particle vs. wave theories of light
5. Newton vs. Einstein in physics
6. fixed continents vs. continental drift
7. intelligent design vs. natural selection
B. Example: Data on Shape of Africa & South America
C. Public Models (Interpretations) of the Iguanodon
1. paleontologists are sometimes in the embarrassing
situation of having their theory-based interpretations on
public display
2. the public models of the Iguanodon are wonderful
examples
3. at first there are usually few bones so interpretation
plays a larger role--then as more bones become available
they provide bottom up evidence that constrains the
interpretation
4. the first Iguanodon drawing appears to based on the
characteristics of living Iguanas
5. Public Models (Interpretations) of the Iguanodon
6. the next model (made for the Crystal Place exhibition in
1851) appears to have been based on the characteristics
of a Rhinoceros
7. the next models (Royal Museum of Natural History in
Brussels) appears to have been influenced by Kangaroos
8. the current models are based on characteristics of
raptors--fast moving predators with a rigid horizontal tail
9. the modern model builders note that in order to allow
the Kangaroo like model to sit the model builders had to
disarticulate the vertebrae (thus a beautiful example of
top- down beliefs overriding bottom-up bone information)
10. conclusion
a. theory --> interpretation --> construction of models
in paleontology
D. Thinking Conclusions
1. both the laboratory studies from psychology and the
historical accounts show that top-down processes have a
powerful effect on scientific thinking
2. interpretation (not perception) probably provides the
strongest evidence for those who want to argue for
relativism since the exact same data can be given
radically different interpretations
3. but we also have the impact of bottom-up information
4. so once again we have both top-down & bottom-up
factors at work
5. no need for relativism
VII. Experimenting--Psychological Evidence
A. Overview
1. perception
2. attention
3. thinking
--> 4. experimenting
5. memory
6. communication
A. Theory Driven Experimentation--Psychol Experiments
(Klahr & Dunbar, 1988)
1. ask participants to learn how to program a
programmable toy vehicle (Big Trak)
2. the experimenters observe the participants experiments
(they punch program keys & see what the truck does)
3. results
a. some participants just push key & see what
happens (inductive experimenters?)
b. some participants developed theories and then tried
them out (theory testing experimenters)
c. theory > induction in this particular task
4. conclusions
a. many participants are carrying out theory-directed
data gathering
VIII. Experimenting--Historical Evidence
A. Overview
1. perception
2. attention
3. thinking
--> 4. experimenting
5. memory
6. communication
A. Physical Constants
1. it turns out those precise looking numbers you see in
physics books are actually arrived at by a committee!
2. every few years the committee meets and reviews the
data from the current experiments and establishes the
“official” value for a given constant
3. the committee publishes the changes in the official
numbers for each constant over time
4. the data show that often the value for the constant
remains within the error bar of the earlier value from one
year to the next, but sometimes the values continue to
creep higher or lower until they are quite different [see left
graph in the slide]
5. other times there will be a new, very different value
many standard deviations from the first value and then the
new values will track this new value [see right figure]
6. the physicists call this “intellectual phase locking”
7. These data reporting the measurements of physical
constants over time capture the data from hundreds of
experiments, so they establish the generality of intellectual
phase locking; however they do not give an account of
how the effect occurs.
8. Luckily a number of physicists have provided accounts
of how they think the scientists’ beliefs in the data from the
earlier experiments influence the way the later
experiments were carried out.
9. Birge (1957) wrote that E. O. Lawrence stated that “In
any highly precise experimental arrangement there are
initially many instrumental difficulties that lead to
numerical results far from the accepted value of the
quantity being measured. It is, in fact, just such wide
divergences that are the best indication of instrumental
errors of one kind or another. Accordingly, the investigator
searches for the source or sources of such errors, and
continues to search until he gets a result close to the
accepted value. Then he stops” (p. 51).
10 Allan Franklin quoted an ironic comment made within
the physics community about a scientist “who fights the
systematics until he or she gets the ‘right’ answer (read
‘agrees with previous experiment’) and then publishes”
(1984, p. 791).
11. Richard Feynman, in his unique style, described the
intellectual phase lock effect as revealed in measurement
of the charge of the electron. “Millikan measured the
charge on an electron by an experiment with falling oil
drops, and got an answer which we now know not to be
quite right. It’s a little bit off, because he had the incorrect
value for the viscosity of air. It’s interesting to look at the
history of measurements of the charge of the electron,
after Millikan. If you plot them as a function of time, you
find that one is a little bit bigger than Millikan’s, and the
next one’s a little bit bigger than that, and the next one’s a
little bit bigger than that, until finally they settle down to a
number which is higher. Why didn’t they discover that the
new number was higher right away? It’s a thing that
scientists are ashamed of--this history--because it’s
apparent that people did things like this: When they got a
number that was too high above Millikan’s, they thought
something must be wrong--they would look for and find a
reason why something might be wrong. When they got a
number closer to Millikan’s value they didn’t look so hard.
And so they eliminated the numbers that were too far off,
and did other things like that� (1985, p. 342).
12. conclusion
a. powerful evidence for theory driven experimentation
IX. Memory--Psychological Evidence
A. Overview
1. perception
2. attention
3. thinking
4. experimenting
--> 5. memory
6. communication
A. Theory-Based Distortions in Memory
(Zangwill, 1937)
1. stimuli
a. 5 ink-blots (animal like) & 1 ambiguous
b. 5 ink-blots (mountain like) & same 1 ambiguous
2. procedure
a. see one ink-blot & asked to draw it from memory &
tell what it reminded them of
3. the repeated ink-blot was recalled (drawn) in animal-like
form when it had appeared with animal-like blots; while
the exact same blot was recalled in mountain-like form
when it had appeared with mountain-like blots.
4. conclusion
a. theory-based distortions in memory
5. Stimuli & Data from Zangwill, 1937
X. Memory--Historical Evidence
A. Overview
1. perception
2. attention
3. thinking
4. experimenting
--> 5. memory
6. communication
A. Scientists Distort the Sci Literature in Memory
(Vicente & Brewer, 1993)
1. Analysis of the scientific literature shows many
distortions
a. Whorf-Sapir Hyp--number of words for snow
b. Galileo dropped objects from the Tower of Pisa
c. deGroot’s study of expert chess players superior
memory for chess games
2. Vincente & Brewer, 1993 argue many of these may be
memory distortions
3. gave undergrads non-distorted information about
deGroot’s experiments and show they make memory
errors just like those found in the scientific literature
supporting the view that the many of the scientists’
distortions may have been theory-based memory errors
4. this suggests that when scientists are debating the
merits of some scientific theory they may each be relying
on a somewhat differently recalled set of evidence
(5. note that the practice of requiring the keeping of timely
lab notebooks was likely introduced to reduce the
possibility of memory errors)
B. Conclusions: Theory-Based Processes in Memory
1. both the laboratory studies from psychology and the
historical accounts suggest that there may be theorydirected memory distortions of the scientific literature by
scientists
XI. Communication--Psychological Evidence
A. Overview
1. perception
2. attention
3. thinking
4. experimenting
5. memory
--> 6. communication
A. Summarization of Text
(van Dijk & Kintsch, 1978)
1. participants delete nonthematically related information
2. so theory-laden communication of information
XII. Communication--Historical Evidence
A. Overview
1. perception
2. attention
3. thinking
4. experimenting
5. memory
--> 6. communication
A. Biased Reviews of the Scientific Literature
1. Chalmers, Frank, and Reitman (1990)
2. theory-based evaluation of the literature of review
articles on radiotherapy after radical mastectomy
3. radiotherapists & surgeons: 72% classified as
“enthusiastic”
4. other medical specialists: only 15% “enthusiastic”
XIII. Multiple Sources of Theory-Ladenness
C. Darwin on Theory Ladenness (Autobiography, p. 70)
[mixture of perception, attention, & possibly interpretation]
1. [in 1831] “We spent many hours in Cwm Idwal,
examining all the rocks with extreme care, as Sedgwick
was anxious to find fossils in them;”
2. “But neither of us saw a trace of the wonderful glacial
phenomena all around us; we did not notice the plainly
scored rocks, the perched boulders, the lateral and
terminal moraines.”
3. “Yet these phenomena are so conspicuous [when
Darwin visited in 1842 after learning about Ice Ages] ...a
house burnt down by fire did not tell its story more plainly
than did this valley.”
XIV. Overall Conclusion
A. There is Psychological Data and Historical Data to
Support the View that Theory-Ladenness Occurs in:
1. perception
2. attention
3. thinking
4. experimenting
5. memory
6. communication