Kids, Cats and Concepts:
Toward a Grand Unified Theory of
Thinking
Geoff Norman
The Goal
 To link research in three domains:
Dual processing models of thinking
 Exemplar and prototype models of
categorization / concept formation
 Expertise and clinical reasoning
to a greater understanding of human
information processing
How I got there
 Distant
 Studies of clinical problem-solving
 Intermediate
 Role of experience in clinical reasoning
 Recent
 Diagnostic errors and “dual processing”
A Difficult Diagnostic Task
An Easy Diagnostic Task
 The rule is insufficient for the
classification task
 But we can do the task quickly,
accurately, and effortlessly
HOW?
The Role of Similarity
DUAL PROCESSING
Two basic strategies
 System 1
Based on holistic similarity to prior
examples
“Exemplar theory” (more later)
 System 2
Based on underlying conceptual
characteristics
 “Causal models”
System 2 thinking
Playing by the rules
Analytic View of Expertise
“The matters that set experts apart from
beginners are symbolic, inferential, and
rooted in experiential knowledge…Experts
build up a repertory of working rules of
thumb or “heuristics” that, combined with
book knowledge, make them expert
practitioners.”
E. Feigenbaum. The fifth generation: artificial intelligence and
Japan's computer challenge to the world. 1983
System 1 thinking
I’ve seen it before and here it
comes again
 Successful categorization
From 2-D abstract representation
without analysis of features
without language
 Successful generalization
To other 2 D abstraction in atypical
orientation
The Non- Analytic View
“We must be prepared to abandon the
traditional view that runs from Plato to
Piaget and Chomsky that a beginner starts
with specific cases and… abstracts and
interiorizes more and more sophisticated
rules.It might turn out that skill acquisition
moves in just the opposite direction; from
abstract rules to particular cases.”
H.L. Dreyfus, 2002
Outline
Dual processing
Concept formation and categorization
Expertise and Clinical Reasoning
Applications -- Implications
Levels of Processing
Perception
Transfer
Aging and reasoning
Intelligence
Three Literatures
Dual Processing Concept formation
(Thinking)
(categorization)
Stanovich,
Evans,
Kahnemann
Medin,Brooks
Clinical Reasoning
Norman,Schmidt
Dual Processing
System 1
Rapid, unconscious, based on concrete
similarity, “just” pattern recognition
System 2
Slow, logical, conceptual, energyintensive,
CHARACTERISTICS
System 1
System 2
 Unconscious
 Conscious
 Implicit
 Explicit
 Automatic
 Controlled
 Effortless
 Effortful
 Rapid
 Slow
 Holistic,
 Analytic
 Old (evolution)
 New (evolution)
 Contextualized
 Abstract
Neuroanatomy of System 1,2
System 1
“right inferior
prefrontal cortex”
Evans, 2008
“Involves
hippocampus”
Smith & DeCoster,
2000
System 2
“ventral medial
prefrontal cortex”
Neurophysiology of System 1,2
 Glucose dose (vs. Placebo)
 Shift of processing strategy toward
System 2 (more energy demand) with
glucose load
(Attraction effect - 17% vs. 47%)
(Masicampo & Baumeister, 2008)
Mental representations
 SYSTEM 1
Abstract concepts
Feature list, probability, causal mechanism,
process
 SYSTEM 2
???????
Categorization /
Concept Formation
Exemplar Theory Medin, Brooks
 Categories consist of a collection of prior instances
 identification of category membership based on availability of
similar instances
 Retrieval process is “non-analytic (unaware), hence can result
from objectively irrelevant features
 Retrieval process is not deliberate, not available to introspection
 (Like System 1)
Dual Processing in Medicine
From Process to Knowledge
(Analytical and Experiential)
The beginnings clinical reasoning as a process
“Hypothetico-deductive method”
(Elstein, Shulman, Sprafka, 1977)
Expert (and novice) clinicians
generate multiple diagnostic
hypotheses early in the encounter
then gather data to confirm
(usually) these hypotheses
Does hypothesis predict
accurate solution?
100
90
80
70
60
Correct
50
on chart
40
30
20
10
0
Present
Absent
Correct hypothesis?
Barrows, Neufeld, Norman, 1981
Where do hypotheses come from?
“Medical experts differed from novices in that
they generated better hypotheses……
and we don’t know why!”
A. Elstein
Dx Error Conference
May 31, 2008
Expert Physicians and
Dual Processing
To what extent does the:
formal knowledge of medical school
vs.
experiential knowledge of practice
contribute to expertise
Schmidt & Norman, 1991
Novice
Intermediate
Expert
Basic Science
Mechanisms
Clinical Rules
Examples
System
2
Basic Science
Mechanisms
System
1
Clinical Rules
Basic Science
Mechanisms
Who do you pick?
 Dr. JW completed the specialty exam
last year and stood 14th in the
country.
 Dr. WS completed the specialty exam 6
years ago. At the time, she was in the
top 1/3 of all candidates.
The Conundrum
Why do we prefer the candidate with
apparently less “competence” but much
more experience?
What did she get from 10 years of
experience?
10 years of experiences
(System 1 knowledge)
BUT
Every measure of formal (System 2)
knowledge decays right after
graduation
Day and Norcini, 1988
540
520
500
480
460
440
420
<20
21-24
25-29
Years since Graduation
30-34
35-39
Evidence of System 1
in Diagnostic Reasoning
Visual Diagnosis and Response Time
STUDY
 100 slides in 20 categories
 Students, clerks, residents,
GPs, Dermatologist
 Accuracy and Response Time
Accuracy by Educational Level
100
90
70
60
50
40
30
20
10
og
is
t
er
m
to
l
GP
Re
si
de
nt
Cl
er
k
0
St
ud
en
t
% correct
80
Response time by Educational Level
30
20
Correct
Incorrect
DK
15
10
5
og
is
t
er
m
to
l
GP
Re
si
de
nt
Cl
er
k
0
St
ud
en
t
Response Time
25
Evidence of Exemplars
Effect of Similarity
(Allen, Brooks, Norman, 1992)
 24 medical students, 6 conditions
Learn Rules
Practice rules
Train Set A
(6 x 4) x 5
Train Set B
(6 x 4) x 5
Test (9 / 30)
Accuracy by Bias Condition
90
80
70
60
50
40
Correct
30
Other
Incorrect
20
10
0
Bias Corr
Bias Incorr
 Is it just visual similarity?
 If it’s “non-analytic” does it apply to
objectively irrelevant features?
ECG Interpretation
Hatala et al, 1999
 Medical students/ Fam Med residents
 PRACTICE (4/4 + 7 filler)
 middle aged banker with chest pain
OR
 elderly woman with chest pain
 Anterior M I
 TEST ( 4 critical + 3 filler)
 Middle aged banker with chest pain
 Left Bundle Branch Block
RESULTS
Percent of Diagnoses by Condition
50
Percent mentioning
40
30
Bias
20
No bias
10
0
Correct
Prior
Diagnosis
CONCLUSIONS Medical Diagnosis and Dual Processing
 Experiential knowledge is a major contributor to
diagnostic expertise
 Categories and concepts are based on our specific
experience with the world
 These specific experiences are accessed and used
without awareness
When do experts use system 2?
Analytic reasoning and Diagnosis
 Invoked for confirmation in all Dx encounters
 Analytic knowledge of many forms:
 Illness scripts
 Symptom-disease probabilities
 Semantic axes
 Feature lists (e.g. DSM 4)
Where Do Clinicians Use Basic Science?
 Most use basic science rarely?
 Observational studies (Schmidt, Patel)
 Some use basic science some of the time
 Difficult problems in nephrology
 Some use physiology ALL the time
 Intensivists, anesthesiology
Most use it rarely
(Patel, Schmidt)
 Clinicians rarely use basic science
explanation in routine practice.
 While they may possess the knowledge,
it remains “encapsulated” until
mobilized for specific goals (to solve
specific problems) (Schmidt, HG)
Some Use it with Difficult
Cases
(Norman, Brooks, Trott, Smith)
 When experts are confronted with
difficult cases, do they revert to causal
reasoning?
Experimental Design
R1 --GP
n=4
R2 -- IM
n=4
Clinical Cases
k=8
Explain and Diagnose
Nephrol
n=4
Diagnostic Accuracy
1
0.8
0.6
0.4
0.2
0
R1-FM
R2-IM
Nephrol
Causal Explanations
2.5
2
1.5
1
0.5
0
R1-FM
R2-IM
Nephrol
No of Diagnoses /
Investigations
6
5
4
3
2
1
0
R1-FM
R2-IM
Nephrol
Conclusions Use of Basic Science
 In difficult diagnostic situations, clinicians
use causal physiological knowledge and
analytic reasoning
 Expertise associated with more coherent
explanations, better diagnosis
Dual Processing and Experience
 With increasing experience, do people
rely more or less on System 1 -- Nonanalytic reasoning?
Studies of Relative Experts
(Moruzi, Brooks, Norman, 2003)
 Dermatologists/ GPs / residents
 36 slides (typical / atypical)
Condition A
Verbal description of slide (verbal)
then photo (visual + verbal)
Condition B
 Photo only (visual)
Diagnostic Accuracy
90
80
70
60
50
Resident
40
30
20
10
0
Verbal
Verbal+Visual
Visual
Diagnostic Accuracy
90
80
70
60
50
40
30
G.P.
Dermatol
Resident
20
10
0
Verbal
Verbal+Visual
Visual
Hatala et al.
 ECG Diagnosis
Prior match / unmatch history
Postgraduate residents and med students
RESULTS
Percent mentioning
Percent of Diagnoses by Condition
50
45
40
35
30
25
20
15
10
5
0
Bias
No bias
Correct
Prior
Diagnosis
Medical
Students
Dual Processing and
Instruction
Role of Instruction in reasoning
Since NA (System 1) reasoning occurs at all levels, is
effective, is “automatic”
You can’t:
 tell student to not do it
 tell student to beware of biases
 tell student to think of better diagnoses
Does a coordinated strategy
improve accuracy?
 Norman, Brooks, Colle (ECG)
 Schmidt and Mamede (Gen Medicine)
 Ark & Eva, (ECG)
Norman, Brooks Colle, 2000
 Contrast instructions to:
 Think of the first thing that comes to mind, then consider
features
vs.
 Gather all the data then arrive at diagnosis
 32 Undergrad Psychology students
 11 disorders, rules + examples
 Test -- 10 new ECG’s
Diagnostic Accuracy
70
60
50
40
30
20
10
0
Pattern + Rules
Rules
Schmidt & Mamede, 2005
 42 I.M. residents
 16 written cases --- simple / complex
 Within subject/case design
 Instructions:
“First thing that comes to mind”
vs.
“Hypotheses, findings for/against, differential, ….”
Diagnostic Accuracy
80
70
60
50
Sim ple
40
Comp lex
30
20
10
0
Exemp lars
Rule s
ECG Diagnosis - Ark & Eva
 48 undergrad psychology students
 8 ECG diagnoses (A/A’, B/B’, C/C’,D/D’)
Instructions
• Compare and contrast vs. Sequential
• Combined Analytical/Non-analytical vs.
usual approach
Test
20 ECG’s (10 old, 10 new)
Immediate / 1 week later
Effect of Examples and Instructions
on New Cases after One Week
80
75
70
65
60
55
50
45
40
35
30
Combined
No Instruction
Compare
Ark & Eva, 2005
No Compare
Conclusions - Dual Processing
and Diagnosis
 Evidence that clinicians access both kinds of
knowledge/ use both processes
 Evidence that with increasing experience, greater
reliance on system 1
 Evidence that students benefit from explicit
instruction to use both
Dual Processing and Thinking
 DP and levels of processing
 DP and perception
 DP and transfer
 DP and aging
Dual Processing and Levels of
Processing
Are conceptual structures and
deep processing an underpinning
to development of System 1 (nonanalytic) knowledge?
(Dreyfus)
Role of Basic Science in Novice Reasoning
(Woods, Brooks, Norman, 2003)
 4 neurology / muscular diseases
 36 medical students
 Basic Science or Symptom/Disease
probability
Measurement
 Diagnostic Test
15 cases, 4-6 features
 Administered at 0, 7 days
Score on Dx Test
55
50
45
Feature List
Basic Sci
40
35
30
Immediate
1 Week
Score on Dx Test
55
50
45
Feature List
Basic Sci
40
35
30
Immediate
1 Week
Score on Dx Test
55
50
45
Feature List
Basic Sci
40
35
30
Immediate
1 Week
Dual Processing and
Perception
Word Superiority Effect
 Higher - level concepts (words) in memory facilitate
recognition of elements of words and pseudo-words:
RINK
BINK
NRIK
- possibly because of rapid (top-down) then slow (bottom up)
processing
Influence on Feature Interpretation
 Diagnostic hypotheses arise from pattern
recognition processes based on similarity to
prior examples
 In situations of feature ambiguity,
hypotheses may influence what is seen
Influence of Diagnosis on Feature
Perception (LeBlanc et al)
 20 residents, 20 final year students
 8 photos of classical signs from clinical
diagnosis textbooks
 Correct history and diagnosis
vs.
Incorrect history and diagnosis
RESULTS
Diagnostic Accuracy by Bias
90
80
70
60
50
Student
Resident
40
30
20
10
0
Correct
Alternate
No. of Features
RESULTS
Number of Features of Correct
Diagnosis by Condition
0.5
0.45
0.4
0.35
0.3
0.25
0.2
0.15
0.1
0.05
0
Student
Resident
Correct
Alternate
Diagnosis
RESULTS
Number of Features of Alternate
Diagnosis by Condition
No. of Features
0.25
0.2
0.15
Student
Resident
0.1
0.05
0
Correct
Alternate
Diagnosis
Dual Processing and Transfer
Although medical (and other) study is
directed at conceptual learning, use of
conceptual knowledge to solve
problems (transfer) is rare and
difficult.
WHY????
Spontaneous Transfer
 8 high performing undergrad (Health Sciences) students.
 3 principles (Laplace, Poiseuille, Starling)
 12 test cases
 Score
 0= wrong answer,
 1= right answer, wrong explanation
 2 = right answer, right but poor explanation
 3= right answer, good explanation
Laplace’s Law
In a cylindrical vessel, the wall tension is
proportional to the radius and pressure
exerted by the vessel contents. This can be
expressed as T = PR where T is wall
tension, P is pressure exerted by the
contents, and R is the radius of the vessel.
A 72 year old female has been diagnosed with an
aneurysm (dilatation) of the aorta. The doctor tells
her that if it grows to 5 cm in diameter she will need
surgery to prevent bleeding. Explain why the
increasing diameter is a problem.
Average Score
15.2/36 = 42%
“…during early learning, the principle is only
understood in terms of the earlier example… the
principle and example are bound together. Even if
learners are given the principle or formula, they
would use the details of the earlier problem in
figuring out how to apply that principle to the
current problem”
(Ross, 1987)
Why are the examples so
seductive?
System 1
Fast, unconscious, contextualized, concrete
System 2
Slow, logical, abstract
Transfer amounts to overriding System 1 to
utilize abstract, conceptual information
DP and Age
 Evidence from psychology that with
increasing age, we rely more on
System 1 thinking
Eva & Cunnington, 2006
 15 family docs, 7 < 60 yr., 8>60 yr.
 8 cases:
2 diagnoses, 4 conditions
Generated, Provided, Privileged, Extreme
--------------->>>>>> weight on second
diagnosis
Diff (Dx 1 - Dx2)
50
40
30
20
Young
Old
10
0
-10
-20
-30
Gen
Prov
Priv
Extreme
Some Last Words
(from other people)
{the expert} does not solve problems. He
does not even think. He just does what
normally works, and, of course, it
normally works…. The expert is simply
not following any rules! He is…
discriminating thousands of special
cases.
H Dreyfus
“In general, to preserve expertise we
must foster intuition at all levels of
decision-making, otherwise wisdom
will become an endangered species of
knowledge.”
H. Dreyfus
First and Last Word on Expertise
“It is a profoundly erroneous truism,
repeated by all copy-books and by eminent
people making speeches, that we should
cultivate the habit of thinking about what
we are doing. The precise opposite is the
case. Civilization advances by extending the
number of operations which we can perform
without thinking about them. Operations of
thought are like cavalry charges in a battle -they are strictly limited in number, they
require fresh horses, and must only be made
at decisive moments.”
A.N. Whitehead, 1911 (in J Bargh, 1999)
Bibliography
 Dreyfus HL From Socrates to expert systems: The
limits and dangers of calculative rationality.
http://socrates.berkeley.edu
 Evans J St BT. In two minds: dual - process accounts
of reasoning. Trends in Cognitive Science 2003; 7:
454-459
 Evans J StBT. Dual processing accounts of reasoning,
judgment and social cognition. Ann Rev Psychol
2008;59: 255-78.