A Logic of Diversity II
Scott E Page
Complex Systems, Political Science, Economics
and
Institute for Social Research
University of Michigan
Santa Fe Institute
Michigania 02005
Enlarging The Mantra
Identity,
Training,
Diverse
Better
Experiential
Perspectives
Outcomes
Diversity
Michigania 02005
Monday’s Talk:
Unpacking The First Box
Diverse
Perspectives
Michigania 02005
Monday’s Talk:
Unpacking The First Box
Perspectives
Heuristics
Interpretations
Michigania 02005
Today’s Talk:
Demonstrating Causality
Better
Diverse
Perspectives
Michigania 02005
Outcomes
Specific Tasks
Problem Solving
Prediction
Preference Aggregation
Michigania 02005
Why Construct Models?
Models allow us to provide conditions for when
a statement is true.
The Pythagorean Theorem: ``A-squared equals
B-squared plus C squared’’ only holds for
right triangles.
Michigania 02005
Finding the Conditions
``Two heads are better than one!’’
``Too many cooks spoil the broth’’
Which one wins? Which do we apply in a given
setting.
Michigania 02005
Finding the Conditions
``Two heads are better than one!’’
``Too many cooks spoil the broth’’
Condition: For an irreversible process, too
many cooks spoil the broth.
Michigania 02005
Swarm of Bees
Almost all of social science looks at averages
and changes in those averages.
Analogy: if you look at a swarm of bees, the
path of any one bee is hard to predict and
understand, but in the swarm all of those
idiosyncratic behaviors cancel out and we
can identify general trends.
Michigania 02005
The Buzz
Bee hives must stay around 96 degrees in order
for bees to reach maturation. Bees achieve
this by genetic mechanisms that drive two
behaviors:
When hot: fan out or leave the hive
When cool: huddle together
Michigania 02005
Diversity and Homeostasis
Genetically homogeneous bees: All get cool
(or hot) at the same time. Temperature in
hive fluctuates wildly. (1930’s heating
system)
Genetically diverse bees: Get cool (or hot) at
different temperatures. Temperature
stabilizes.
Michigania 02005
Rethinking the Swarm
The logic of cancellation does not hold because
there are feedbacks between the bees.
Those feedbacks imply we cannot look at
averages.
Groups of people solving problems, making
predictions, and making choices create
feedbacks in abundance.
Michigania 02005
Problem Solving
Michigania 02005
Problem Solving
Perspectives
Heuristics
Michigania 02005
The Idea
135
A,b,x
91
Thermometer:
SAT,IQ
x,M
Toolbox:
skills, heuristics
Michigania 02005
Perspective
number of chunks
size
Ben & Jerry’s Ice Cream Array
Michigania 02005
Heuristic
number of chunks
size
Ben & Jerry’s Ice Cream Array
Michigania 02005
Consultant
perspective:
caloric rank
Michigania 02005
Consultant
perspective:
heuristic:
caloric rank
look left and right
Michigania 02005
Performance
• Average Performance Given
– solution in perspective
– application of heuristics
• Ben and Jerry
– average quality of solution = 82
• Consultant
- average quality of solution = 74
Michigania 02005
Perspective Diversity
Ben and Jerry
stuck at 83
75
80 83 73
81
Michigania 02005
Perspective Diversity
Ben and Jerry
stuck at 83
consultant
Gets to 86
75
80 83 73
81
80 86 83 74
Michigania 02005
Diversity or Ability: A Test
Create a bunch of artificial problem
solving agents and rank these agents
by their average performances on a
difficult problem.
All of the agents must be “smart”
Michigania 02005
Two Groups
• Group 1: Best 20 agents
• Group 2: Random 20 agents
Have each group work collectively - when one
agent gets stuck at a point, another agent
tries to find a further improvement. Group
stops when no one can find a better solution.
Michigania 02005
The IQ View
139
138
137
121
84
111
135
132
135
75
135
31
Alpha Group
Diverse Group
Michigania 02005
And the winner is..
“Most of the time” the diverse group
outperforms the group of the best by
a substantial margin.
See Lu Hong and Scott Page Proceedings of the
National Academy of Sciences (2002)
Michigania 02005
The Toolbox View
ABC
ABD
ACD
AHK
FD
AEG
BCD
ADE
BCD
EZ
BCD
IL
Alpha Group
Diverse Group
Michigania 02005
Formal Version
Theorem: Given a set of diverse problem
solvers, a random collection
outperforms a collection of the “best”
individual problem solvers provided
-the set is large
-the problem is hard
-the problem solvers are smart
Michigania 02005
.
Prediction
Michigania 02005
Prediction
.
Interpretations
Mental Models
Michigania 02005
The Madness of Crowds
We tend to think of crowds of people as
irrational mobs. And that can be true.
When people hear the ideas and
opinions of others, they often succumb
to peer pressure rather than speaking
their own minds.
Michigania 02005
Which Line is Longer?
A:
_____________
B:
___________
Michigania 02005
The dim boy claps because the others clap.
- Richard Hugo
Michigania 02005
The Wisdom of Crowds
If people do not hear the opinions of
others, or if they render their true
predictions anyway crowds can be
incredibly wise.
Michigania 02005
Suroweicki’s Examples
Morton Thiokol’s stock plunge
Prediction Markets
Hollywood Stock Exchange
Iowa Electronic Market
Sports Betting Markets
Who Wants to be a Millionaire
1906 West of England Fat Stock and
Poultry Exhibition
Michigania 02005
Two Separate Phenomena
1. Information known by part of the crowd
2. Aggregative diverse predictive models
Michigania 02005
Revealing Known Information
Which of the following books would you NOT find in
the Point o’ Pines Library
A. The Periwinkle Steamboat - Lancaster
B. Curtains - Agatha Christie
C. Unabridged Crossword Puzzle Dictionary
D. I am Charlotte Simmons - Tom Wolfe
Michigania 02005
Information Rising
Suppose that no one know the answer but
that 18 people know one of the books
on the list is in the library and that 18
people know two of the books on the
list are in the library. This means that
64 people guess randomly.
Michigania 02005
Information Rising
Of 64 Clueless:
Of 18 know one:
Of 18 know two:
Correct answer gets 16
Correct answer gets 6
Correct answer gets 9
Total
31
Other answers get 23 (on average)
Michigania 02005
The Answer Is…
Which of the following books would you NOT find in
the Point o’ Pines Library
B. Curtains - Agatha Christie
Michigania 02005
Aggregating Diverse Predictions
In most of the situations described, people do
not know the answer yet. We can assume
that people have diverse predictive
models. We’d like to understand how that
aggregation occurs and what roles
diversity and ability play.
Michigania 02005
Reality
H
Charisma
MH
ML
L
H
G
G
G
B
MH
G
G
G
B
G
G
B
B
B
B
G
B
B
Experience
ML
L
Michigania 02005
Experience Interpretation
75 % Correct
H
G
G
G
B
G
B
MH
G
G
G
B
G
G
B
ML
G
B
B
B
B
L
B
G
B
B
B
Experience
Michigania 02005
Charisma Interpretation
75%
Correct
H
MH
ML
L
G
G
G
B
G
G
B
G
G
G
B
B
B
B
G
B
B
G
B
G
B
B
Michigania 02005
Balanced Interpretation
H
75% Correct
H
Good to be
extreme on one MH
measure, bad
on other
ML
L
MH
ML
G
B
G
G
B
G
B
G
B
G
G
B
B
G
B
B
G
G
B
B
Michigania 02005
L
Voting Outcome
H
Charisma
MH
ML
L
H
GGB GGG GBG BGB
MH
GGG GGB GBB
G GBG
ML
BGG BBG BBB BBG
L
BGB BGG BBG BBB
Michigania 02005
The Mathematics of Prediction
Prediction: # runs scored by winning
softball team
Mon
Tue Wed
Brad
8
10
9
Matt
10
12
8
Michigania 02005
“Crowd’’ Prediction
Brad
Matt
Crowd
Mon
8
10
9
Tue Wed
10
10
12
8
11
9
Michigania 02005
Actual Numbers
Brad
Matt
Crowd
Actual
Mon
8
10
9
8
Tue
10
12
11
12
Michigania 02005
Wed
10
8
9
9
Squared Errors
Brad:
(8-8)2 +(10-12)2 +(10-9)2 = 5
Matt :
(10-8)2 +(12-12)2 +(8-9)2 = 5
Crowd:
(9-8)2 +(11-12)2 +(9-9)2 = 2
Michigania 02005
Diversity of Predictions
(Brad-Crowd)2 = 1 + 1 + 1 = 3
(Matt-Crowd)2 = 1 + 1 + 1 = 3
Michigania 02005
Notice: 2 = 5 - 3
Crowd Error = Average Error - Diversity
Michigania 02005
Diversity Prediction Theorem
Crowd Error = Average Error - Diversity
(note: proven by statisticians, computer scientists, and economists)
Michigania 02005
Crowd = Average - Diversity
• Diversity as important as ability
• Limit to how much diversity
(otherwise crowd error would be negative)
Michigania 02005
Experts on NFL Draft
Player
Alex Smith
Ronnie Brown
Braylon Edwards
Cedric Benson
Carnell Williams
Adam Jones
Error^2
Average Error:
Diversity:
#1 #2 #3 #4
1 1 1 1
2 2 4 2
3 3 2 7
4 4 13 4
8 5 5 5
16 9 6 8
#5
1
2
3
8
4
6
#6
1
5
2
4
13
6
#7
1
2
3
8
4
9
158 89 210 235 112 82 39
153.13
101.52
Michigania 02005
#8
2
6
3
4
8
17
300
Crowd of Experts on NFL Draft
Player
Alex Smith
Ronnie Brown
Braylon Edwards
Cedric Benson
Carnell Williams
Adam Jones
Crowd
1.13
3.13
3.25
6.13
6.50
9.63
Error^2
51.61
Michigania 02005
Crowds Beat Averages Law
Crowd Error < Average Error
Michigania 02005
Does Crowd Beat Best?
In the NFL draft example, the best predictor
Pete Brisco had an error of only 39. He
outperformed the crowd, which had an
error of 51.6.
Michigania 02005
Novices and Experts
Novices: Base their models on only a few
variables or a few boxes.
Experts: Base their models on many
variables or many boxes.
Michigania 02005
In Praise of Experts
Theorem: If an expert contains every
variable considered by any one of the
novices, the expert predicts better than
the crowd of novices.
Michigania 02005
Crowds vs Experts
Test Set: Linear functions defined over 20
variables.
Crowd: Each of 100 novices looks at N
randomly chosen variables
Expert: Looks at E>N variables
Training: 300 independent variables
Contest: 300 independent variables
Michigania 02005
Crowds vs Experts
N
3
3
5
5
E
20
15
10
7
% of Time
Expert Wins
94.66%
34.66%
29.33%
9%
Michigania 02005
What’s Happening
Expert: Getting best fit over all his
variables.
Crowd: Getting an average of many fits
over many distinct subsets of variables.
Michigania 02005
Put Another Way
Expert: Great partial view
Crowd: So-so complete view
Michigania 02005
Diversity and Prediction
Diverse predictors generate better
predictions unless someone’s head is
large enough and data is sufficient
enough for a complete model.
Michigania 02005
Preference Aggregation
Michigania 02005
Instrumental vs Fundamental
Fundamental Preferences: Preferences
over outcomes
Instrumental Preferences: Preferences
over policies to attain outcomes
Michigania 02005
Instrumental Politics
“I am the _____ candidate”
A. Pro crime
B. Anti child
C. Anti environmental
D. Pro drug addiction
E. Higher health care costs
Michigania 02005
Preference Diversity Problems
• Preference Cycles
• Manipulation
• Underprovision
Michigania 02005
Preference Cycle
A = Arts & Crafts, B = Boating, T = Tennis
Lindsey: A > B > T
Samuel: B > T > A
Becca : T > A > B
Michigania 02005
Preference Cycle
Lindsey: A > B > T
Samuel: B > T > A
Becca : T > A > B
• Majority Vote Outcome: A > B > T > A
Michigania 02005
Manipulation
Given any voting rule, people with diverse
preferences will always have an incentive to
misrepresent themselves.
Implication: People in diverse groups will not
trust one another as much.
Michigania 02005
Under Provision
If we want different outcomes and have a
fixed budget, we are likely to spread our
money too thin.
Idea: Rather than have a good car or a
nice boat, we have a lousy car and a
lousy boat.
Michigania 02005
Theoretical Summary
Tasks involve
- Solution Generation (problem solving)
- Evaluation (prediction)
- Choice (preference aggregation)
Michigania 02005
“Diversity is Ability”
To be different is to be able to make a
contribution.
Diversity Trumps Ability: Diverse group
does better than “able” group at problem
solving
Michigania 02005
“Diversity is Ability”
Diversity Prediction Theorem
Crowd Error = individual error - diversity
(ability and diversity enter equally)
Michigania 02005
Complication
Preference diversity creates cycles. It
creates incentives to act strategically
and to manipulate agendas.
At the same time, preference diversity
may be a primary cause of the other
types of diversity.
Michigania 02005
Summary
The empirical evidence suggests that
diverse perspectives, mental models,
and tools lead to “better outcomes” but
that value diversity creates problems.
Michigania 02005
Pudding
Michigania 02005
Quick Look at the ``Facts’’
•
•
•
•
Growth of modern civilization
National level GDP
City level productivity
Diverse team performance
Michigania 02005
Rise of Modern Civilization
• Jared Diamond: diversity/easy problems
• Joel Mokyr: exploiting diversity
• Michael Kremer: 1 million years of data
shows growth and population size
correlated
Michigania 02005
National Level GDP
• Paul Romer: Diversity crucial to
economic growth
• Ethnic Linguistic Fractionalization (ELF):
strongly negatively correlated with
economic growth
Michigania 02005
Performance of Cities (42)
• Doubling of city size increases
productivity by 6% to 20%
• Arrow, Lucas: spillovers within an
industry (silicon valley)
• Jacobs, Auerbach: spillovers between
industries (just in time)*
Michigania 02005
Identity Diverse Teams
• Generate more solutions (many worse)
• Thomas and Ely: do better if they have
diverse heuristics and perspectives
• People in diverse groups are less happy
- world views are challenged
- feel like outcomes were manipulated
Michigania 02005
The End of Great Scientists
First 10
Physics Nobels:
14
Chemistry Nobels: 10
Michigania 02005
The End of Great Scientists
First 10
Physics Nobels:
14
Chemistry Nobels: 10
(There’s a maximum of three)
Michigania 02005
Last 10
28
27
Final Thought
Individual ability not likely to grow much.
Collective diversity can grow.
Diversity is our best hope to solve
problems and to create innovations.
Michigania 02005
www.cscs.umich.edu/~spage
Michigania 02005