When Can
Diversity Trump
Ability?
Dan Teague
NC School of Science
and Mathematics
Scott Page, Lu Hong, John Miller
Wisdom of the Crowd
Information Aggregation
On Who Wants to be
a Millionaire, the
lifeline is correct 2/3
of the time.
The audience is
correct 9/10
times!
Identify the non-Monkee
a)
b)
c)
d)
Peter Tork
Michael Nesmith
Roger Noll
Davy Jones
Crowd of 100 people
•
•
•
•
7 know all three of the Monkees listed
10 know two of the Monkees listed
15 know only one of the Monkees listed
68 know nothing about the Monkess
What would the wisdom of the crowd
show?
The Vote
• 7 know all three of the Monkees
7 votes for Roger Noll
• 10 know two of the Monkees
Expect 5 Votes for Roger Noll
• 15 know only one of the Monkees
Expect 5 Votes for Roger Noll
• 68 know nothing about the Monkees
Expect 17 Votes for Roger Noll
34 votes for Roger Noll
Goldcorp Challenge
In 1999 CEO Rob McEwen
instructed his geologists to release
all geological records to the public.
The “Goldcorp Challenge” offered
$575k to anyone who could find
the gold and drew 1,200 people
from 50 countries.
Results:
• 110 sites identified 50% new,
80% produced gold.
• 8 million ounces found.
• Company value up from $100
million to $9 billion.
Galton’s
Steer
At the 1906
West of England
Fat Stock and Poultry Exhibition, 787 people
guessed the weight of a steer. Francis Galton
collected the data and found the average guess
was 1,197 pounds. The actual weight of the steer
was 1,198 pounds.
Accuracy of Group
Predictions
Computing the Squared Error
(Variance)
Computing the Crowd’s
Diversity
Diversity Prediction Theorem
38
=
39
- 1
In the case of Galton’s steer, the collective
accuracy was approximately 1, the average
accuracy was 2,956 and the diversity was 2,955.
This means that individuals missed by about 55
or 60 pounds each. The fairgoers owe their
collective accuracy more to their remarkable
diversity than to the prescient individual
abilities.
When Diversity Matters
• If Average Error is small, the task is easy.
Diversity doesn’t help.
• Group-think and deference to experts reduce
the quality of the decisions.
Cognitive Diversity Matters
in Problem Solving
Page Distinguishes:
Diversity in Perspective
Diversity in Heuristics
Sum to Fifteen
(Herb Simon, Nobel Prize in Econmics)
One player randomly chosen to go first.
Alternate turns selecting cards.
The winner is the first player who has
exactly 3 cards which sum to 15.
Picnic Basket Game
•
•
•
•
•
•
•
•
Nachos (N)
Eggs (E)
Sausage (S)
Water (W)
Hot Dogs (H)
Vinegar (V)
Lemons (L)
Raisins (R)
Goal is to Collect all 3
Copies of One Food Item
Tic Tac Toe
Perspectives
• Trig functions
If y = sin(x), is x
a) an angle
b) a distance around the unit circle
c) a position on the real axis?
• Coordinate Systems: Cartesian, Polar,
Spherical
• Functional representations: recursive,
parametric, vector
Mendeleyev’s Periodic Table
Einstein and Bohr
Every Tuesday from 3:00 to 4:00 in the afternoon,
Niels Bohr and Albert Einstein would play duets
together when they were both at Princeton.
Einstein would play
the violin and Bohr
would play the piano.
How would the history of science be different if
Einstein had played the discrete instrument and Bohr
the continuous instrument?
Differing Perspectives
Perspectives
• When people see a problem from the
same perspective, they are likely to get
stuck on the same local peaks.
• New perspectives can clarify or muddy.
Sum to 15 can be seen as Tic Tac Toe or
as the Unpacking Game.
Heuristics
• Process by which solutions are found
within a perspective. Problem Solving
Techniques.
• Calculus
• Do the Opposite (Castanza Rule Princeline.com)
• Error Allowing Heuristics (Explore vs Exploit)
Simulated Annealing
(aka Brainstorming)
IQ Test Question or 1+1=3
In each sequence, replace the X with the unique
number that makes the sequence logically
consistent.
• Sequence 1:
• Sequence 2:
• Sequence 3:
1 4 9 16 X 36
1 2 3 5 X 13
1 2 6 X 1,806
• Sequence 1:
1 4 9 16 25 36
Square
• Sequence 2: 1 2 3 5 8 13
Differences
1 2 3
5
• Sequence 3:
1 2 6
X
1,806
1+1 = 3 or maybe 4
Sequence 3:
1 2 6
X 1,806
42
Differences and Squares
2 – 1 = 12
6 – 2 = 22
X – 6 = 62
1,806 – X = 422
• Perspectives are ways of seeing the problem.
They create different landscapes.
• Heuristics are ways of constructing solutions.
• The more productively a perspective organizes
reality, the more heuristics people can create
to work in that perspective.
• Innovations can arise from rearranging the
box with a new perspective or from exploring
parts of the box that have been ignored with
new heuristics.
• Diverse perspectives are more likely to lead to
breakthroughs; diverse heuristics are more
likely to leader to iterative improvements.
Multiple Perspectives and
Heuristics are Essential
We Now Work in Teams
…and when working in teams…
Selecting Talent
(how we think about who’s good)
Scores vs Toolkits
Two
Views of
Ability
Under what conditions does
diversity trump ability?
Page’s Initial Experiments
Condition #1
“Calculus Condition”
All Problem Solvers are Smart
(relevant congnitive skills)
All problem solvers can move the
ball up or, at least, keep it
at the same level.
Condition #2
The Problem is Difficult
No individual problem solver
always locates the global
optimum
Condition #3 Diversity
Any solution other than the global
optimum is not a local optimum for some
non-zero percentage of problem solvers.
When one agent gets stuck, there
is always another agent that can
find an improvement using a
different perspective or heurisitic.
The intersection of all local max
contains only the global max.
Condition #4
Good-sized collections drawn at random
from large population of potential problem
solvers.
The initial population of problem
solvers must be large and the
collections of problem solvers
working together must contain more
than a handful of problem solvers.
Diversity Trumps Ability
Theorem
Given conditions 1-4, a randomly
selected collection of problem
solvers outperforms,
on average,
a collection of the best individual
problem solvers. (in practice, a.s.)
The Theorem
The Basic Idea of the Proof
Making a Difference: Applying
the Logic of Diversity
• The best problem solvers likely have similar
perspectives and heuristics. The random
problem solvers bring diverse ways of
thinking.
• The best problem solvers all get stuck at the
same place. The random problems solvers
don’t.
•
Academy of Mangement Perspecitves, Nov. 2007, page 11
Making a Difference: Applying
the Logic of Diversity
The best problem solvers likely have similar
perspectives and heuristics. The random
problem solvers bring diverse ways of thinking.
The best problem solvers all get stuck at the
same place. The random problems solvers
don’t.
Scott Page, from Academy of Mangement Perspecitves, Nov. 2007, page 11
Making a Difference: Applying
the Logic of Diversity
The logic of the theorem does not imply the
irrelevance of ability. …Ability still matters, but
so does diversity.
And, as the theorem shows, once an ability
threshold has been met, diversity matters more
than ability.
Scott Page, from Academy of Mangement Perspecitves, Nov. 2007, page 11
Problems of Diversity
Communication
(Problem solvers with diverse perspectives may have difficulty
understanding each other)
Misunderstanding and Mistrust
Less comfortable atmosphere
(We are all more comfortable with like-minded individuals
If people do not believe in the value of diversity, then when part of a
diverse team, they are not as likly to produce good outcomes.)
Final Thoughts from Scott Page
Our individual abilities are not likely to
growth much anytime soon.
Our collective diversity can grow.
Diversity is our best hope to solve
problems and to create innovations.
References
Scott Page, The Difference, Princeton University Press, 2007.
Scott Page, Making the Difference: Applying a Logic of Diversity, Academy of Mangement
Perspecitves, Nov. 2007. (Google Scott Page Academy of Mangement )
Lu Hong and Scott Page, Groups of diverse problem solvers can outperform groups of high-ability
problem solvers , PNAS November 16, 2004 vol. 101 no. 46 16385–16389. (Google Hong and
Page)
Scott Page, Diversity and Complexity, Princeton University Press, 2011.
John Miller and Scott Page, Complex Adaptive Systems, Princeton University Press, 2007.