Round Numbers as Goals:
Evidence from Baseball, SAT & ‘the Lab’
(with Devin Pope,
In press, Psychologial Science)
The Paper in one slide
• Rosch (Cog Psych 1975): ‘Cognitive Reference Points’
– Focal values in categories used to judge other values
• Our question: in a JDM way?
• Focus on performance scales
• Prediction:
P1: more effort just below RN
P2: more f() just above RN
Findings:
• Baseball:
– ‘Too many’ batters with a .300 batting average
8
– ‘Too many’ retake with __90 vs. __00
Lab: 7.7
• SAT:
•
– More likely to keep trying _9 vs. _0
Study 1: Baseball
Background
• Balls are thrown
• Batters take turns (“at-bats”)
• If ball is hit ~ >“hit”
• Batting average: “hits” / “at-bats”
• BA is a good DV because:
– Granular
– Paid attention to by players
• BA ~ {.200-.400}
Study 1: Baseball (2)
• Sole ‘round’ number: .300
• Hypothesis: batters disproportionately
prefer .300 to .299
• Predictions:
1) ‘too many’ .300 season averages
2) Try hard to get/keep .300
Data
• All player-seasons 1975-2008
– N=11,430
• Granularity: > 200 at-bats
– N=8,817
• Graphs will focus on those with .280-.320
– N=3,083
Graph: Batting Averages
(raw freqs)
At the end of the season
With 5 plate-appearences left
Z = 7.35, p<.001
How do batters achieve that?
• Next, look at last play of season.
– Hits
– Walks
– Substitutions
Do .300 players substitute more
out of their last at-bat?
Do .299 players ‘walk’ less?
Do .299 hit more on
their last at-bat?
Endogenous exit for sure.
Better actual performance, maybe.
Summary Study 1
• “too many” .300 season averages
• Achieved by
– Fewer walks at .299
– Substitutions at .300
– Maybe: greater hitting %.
Limitations
1. One round number  got lucky?
2. It is a small effect
– Not in p-value
– Not in SD
– In terms of consequences
• (just one play in the season)
3. Agents, managers, advertisers?
Study 2: SAT re-taking
• Many round numbers
• Stakes are larger
• Third party problem remains
– But addressed empirically
– Also: see Study 3
Background on the SAT
• Scored 400-1600
– Intervals of 10
• Retaking is allowed
– (about 50% do)
• HS Juniors and Seniors take it
• Prediction: “too many” retake it if
__90 vs __00
Data
•
•
•
•
College Board Test Takers Database
N= 4.3 million; 1994-2001
Last test only
Did individual retake it?
– D/K!
– Infer retaking rates from score distributions
Inferring Retaking Rates
• Don’t observe key DV
• But:
– Juniors can easily retake
– Much more difficult for seniors
• Juniors (but not seniors) should have
• “too few” __70,__80,__90 scores
• “too many” __00, __10 __20
Let’s see
Graph with raw frequencies next
SAT by Juniors and Seniors
A better graph
Plotting the slope
F(x)/F(x-10)
(Uri: Explain Ratio=1)
Graph with
F(x)/F(x-10)
Explain the
effect is not
ONLY at __90
Interpretation and
Alternative Explanations
• Find: big jumps in F(x) at _00 (for juniors)
• Infer: disproportionate retaking below _00
• Interpret: _00 is a goal
• BUT
1) Maybe _00 really is discontinuously better
• Version 1. Same effect, different agent
– (can live with)
• Version 2. Arbitrary thresholds
– (less so)
2) Maybe _00 is perceived as discontinuously
better by test-taker
Next, look at (1) & (2) empirically.
1) Is it discontinuously better to
get a _00 than _90 in the SAT?
• Compare admission with _90 and _00
• Data 1: (JBDM 2007) “Clouds Make Nerds Look Good”
– N=1100 undergrad admission decisions
– Null:
pr(admit|SAT=1000) -pr(admit|SAT=990)=
pr(admit|SAT=1010)-pr(admit|SAT=1000)
- Tested at:
-
1200, p=.96
1300, p=.99
1400, p=.20
1500, p=.92
- Small N, but nothing there directionally.
- SAT not that important.
Same test, different dataset
• Data 2: ‘Ongoing’ project with Francesca
Gino
– MBA admission decisions & GMAT (<800)
– GMAT=600, p=.09 (wrong sign)
– GMAT=700, p=.93
Alternative Explanations
1) Maybe _00 really is discontinuously better
2) Maybe _00 is perceived as discontinuously
better by test-taker
Back to SAT dataset
• Score sending reveals info.
• If _00 disc. better than _90
 scores sent to disc. different schools.
• Next: the graph
– Schools predicted by score
Summary
• Too many _70,__80,__90 retake SAT
– About 10%-20% percentage-points too many
• No effect on admission decisions
• No effect on score sending decisions
• We interpret:
– _00 (becomes) a goal influencing retake
decision if met/not-met.
Motivation of Study 3
• Studies 1 & 2 show large effects in the field
• Alternative explanation: third party
• Keep in mind though, that:
– Baseball managers think locus is players
• Also, here 3rd party locus is interesting.
– Does not predict admissions
– Does not predict where SATs are sent
• Study 3, eliminate by design
Study 3
• Scenarios inspired by Heath Larrick and
Wu (Cog Psyc 1999)
• “Imagine your performance is x”
• “how motivated to do more”? 1-7
• X is
– below round number
– just below round number
– above round number.
Scenario 1
Imagine that in an attempt to get back in shape, you
decide to start running laps at a local track.
After running for about half an hour and having done
[18/19/20
; 28/29/30] laps
you start feeling quite tired and are thinking that you
might have had enough.
How likely do you think it is that you would run one
more lap?
Results for 3 scenarios combined