Contexts in Which Best and
Worst CBC are Most Valuable:
Application to School Choice
Joel Huber: Duke University
Namika Sagara: Duke University
Angelyn Fairchild: Research Triangle Institute
Why Study School Choice?
• School choice is an increasingly difficult reality
for parents.
• It is possible to mimic aspects of the actual
choices
• There are many analyses of actual school
choices but few published conjoint studies
• Positive and negative reaction to features
makes school choice ideal for Best/Worst
choice based conjoint
Our process
• Identified attributes differentiating public
schools in actual choices
• Pretested these to make sure we covered the
most important ones with 4 continuous
attributes and 4 binary ones
• Built a fixed B/W CBC design
• Ran the study on a national sample of 150
parents with a child entering grades 6-11.
Our Sample
• Sample: Parent of a child age 11 to 17 attending
public school
–
–
–
–
57% Female
59% Caucasian
59% had at least some college
Median income was between $50,001 and $75,000
• “Think about your youngest child that is more than
11 years old.”
– Age and gender
– School grade
– Select five important characteristics for child’s school (e.g., close to home)
An Important One-time Decision
• “Suppose you just moved to a new area where
families are able to choose which school they would
most like their children to attend.”
• Introduction to attributes and practice questions
– We defined ranges of each attribute and asked relative
importance
– We built up gradually to more complex, realistic choices
Ultimate choice task
Attribute 1: Distance
Travel time to school
Bus Ride
5 min
15 min
30 min
45 min
“About how many minutes does it take for your child to get to
school now?”
Attribute 2: Academic Quality
School Attributes
Travel time
5 min
15 min
30 min
45 min
% under Grade
Level
15%
25%
35%
45%
“At your child's current school, what percent do you think are below grade
level?”
Warm up Choice 1
“Next we will ask you to choose between two schools with different
travel time and percent of students below grade level.
Imagine that only two school options are available for your child, and
the schools are the same except for the differences shown below.”
Attribute 3: Income
School Attributes
Travel time
5 min
15 min
30 min
45 min
% under Grade
Level
15%
25%
35%
45%
% Economically
Disadvantaged
10%
30%
50%
70%
“At your child’s current school, what percent of students are economically
disadvantaged?”
Attribute 4: Diversity
School Attributes
Travel time
5 min
15 min
30 min
45 min
% under Grade
Level
15%
25%
35%
45%
% Economically
Disadvantaged
10%
30%
50%
70%
Percent
Minority
25%
40%
55%
70%
“At your child's current school, about what percent of students are
minorities?”
Warm up Question 2
Featured Programs
International Baccalaureate (IB) Program
Featured Programs
International Baccalaureate (IB) Program
Science, Technology, Engineering and Math (S.T.E.M)
Featured Programs
International Baccalaureate (IB) Program
Science, Technology, Engineering and Math (S.T.E.M)
Expanded Arts Program
Featured Programs
International Baccalaureate (IB) Program
Science, Technology, Engineering and Math (S.T.E.M)
Expanded Arts Program
Promote Sports Teams
One of 8 Best/Worst Choice Tasks
“Best” Schools
150
% Below
Grade Level
100
Travel Time
%
Economically
Disadvantaged
50
%
Minority
0
15 25 35 45
-50
-100
-150
5
15 30 45
10 30 50 70
25 40 55 70
Arts
Sports
IB
STEM
“Worst” Schools
150
% Below
Grade Level
100
Travel Time
% Economically
Disadvantaged
50
%
Minority
0
15 25 35 45
-50
-100
-150
5
15 30 45
10 30 50 70
25 40 55 70
Arts
Sports
IB
STEM
What is important in a school?
1.0
Academic quality
0.5
Income
0.0
-1.0
-0.5
0.0
0.5
% Minority
STEM
-0.5
Travel
time
Sports
-1.0
Arts
1.0
What is important in a School?
1.0
Academic quality
0.5
Income
0.0
-1.0
-0.5
0.0
0.5
IB
% Minority
STEM
-0.5
Travel
time
Sports
-1.0
Arts
1.0
Who cares?
1.0
Academic quality
Educated Parent
0.5
Nonwhite
Income
Older child
0.0
-1.0
-0.5
% Minority
0.0
0.5
Part time
Travel
-0.5
time
Sports
-1.0
IB
Lower income
STEM
Arts
1.0
Who cares?
1.0
Academic quality
Educated Parent
Employed full
time
0.5
Nonwhite
Higher income
Income
Older child
0.0
-1.0
-0.5
0.0
Younger Child
% Minority
Part time
Travel
-0.5
time
Less educated Sports
parent
White
-1.0
0.5
IB
Lower income
STEM
Arts
1.0
What we learned
• By building complexity gradually it is possible
to generate reliable responses for a difficult
and important choice
• 8 choices is sufficient to separate those with
quite different values
• B/W provides both insight into what is desired
and feared, and generates stable individual
estimates
Simple individual level estimation
• All the analysis used standard Sawtooth
Software HB analysis
• Can we generate a simplified model that can
allow feedback to subjects on the fly?
• Concert 4-level variables to linear, producing 4
linear, 4 binary variables from 8 Best and 8
Worst choices
• This follows from work by Saigal and Dahan
(Sawtooth Software Proceedings 2012)
Individual Linear Choice Using
Simple Vector Product
• Choice vector Y has 32 items, 4 for each
choice set, code Best as a 1, Worst as -1
and Zero otherwise
• Design matrix X(32,8) is zero centered
within each choice set
• B-hat = (X’X)-1X’Y
• Since we use a fixed design X we multiply
(X’X)-1X’ (8 x32) by Y(32x1) to get B(8x1)
Correlation between HB and linear
choice model
1
0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0
% Under Grade
Sports
STEM
Arts
Minority
Bustime
IB
Poverty
Correlation of HB and Linear with Self
Explicated
0.60
0.50
0.40
0.30
0.20
0.10
0.00
Academic
Bustime
STEM
Poverty
Arts
Minority
IB
Sports
Effective use of Best/Worst Choice
Based Conjoint
• Best/Worst is appropriate for choices with
levels people avoid or fear
• Where there is both attraction and avoidance,
combining Best + Worst choices results in
better results
• Analyzing Best + Worst with a linear model
generates results reasonably close to HB
analysis
Thank you!
What is important in a school?
1.0
Academic quality
0.5
Income
STEM
0.0
-1.0
-0.5
0.0
Income
0.5
IB
% Minority
STEM
Travel time
Arts
-0.5
Sports
-1.0
1.0
Correspondence between HB and
linear estimates
HB Importance Estimates