Advanced MMBR
Conjoint analysis
Conjoint analysis -> Multi-level models
You have to understand:
- What it is
- Which different kinds of Conjoint
Analysis there are
- How it can be of use in typical TIW
research
- How it can lead to different kinds of
statistical analyses (of the repeated
measures or multi-level kind)
For this, you can use these slides AND
the literature online
In the laptop exam, running a
repeated measures analysis is part of
the requirements.
Advanced Methods and Models in Behavioral Research
The logic of the course
• binary Y
 logistic regression
• conjoint analysis: way of data
collection that might come in
handy
 "repeated measures" / "multilevel" data
We practice on self-collected data
 some practice/training in survey
design and execution
Advanced Methods and Models in Behavioral Research
Conjoint Analysis
Underlying assumption: for
each user, the "utility" of a
product can be written as
-
10 Euro p/m
2 year minimum
free phone
...
How do you rate this
proposal? (-5 ... +5)
U(x1,x2, ... , xn) = c0 + c1 x1 + ... + cn xn
Advanced Methods and Models in Behavioral Research
Two kinds of research questions
• Which phone do people tend to prefer?
• How do different attributes of a proposition affect
the utility of a proposition?
(and how does that differ
across different kinds of
people)
Advanced Methods and Models in Behavioral Research
Why is this important?
It is an important tool in
social science when you
want to investigate how
someone’s behavior or
evaluations depend on
circumstances
and
a useful tool in typical TIW
Master’s Theses
Advanced Methods and Models in Behavioral Research
One of the main advantages = more data:
Example - Adoption of technology
1. If you ask for behavior only (“did you
adopt” etc), then you get one piece of
info per person, and the rest you have
to infer by comparing different persons.
-> OK, and gives “real” behavioral data,
but data are sparse.
2. If you offer different scenarios and then
ask whether someone would adopt, you
get how adoption depends on the
context for this person.
-> Richer data per person, but not
behavioral
Given that sample sizes of >200 are
often necessary, often option 2 is more
feasible than option 1.
Advanced Methods and Models in Behavioral Research
Or, use it on ...
New light bulb
- price of bulb
- color of light
- expected lifetime
Submit a motion
in parliament
- ...
Would you be interested in buying
this light bulb?
(-5 ... +5)
-
strongly favor issue personally
core of party's strategy
many other parties against it
gets a lot of media attention
...
Do I submit a motion?
(-5 ... +5)
??
-...
-...
-...
-...
-...
Example: properties of mobile phones
Suppose you get a
phone that has …
-
large b/w screen
long battery life
not flashy
costs 6 Euro/month
free
monthly contract
How do you rate this
option? (-5 ... +5)
Hmm... How would
that work anyway?
1. Give each person 200 of these
cases (“vignettes”)
2. Per person, run multiple
regression on their 200 answers -> you get (personal) values for
each of the dimensions.
You could then:
1. Create groups of people with the
same kind of values, or …
2. … get an estimate of the average
trade-off between dimensions, or
…
3. … compare different groups of
respondents
Advanced Methods and Models in Behavioral Research
The data would look like this:
ISSUES
Y
D1
D2
D3
D4
D5
id
+4
-1
-1
0
1
0
1
• The D-values are
chosen by the
researcher ->
experiment
-3
1
1
1
0
-1
1
0
0
0
1
0
-1
1
0
1
0
-1
1
0
1
+1
…
…
…
…
…
1
• 200 is way too much
+1
…
…
…
…
…
1
-1
…
…
…
…
…
1
• 35 = 243 > 200
(or even: 410 = …)
+4
…
…
…
…
…
1
…
…
…
…
…
…
…
• If you look at the
complete data set, this
is repeated measures:
the data are clustered
-> standard multiple
regression will not work
Other considerations
-
How many dimensions?
How many levels per dimension?
How many cases per person?
Which cases from all possible cases per person?
How many persons (sample size)?
Judgment or choice?
How do I make sure that all given cases are ok?
Advanced Methods and Models in Behavioral Research
…
- Sidestep: design of experiments literature • Suppose you have a large set of
potential vignettes (way too
large to use them all)
• One option: choose a random
subsample of all possible
vignettes
• ... but that might not be
optimal, here’s why:
-1
(X’X)
Advanced Methods and Models in Behavioral Research
Alternatives (1)
• Ask people directly what their favorite phone is
If you could choose, which phone (and monthly
plan) would you prefer?
+ much easier to collect
- you will end up with as many suggestions as you
have respondents
- does not capture trade-offs between attributes
Advanced Methods and Models in Behavioral Research
Alternatives (2)
• Ask people directly for "their" utility model
For instance:
Which of the following factors do you find important when it
comes to <buying a phone> / <adopting an innovation> /
<choosing a buyer on eBay> / ...?
Please divide 100 points over the following factors:
- price
- battery life
- ...
+ much easier to collect
- asks for introspection about choice process, not for a “real”
choice
- nonlinear effects can never occur like this
Advanced Methods and Models in Behavioral Research
Advantage of conjoint analysis (subtle)
Usually, the comparison of interest is within persons
person
person
person
person
person
1
2
3
4
5
But if you consider only the <sold phones
get the comparison between persons
>, you
(to get a feeling for this difference, think about what would
happen to phones that are typically evaluated in 2nd place)
Disadvantages of conjoint analysis
• All fictituous decisions (do you like / would you buy /
how would you evaluate ...)
• (Often) assumes weighted average. This does not
allow "all or nothing" weights
• Can be quite complicated and/or boring for the
respondents
• Can be quite complicated for the researcher, both to
implement and to analyze
• ...
Kinds of Conjoint Analysis
1. CVA: Conjoint Value Analysis
Respondent evaluates single vignette at a
time
2. ACA: Adaptive Conjoint (Value)
Analysis
Adapting the cases you offer based on
previous answers of the respondent.
3. CBC: Choice Based Conjoint
Asking for preferences between 2 (or 3 or
4) cases.
4. PP-CBC: Partial-profile Choice Based
Conjoint
Comparing only part of the attributes of
the cases.
-
10 Euro p/m
2 year minimum
free phone
...
How do you rate this
proposal? (-5 ... +5)
Advanced Methods and Models in Behavioral Research
1. CVA: Conjoint Value Analysis
Respondent gets to evaluate one vignette at a time
Note
•
•
Researcher chooses the dimensions
The variance of the multiple regression estimator equals
 (X’X)-1
(with X the matrix of D’s)
•
•
This implies that choosing the subset of cases that you are
going to use, affects how broad your confidence intervals
will be
-> Experimental design literature: full-factorial design, Doptimal designs, etc
Advanced Methods and Models in Behavioral Research
2. ACA: Adaptive Conjoint Analysis
Adapting the cases you offer based on previous answers of the
respondent.
• You give, say, 20 cases to each respondent.
• Which cases the respondent gets, depends on his answers to the
first couple of cases.
 Much more efficient than just randomly choosing cases
But
impossible to do off-line or by phone, and even online quite
difficult to implement
Advanced Methods and Models in Behavioral Research
Software … is problematic
-
Rare (part of general survey software mostly)…
Expensive …
Visually not very nice…
No output of raw data …
Advanced Methods and Models in Behavioral Research
Sawtooth software / SKIM Research
www.sawtoothsoftware.com
www.skimgroup.com/software
Advanced Methods and Models in Behavioral Research
3. CBC: Choice Based Conjoint
Asking for preferences between 2 (or 3 or 4) cases.
-
10 Euro p/m
2 year minimum
free phone
internet = per Mb
-
15 Euro p/m
1 year minimum
phone costs 70
internet = per Mb
-
30 Euro p/m
2 year minimum
free phone
internet = free
Which of these three offers do you prefer?
Or …
Rank these three offers
Or …
Distribute 10 points over these 3 offers
Advanced Methods and Models in Behavioral Research
3. CBC: Choice Based Conjoint (2)
Resp chooses preferred
vignette
Issues:
• Which of these kinds
of questions works
best? We don’t know.
• If you have only
choice data (or
ordinal data), how
can you arrive at
values for the
different dimensions?
Y
D1
D2
D3
D4
D5
id
set
1
-1
-1
0
1
0
1
1
0
1
1
1
0
-1
1
1
0
0
0
1
0
-1
1
1
0
1
0
-1
1
0
1
2
1
…
…
…
…
…
1
2
0
…
…
…
…
…
1
2
1
…
…
…
…
…
1
3
0
…
…
…
…
…
1
3
…
…
…
…
…
…
…
3
Y = 0/1 => Logistic regression, but …
… that does not use all the available
information and if we use the data for
all persons we have dependencies in
the data (more cases per person) and …
what if we have ordinal data?
Advanced Methods and Models in Behavioral Research
Which method to choose when?
We have rules of thumb only …
www.sawtoothsoftware.com/products/advisor/
Advanced Methods and Models in Behavioral Research
Sample sizes in conjoint analysis
• This is difficult ... for many reasons
– Quite a lot of degrees of freedom (#people, vignettes pp,
dimensions per vignette, values per dimension, ...)
– Not clear up front how different answers from the same
person are going to be
• Rules of thumb (see paper online)
• Generally 150 – 2000 participants
• Comparing groups? 200+ participants per group
• Testing purposes not comparing groups: 300+
Advanced Methods and Models in Behavioral Research
Sample sizes in conjoint analysis (2)
For choice-based conjoint:
nta
> 500 c
With
n
t
a
= number of respondents
= number of tasks per person
= number of alternatives per task
c
= number of “analysis cells”
(main effects model: max number of levels in an
attrib.)
(with interactions: max number of interaction levels)
Advanced Methods and Models in Behavioral Research
Statistical issues
We need to know how we can deal with “nested
data” (for instance, more than one answer per
person)
Can't the
Sawtooth
people take
care of this?
Advanced Methods and Models in Behavioral Research
Marketing vs other social science
Marketing ("conjoint analysis")
Try to come up with the weights for each
dimension for each respondent
(e.g. to then segment the population)
Other social science ("vignette study")
Try to come up with the average weights for
groups of people
(so other aim, and you need less choices per
person)
To Do
• Read and understand the literature on the course
website on Conjoint Analysis.
Advanced Methods and Models in Behavioral Research