Subjectivity in Decision Analysis
David L. Olson
James & H.K. Stuart Professor in MIS
University of Nebraska Lincoln
Human Centered Processes - 2008
Rational Choice Theory
K. Morrell, Journal of Business Ethics [2004]
• Dominant model for business decision
making
• Compared with Image Theory
– Utilitarian ethics
• both consistent
– Kantian ethics
• Image theory yes, RCT no
– Virtue-based ethics
• Image theory yes, RCT no
Human Centered Processes - 2008
Ideal
• Objective measures
Max Weber
• Accurate preference input
• “Rational” decision maker
Accounting: Jensen - Agency Theory
Economics: Williamson - Transaction Cost
Analysis
Human Centered Processes - 2008
Basic Preference Model
K
Valuej   wi  sij
i 1
i  criterion index
j  alternativ e index
K  number of criteria
wi  weight of criterion i
sij  score of alternativ e j on criterion i
Can use multiplicative model for interactions
Human Centered Processes - 2008
Objective Measures
• Objective preferred
– can measure
• past profit, after tax
• Subjective
– know conceptually, but can’t accurately
measure
• response to advertising
Human Centered Processes - 2008
Problems with Objective
Approach
von Neumann & Morgenstern [1944] Theory of Games– utility is
measurable
Georgescu-Roegen [1954] The Quarterly Journal of Economics – requires
to many assumptions of rationality
Lindblom [1965] Public Administration Review– muddling through
Morgenstern [1972] Journal of Economic Literature – 13 critical points
• uncertainty
• ambiguity
• disagreement in groups
Human Centered Processes - 2008
EMPIRICAL EVIDENCE
contrary to rational choice models
Braybrooke & Lindblom [1969]; Simon [1985] Payne, et al. [1993]
• Some problems never reach decision maker
• decision makers often have simple maps of real problems
• all alternatives not known, so decision makers do not have full,
relevant information
• individual altruism
Tversky [1969]
• systematic & predictable economic intransitivities
Kahneman, Slovic & Tversky [1982]
• people use heuristics rather than follow rational model
Human Centered Processes - 2008
James G. March
Bell Journal of Economics [1978]
• Rational choice involves guesses:
– About future consequences of current actions
– About future preferences of those consequences
Administrative Science Quarterly [1996]
• Alternatives & their consequences aren’t given, but need to
be discovered & estimated
• Bases of action aren’t reality, but perceptions of reality
• Supplemental exchange theories emphasize the role of
institutions in defining terms of rationality
Human Centered Processes - 2008
Overview
• Inputs to preference models involve subjectivity
– Weights are function of individual
– Scores also valued from perspective of individual
• Subjective assessment MAY be more accurate
• Purpose of analysis should be to design better
alternatives
Human Centered Processes - 2008
Means to Cope
Payne, Bettman & Johnson [1993]
• strategy will differ by number of alternatives
– few - focus on all relevant information
– many - noncompensatory simplifying heuristics
• $/lives tradeoff varies by context
• Hogarth: many find explicit tradeoffs uncomfortable
• PROSPECT THEORY: initial analysis simple, weed out;
for selected alternatives, more detailed
• As people learn more about problem structure, construct
choice strategies
Human Centered Processes - 2008
Objective/Subjective
• OBJECTIVE: what is convenient to model
– ideal - eliminate bias, arbitrary judgment
– extreme: cost/benefit analysis spanning years of
measuring the unmeasurable
• SUBJECTIVE: what people do to cope
– value is subjective after all anyway
– value is what MAUT, MCDA seeks to measure
Human Centered Processes - 2008
ACCURATE PREFERENCE
INPUT
• incomplete information
• uncertain measures
• uncertain preferences
• group participation
• risk
• time pressure: Edwards - how can you calculate
expected utility in available time?
• change
competition
complexity
Human Centered Processes - 2008
RELATIONSHIP TO MCDA
• We shouldn’t expect so much theoretical
purity
– the world has shifted away from appreciation of
numerical analysis
• Just because assumptions are not met
doesn’t mean pure approach better
Human Centered Processes - 2008
MCDA Methods
• Multiattribute utility theory
• Analytic hierarchy process
• Outranking
– ELECTRE, PROMETHEE
• Fuzzy, DEA, Verbal Decision Analysis
• Image Theory
Human Centered Processes - 2008
Spectrum
MAUT with strictly objective measures
MAUT with constructed measures
Likert scales
SMART - swing weighting rather than lottery
tradeoffs
AHP - ratios of subjective scale
Human Centered Processes - 2008
PROMETHEE Spectrum
Class I:
Class II:
Class III:
Class IV:
Class V:
ordinal
step advantage
proportional advantage (in range)
three step
proportional with indifference
range
Class VI: normal distribution
Human Centered Processes - 2008
MAUT Hierarchy
Overall
Cost
Lives lost
Risk
Civic improvement
$billions
Expected value
Probability
of major
catastrophe
Families with upgraded
housing
Human Centered Processes - 2008
Objective Measures
Cost
Lives Lost
Risk
Civic
improvement
Nome AK
39.548 billion
61
0.0165
312 upgrades
Newark NJ
98.467 billion
143
0.0002
68,472 upgrades
Rock Springs
WY
58.930 billion
41
0.0036
4,138 upgrades
Duquesne PA
60.156 billion
39
0.0069
20,653 upgrades
Gary IN
69.693 billion
86
0.0027
56,847 upgrades
Human Centered Processes - 2008
Swing Weighting
Attribute
Lives lost
On BEST
On
WORST
W on Best
W on
Worst
Final
100
290
0.541
0.569
0.556
Risk
60
175
0.324
0.343
0.333
Cost
20
35
0.108
0.069
0.089
Civic
5
10
0.027
0.020
0.022
185
510
Human Centered Processes - 2008
SMART with swing weighting
Cost
Weights 0.089
Lives lost
Risk
Civic imp
0.556
0.333
0.022
RESULT
Nome AK
0.991
0.564
0.175
0.003
0.468 (4)
Newark NJ
0.026
0.007
0.990
0.685
0.351 (5)
Rock
Springs
WY
0.685
0.707
0.820
0.041
0.736 (1)
Duquesne
PA
0.664
0.721
0.655
0.207
0.675 (2)
Gary IN
0.505
0.386
0.865
0.568
0.552 (3)
Human Centered Processes - 2008
Logical Decision
• Hierarchy of criteria
• Single-attribute Utility Functions
– Worst imaginable utility = 0
– Best imaginable utility = 1
– Assess 0.5 level of either value or utility
• Tradeoffs
– Pairwise comparisons
– Select preferred extreme
– Improve other until equal
Human Centered Processes - 2008
SUF for Lives Lost
PROBLEM: sensitive to limits set – may warp values more than intended
Human Centered Processes - 2008
Weight Tradeoffs
PROBLEM: weights reflect both choices, scale – again hard to control
• Cost < Lives Lost
[0,1000]>[500,0]
[0,1000]=[5,1000]
• Cost < Risk
[0,1]>[500,0] [0,0.35]=[500,0]
• Cost > Civic Improvement
[0,100000]>[500,0]
[0,100000]=[350,0]
• Weights (including scale):
–
–
–
–
Risk
5.029
Cost
1.577
Lives Lost
29.337
Civic Improvement64.058
Tradeoff: Cost vs. Civic
Improvement
Human Centered Processes - 2008
Result
Human Centered Processes - 2008
Contributions by Criteria
Human Centered Processes - 2008
Rock Springs vs. Newark
Human Centered Processes - 2008
Subjective Ratings
Cost
Lives Lost
Risk
Civic improvement
Nome AK
Moderate
Low
Very high
Low
Newark NJ
Very high
Very high
Very low
Very high
Rock Springs
WY
High
Very low
Low
High
Duquesne PA
High
Very low
Medium
Medium
Gary IN
Higher
High
Low
Very high
Human Centered Processes - 2008
Subjective SMART
Cost
weights 0.089
Lives Lost
Risk
Civic Imp
0.556
0.333
0.022
RESULT
Nome AK
0.8
0.7
0
0.2
0.465 (3)
Newark NJ
0
0
1
1
0.355 (5)
Rock
Springs
WY
0.4
0.9
0.8
0.8
0.820 (1)
Duquesne
PA
0.4
0.9
0.5
0.6
0.716 (2)
Gary IN
0.2
0.2
0.8
1
0.417 (4)
Human Centered Processes - 2008
Output Comparisons
MAUT
SMART
SMART
subjective
Nome AK
0.629 (5)
0.468 (4)
0.465 (3)
Newark NJ
0.794 (3)
0.351 (5)
0.355 (5)
Rock Springs WY
0.723 (4)
0.736 (1)
0.820 (1)
Duquesne PA
0.839 (1)
0.675 (2)
0.716 (2)
Gary IN
0.830 (2)
0.552 (3)
0.417 (4)
Human Centered Processes - 2008
Comparison with SMART
• Simpler allows decision maker to see exactly what
ratings are
• MAUT
–
–
–
–
Distrusts human – masks tradeoffs in effort to make “objective”
“Objective” here means have no idea
Theoretically, preferences will be identical
Does allow for nonlinear interaction, but severe impact
• My contention:
– DIRECT IS BETTER THAN MACHINE
Human Centered Processes - 2008
Image Theory
INTEREST GROUP
Criteria reflecting concerns
Government
Cost, lives lost, risk, civic improvement
Nuclear Industry
Permanent storage of nuclear waste
Nuclear power demand
Local Citizens
Equity
Cultural artifacts
Employment
General Population
Nuclear power generation safety
Transportation risk
Low-cost electricity
Human Centered Processes - 2008
Image Theory process
• Frame decision
– Desired states
– Actions needed to attain desired states
– MORE CRITERIA
• Helpful to MCDA in structuring
– Context
• Elicit participation of as many views as possible
– Identify alternatives
• Design an ideal rather than settle for existing
• MORE ALTERNATIVES
Human Centered Processes - 2008
Verbal Decision Analysis
• Controlled pairwise comparisons of
tradeoffs
• Focus on critical criteria
– Don’t use falsely precise measures
• Fuzzify – categorical ratings
• Screen alternatives
– Preemptive
• Focus on critical tradeoffs
Human Centered Processes - 2008
VDA Process
• Eliminate very high lives lost
– Newark eliminated
• Eliminated risk high or worse
– Nome eliminated
• Rock Springs now dominates Duquesne
• FOCUS ON
– Rock Springs
– Gary
• TRADEOFFS
– Rock Springs – a little lower cost, improved lives lost
– Gary – civic improvement slightly better
Human Centered Processes - 2008
Inferences
• Objective can’t capture all the complexity of real
decisions
– OBJECTIVES ARE ALWAYS LEFT OUT
– Conventional wisdom – at most 7 matter
– BUT THERE IS NO PARETO OPTIMAL unless all considered
• When Groups are involved, THERE IS NO ONE
BEST DECISION
– Ward Edwards – never saw a group pick an option that was first
choice of one subgroup
– NEGOTIATION
Human Centered Processes - 2008
Conclusion
• Measures of alternative future performance,
preference for that performance both
subjective
• Objective measures not always better
• Focus should be on:
– Learning (changing preference)
– Design of better alternatives (Image Theory)