Decision/Risk Analysis Software:
Survey for Trade Studies
Dennis M. Buede
Department of Systems Engineering and
Center of Excellence in Command, Control, Communications, and Intelligence
George Mason University
Fairfax, VA 22030-4444
703-993-1727
dbuede@gmu.edu
Abstract. This paper provides an overview of the PC
and Mac-based software market for decision/risk
analyses.
Decision/risk analysis is a powerful
methodology for conducting systems engineering trade
studies and selecting system designs because the
methodology explicitly deals with conflicting
performance and cost objectives, uncertainty and risk
preference. Information on 28 software packages is
provided. Before describing these packages several
applications of decision/risk analysis in systems
engineering design are presented. In addition, technical
performance, usability, compatibility and output
documentation issues are discussed.
Finally, we
forecast the future trends for this software.
OVERVIEW OF THE DECISION/RISK
ANALYSIS SOFTWARE MARKET
The software market for decision and risk
analysis packages has been active since the early 1980’s
but has never been a major market niche like
spreadsheets. Software packages in this market address
problems in which there are multiple options from which
one is to be chosen; each option may be a sequence of
decisions made at one time or over some time interval.
The packages reviewed here address one or more of the
following issues: uncertainty, conflicting objectives,
and risk. One taxonomy of this market has the following
four categories: (1) multi-attribute utility and Analytical
Hierarchy Process
(AHP),
(2)
multi-criteria
optimization, (3) Outranking or multi-criteria partial
ordering, and (4) decision tree or influence diagram.
The multi-attribute utility (MAU) and AHP
packages enable the user to create a hierarchy of
objectives and criteria for evaluating the options via a
weighted score. Multi-criteria optimization packages
provide a framework for finding an optimal solution by
weighting or sorting multiple criteria. Multi-criteria
partial ordering packages refrain from identifying the
best option through weighted scores but accept limited
weighting information to separate dominated options
from those that should remain candidates for decision
given the information provided. Decision tree or
influence diagram packages enable the user to evaluate
risky decisions that incorporate both uncertainty and
conflicting objectives.
Finally the group process
packages allow a group of experts and decision makers
to provide inputs and see results as part of a same time,
same place group session.
Tables 1 through 4 summarize most of the
software products currently available on the market.
Previous software surveys can be found in Buede (94,
93, 92a and 92b) and Maxwell (96).
USING DECISION/RISK ANALYSIS SOFTWARE
First we examine a design decision that has
arisen during integration testing for a military helicopter.
The test results show that the gear box accumulates too
much oil when one of the two engines is lost. This oil
accumulation will cause the loss of the gearbox,
resulting in an increased probability of aircraft loss
during a vertical landing. The influence diagram in
Figure 1 shows an analysis of the redesign decision to
add a drain valve to the gear box or leave the aircraft as
it is. The drain valve addition to the aircraft has
negative implications for weight, cost and schedule. The
nodes that have been added on the right edge of the
figure contain weight values for trading off the four
objectives: aircraft and pilot loss, aircraft weight, cost,
and schedule.
Figure 2 depicts the objectives hierarchy for the
originating performance objectives for the U.S. Marine
Corps Mobile Protected Weapons System.
This
material was developed as part of multiattribute
Figure 1. Influence Diagram for Aircraft Redesign of the Gearbox
value analysis and provided as part of the Request for
Proposal in 1980 and used by contractors in the
preparation of their technical and cost proposals. See
Buede and Bresnick.
MAJOR ISSUES IN CHOOSING
THE RIGHT SOFTWARE PRODUCT
We have grouped the analysis-related issues
that could cause one to buy one or more of these
software packages into the following four categories:
(1) technical performance, (2) usability, (3)
compatibility with other software packages, and (4)
output of results for selection and documentation.
Technical performance issues include building
and editing models, eliciting the quantitative inputs,
having flexibility for the sequence and form of the
analysis, and the speed at which the computations are
performed. Most of the packages are quite good at
providing flexibility in building and editing models,
including providing graphical representations of the
analysis structure. Most packages still assume an analyst
exists to elicit the values and probabilities; Expert
Choice provides substantial flexibility in providing AHP
and non-AHP elicitation techniques. More and more
packages are providing substantial sensitivity analysis
capability; DATA continues to perform very well here.
Computation speed is really only critical for decision
tree and influence diagram applications; DPL maintains
its lead in this area.
Usability involves the ability of the user,
whether occasional or frequent, to learn the software
quickly and use it effectively. Software-related design
issues include compatibility of the user interface with
emerging PC and existing Mac standards; program
simplicity; clarity and compactness of the screen display
for data entry and results presentation; command and
control features provided to the user; and documentation
(both written and on-line) for help and reference.
Software designers are always balancing the conflicting
objectives of program simplicity and program power.
Generally, power winds out because the software
designers are analysts. Most of the software packages
need to focus more attention on their screen displays to
make the desired inputs and presented outputs clearer.
Pull-down menus with hot keys have become the
standard user interface on both the Mac and the PC.
Very few programs are providing the user with
substantial control over the screen display, e.g., changing
the scales on graphs, reordering the list of options or
outcomes (DPL), choosing which factors to plot against
each other (HIVIEW). The new standards certainly
provide the user with control of the cursor.
Documentation still needs lots of improvement; I seldom
can find answers to my questions either on-line or in
reference material.
MPWS Op'l Effectvns
Goal
Assault Support
Goal
Firepower
Goal
Accuracy
Goal
Moving-station,2km
Measure
Secondary systems
Measure
Station to station
Goal
Station-moving,2km
Measure
Lethality against
Goal
Helicopters
Measure
Light Armor at
Goal
Figure 2. Multiattribute Value
Application for the Mobile Protected
Weapons System
2 km
Measure
3 km
Measure
4 km
Measure
Materiel
Measure
Personnel
Measure
Tanks at
Goal
2 km.
Measure
3 km.
Measure
4 km.
Measure
Servicing Rate
Measure
Stowed Kills
Measure
Target Acquisition
Goal
Acquisition Range
Goal
Identification rang
Measure
Recognition range
Measure
Field of view
Measure
in bad weather
Measure
Helicopter Transpor
Measure
Mobility
Goal
Cross Country
Goal
% terrain not negot
Measure
Speed on best 80%
Measure
Cruising range
Measure
Road Speed
Measure
in Water
Measure
Rel/Avail/Maint(RAM
Measure
Survivability
Goal
Agility
Measure
Protection
Goal
in front
Measure
on bottom
Measure
on side and back
Measure
on top
Measure
Signature
Measure
Blocking Position
Goal
Subsequent Ops
Goal
Some of the less successful packages in the past
have provided an interview option for the user to
choose. The interview option, not unlike some of the tax
preparation software, takes the user through a guided
process of model creation and numerical elicitation.
Almost all of the software packages assume the user is
very knowledgeable of the underlying analytical process
in the software.
Compatibility with spread sheets is often
critical for decision/risk analyses because users capture
and store data in spread sheets. However at this point
very few packages provide sufficient flexibility for
going back and forth between the analysis package and
the spreadsheet. As a result there are several packages
(TreePlan, @Risk and Top Rank) that are spreadsheet
add-ins. Expert Choice is probably the most flexible
(non add-in) package in this regard.
The effective output of results for selection and
documentation remains a major shortcoming, especially
the documentation of the analysis.
The software
packages are becoming more sensitive to presenting the
information in meaningful ways for decision makers;
DPL certainly has made many strides since its
introduction in this regard by allowing the user great
flexibility in seeing small parts of the decision tree or
large parts.
Using these selections can be time
consuming and awkward, however. HIVIEW continues
to lead here.
In summary, we have many choices in this low
volume market focusing on sophisticated decision/risk
analyses. The software packages range from relatively
simple to highly complex. There is more than one
decision algorithm available for guiding the decision
process; meaning of course that you may well get a
different recommendation depending on which package
you are using. So “buyer beware”!
FUTURE TRENDS
In the past five years substantial improvements
have been made in the decision/risk analysis software.
Part of these strides have come with the standardization
of user interfaces brought about by Windows. Similar
improvements can be expected as Windows 95 gains its
user base.
We can count on the vendors, who are analysts
at heart, to continue to improve and expand the analytical
capabilities of these packages. With these improvements
we should also continue to see modest improvement in
the display and documentation capabilities of the
software.
The improvements that will be least likely to
make substantial advances involve embedding the
analyst’s wisdom and knowledge in the software for the
users who are less skilled analysts.
These
improvements should include problem structuring and
elicitation features. I believe these capabilities may
exist ten years from now, but are not very likely in the
next five years. This software market will always be a
small, specialty market until the less skilled analysts are
better supported.
REFERENCES
Buede, D.M. "Decision Analysis Software: Aiding
Insight II", OR/MS Today, June 1994.
Buede, D.M. "Decision Analysis Software: Aiding
Insight", OR/MS Today, April 1993.
Buede, D.M. "Software Review - Three Packages for
AHP: Criterium, Expert Choice, and HIPRE 3+",
Journal of Multicriteria Decision Analysis,
October 1992, Vol. 1, pp. 119-121.
Buede, D.M. "Software Review - Overview of MCDA
Software Market", Journal of Multicriteria
Decision Analysis, July 1992, Vol. 1, pp. 59-61.
Buede, D.M. "Superior Design Features of Decision
Analytic Software", Computers and Operations
Research, January 1992, Vol. 19, pp. 43-58.
Buede, D.M. and Bresnick, T.A. “Applications of
Decision Analysis to the Military Systems
Acquisition Process”, Interfaces, 1992, Vol. 22,
No. 6, pp. 110-125.
Maxwell, D.T.
“Three Packages for Processing
Influence Diagrams: DPL, InDia and Demos”,
Journal of Multicriteria Decision Analysis,
March 1996, Vol. 5, pp. 72-74.
Table 1. MAU and AHP Software Packages
Software Name
Criterium
http://www.halcyon.com/
infoharv/infoharv.htm
DecideRight
Mac or
PC/Windows
or PC/DOS
PC/Windows
Price per
single
license
$495
PC/Windows &
Windows 95
$149
PC/Windows
$495
PC/DOS
$590
PC/DOS
$295
PC/Windows
600
British
Pounds
PC/Windows
& Windows 95
$395
PC/DOS
$100
PC/Windows &
Windows 95
$395
http://www.avantos.com
Expert Choice
orvis@pgh.nauticom.net
EXPERT 87 with Consensus
Builder
pjhmagic@aol.com
HIPRE 3+
hipre@sms_usa.com
HIVIEW
100333.337@
compuserve.com
Logical Decisions
gary.smith@uchsc.edu
POLICY PC
sschuman@
cnsvax.albany.edu
V.I.S.A for Windows
http://www.vti.co.uk/vti/
Vendor
Info Harvest, Inc.
8238 15th Ave., NE
Seattle, WA 98115
800 869 7150
Avantos
5900 Hollis St., Suite A
Emeryville, CA 94608
510-654-4600
Expert Choice, Inc.
4922 Ellsworth Ave.
Pittsburgh, PA 15213
412-682-3844
Magic7 Software Co.
101 First St., Suite 237
Los Altos, CA 94022
415-941-2616
Santa Monica Software
30033 Harvester Rd.
Malibu, CA 90265
310-451-2382
KRYSALIS Ltd.
28 Derwent Dr.
Maidenhead SL6 6LB
England
(44) 1628 36861
Logical Decisions
1014 Wood Lily Dr.
Golden, CO 80401
800-35-LOGIC
Executive Decision Services, Inc.
P.O. Box 9102
Albany, NY 12209
518-465-8872
Visual Thinking International, Ltd.
141 St. James Rd.
Glasgow G4 0LT
Scotland UK
(44) 141 552 6888
Table 2. Multi-criteria Optimization Software Packages
Software Name
AIM
Mac or
PC/Windows
or PC/DOS
PC/DOS
Price per
single license
PC/DOS
$950
MAC
$499,
$99
PC/DOS
$490 & $470,
respectively
$10
vahid@umich.edu
RADIAL
bxm4@po.cwru.edu
Triple C
VIG & VIMDA
numplan@pp.columbus.fe
Vendor
Professor Vahid Lotfi
School of Management
The University of Michigan-Flint
Flint, MI 48502-3687
810-762-3266
Professor Benham Malakooti
Dept. of Systems Engineering
Case Western Reserve Univ.
Cleveland, OH 44106
216-368-4462
Mr. Fadi Micaelian
Auguri Corp.
650 Bear Island
Redwood City, CA 94063
415-593-3234
Numplan
P.O. Box 128
SF-03101 Nummela
Finland
(358) 0 2271 900
Table 3. Outranking or Multi-Criteria Partial Ordering Software Packages
Software Name
Mac or
PC/Windows
or PC/DOS
PC/Windows
Price per
single
license
7500
French
Francs
GAIA & PROMETHEE
PC/DOS
?
IDEAS
PC/DOS
$50
ELECTRE III & IV
vallee@
lamsade.dauphine.fr
rudolf.vetschera@
uni-konstanz.de
Vendor
LAMSADE
Univ. of Paris-Dauphine
Place du Marechal Del Lattre de Tassigny
75795 Paris Cedex 16
France
(33) 1 44 05 44 72
Professor J. P. Brans
Vrije Universiteit Brussel
Pleinlaan 2
Brussels, Belgium
32 2 641 20 70
Professor Rudolf Vetschera
Faculty of Economics & Statistics
University of Konstanz
POB 5560, D130
Konstanz D-78434
Germany
(49) 7531 88 2599
Table 4. Decision Tree or Influence Diagram Software Packages
Software Name
DATA
Mac or
PC/Windows
or PC/DOS
Mac &
PC/Windows
Price per
single
license
$379
PC Windows
$295
Mac
$695
PC/Windows
$995/495
PC/DOS
$395
Mac &
PC/Windows
$1500
PC/Windows
& Windows
95 & MAC
with EXCEL
Mac or
PC/Windows
$29
http://www.treeage.com
Decide
71052.352@compuserve.com
Demos
http://lumina.com
DPL
dpldept@adainc.com
InDia
Supertree/Sensitivity
LHunter@sdgnet.com
TreePlan for Excel
71330.3445@
compuserve.com
@RISK & Top Rank
http://www.palisade.com
$395 each
Vendor
TreeAge Software, Inc.
1075 Main Street
Williamstown, MA 01267-2637
800-254-1911
Software Inventions, Inc.
7400 Hillside Way
Anchorage, AK 99516
907-345-6347
Lumina Decision Systems
4984 El Camino Real, Suite 105
Los Altos, CA 94002
415-254-0189
Applied Decision Analysis
2710 Sand Hill Rd.
Menlo Park, CA 94025
415-926-9251
Decision Focus, Inc.
650 Castro St. Suite 300
Mountain View, CA 94941-2055
800-327-0325
Strategic Decisions Group
2440 Sand Hill Rd.
Menlo Park, CA 94025
415-233-6282
Decision Support Services
2105 Buchanan St., #1
San Francisco, CA 94115
415-673-6217
Palisade Corporation
31 Decker Rd.
Newfield, NY 14867
800-432-7475