The Delphi Method and
Marketplaces
(original 1975 reference book about Delphi on my
website)
Murray Turoff turoff@njit.edu
http://is.njit.edu/turoff
1
Definition from 1975


Design of a group communication process
structured/tailored around the nature of the
application and the nature of the group
 Original paper and pencil rounds
 Anyone can change their view
 Anonymity or pennames
 Scaling theory to promote understanding
 Voting to focus discussion
 Select “knowledgeable” people
 A round took a month – three to five rounds
 Respondents 15 to 500
Prediction, policy analysis, conditional forecasts,
planning, significance of contributions, new product
characteristics, etc, etc. (book has many examples)
© 2004 Turoff
2
Online Delphi’s Today





Dynamic: entering main voting items, voting, making vote
changes, and pro, con, neutral discussion comments, can
all go on at once.
 Each individual can focus on what they want to
 System notifies user of changes
 Termination of vote changes is a sign of finalization of
the results
 People can propose rewording (Roberts Rules of Order)
What took three months can happen in three weeks for 20200 people (30 users, 80 main items voted, 450 discussion
items, 25 modifications)
Voting really serves to focus discussion on differences of
view
Three recent thesis experiments and field trials in the
Information Systems Ph.D. program at NJIT
Used as a collaborative learning tool in online class
discussions
System designed as a Social Decision Support System
© 2004 Turoff
3
Turoff, M., Hiltz, S. R., Li, Z., Wang, Y., Cho, H., Yao, X.,
Online Collaborative Learning Enhancement through the
Delphi Method, Proceedings for the OZCHI 2004
Conference, November 22-24, University of Wollongong,
Australia




Summarizes three recent thesis efforts
(message me for copy of above)
One experiment that shows structured
communications much more effective in
generating both more and unique ideas for
both small and medium groups
Dynamic large discussions on such things as
“Preventive Security Measures” and “most
important things learned in a course”
Software written in Coldfusion for SQL server
© 2004 Turoff
4
Observations and Implications I

Two original Delphi philosophies
 A numeric result for a group view is most
efficient output (most early prediction
studies)
 No real decision person wants a number
that does not also provide the collective
group cognitive model that generated it
(conditional forecasts, decision support,
and policy analysis)


Exposing such forecasting models has been
integrated into many modern and corporate
Delphi’s
However, collaborative general model building is
still a challenge for design
© 2004 Turoff
5
Observations and Implications II





People use and seek information to make investments
Adam Smith pointed out that a good free enterprise
requires buyer and seller have access to relevant
information
Useful marketplaces must incorporate the evaluation
of information
Let market members scale and/or vote on relative
reliability/confidence/importance of a piece of
information as in Delphi but posed as an information
recommender system with anonymity
How about this community having a marketplace
about research hypothesis on the design and
performance of marketplaces
 A seer upon perceiving a flood should be the first
to climb a tree – Kahlil Gibran
© 2004 Turoff
6