Information & Management 41 (2004) 399–411
Integrating nominal group technique and joint application
development for improved systems
requirements determination
Evan W. Duggana,*, Cherian S. Thachenkaryb
a
Information Systems, Statistics, and Management Science, Culverhouse College of Business Administration,
University of Alabama, 367 Alston Hall, Tuscaloosa, AL 35487-0226, USA
b
Department of Decision Sciences, College of Business Administration, Georgia State University,
University Plaza, Atlanta, GA 30303-3083, USA
Received 24 January 2002; received in revised form 29 January 2003; accepted 27 June 2003
Abstract
Joint application development (JAD) is a facilitated group technique that can be used in systems requirements determination
(SRD); it was designed to encourage team rapport and achieve synergy by leveraging the combined knowledge of participants.
JAD has been reported to cure several problems of conventional SRD techniques and shortened development schedules.
However, its freely interacting meeting structure may curtail effectiveness by encouraging adverse group-related actions that
challenge even the best facilitators. In this study, we integrated JAD and nominal group technique (NGT), a popular group
structure that has been used to reduce the effects of negative group dynamics on task-oriented objectives. We examined this
integrated structure in a laboratory experiment to determine whether it could alleviate the problems that JAD has experienced
during SRD. The results suggest that the integrated approach outperformed JAD in our test environment; it was as efficient as
JAD alone and it appeared to contribute to the reduction of the need for great facilitation skills in group decision-making.
# 2003 Elsevier B.V. All rights reserved.
Keywords: Communication medium; Group support methodology; Human factors; Information systems development; JAD; Meeting
facilitation; Nominal group technique; Systems requirements determination
1. Introduction
While many organizations have developed successful information systems (ISs), several have expended
large amounts on software products that failed to
deliver expected benefits. For many successfully deployed systems, maintenance activities have consumed
*
Corresponding author. Tel.: þ1-205-348-7688;
fax: þ1-205-348-0560.
E-mail address: eduggan@cba.ua.edu (E.W. Duggan).
a disproportionate share of systems development
resources. A large part of this problem can be traced
to miscommunication about required systems features
and misunderstanding among system users and builders
during the systems requirements determination (SRD)
[4,39]. For this reason, SRD has been acknowledged as
a critical determinant of system development success or
failure [16,22,55]. Yet low quality systems requirements are common [17,30,37,46].
The facilitated group process called joint application development (JAD) was instituted, primarily, as
0378-7206/$ – see front matter # 2003 Elsevier B.V. All rights reserved.
doi:10.1016/S0378-7206(03)00080-6
400
E.W. Duggan, C.S. Thachenkary / Information & Management 41 (2004) 399–411
an alternative to the conventional interviewing technique for determining systems requirements [29]. A
successful JAD process may reduce the communication barriers to effective requirements elicitation and
analysis and eventually help to improve the quality of
the final system [20]. However, JAD groups have
experienced problems usually associated with the
freely interacting meeting structure [24]. In this
arrangement, communication takes place between
group members with minimal control being applied
by the meeting structure [74]; hence social and emotional dynamics often obstruct the process [43].
The nominal group technique (NGT), in which
individuals work in the presence of others without
significant interaction [10], has been used to cure
many of the problems that freely interacting groups
encounter; they include ineffective idea generation,
unequal participation in discussions, groupthink, domination by more powerful participants, and ineffective
conflict resolution [11]. In our study, we examined the
effects of integrating NGT and JAD (during the JAD
workshop) on the communication problems that occur
during user–developer interactions in SRD.
We contend that, while JAD has contributed to an
improvement in the quality of requirements generated
by the conventional (interviewing) technique, its
intended benefits are often reduced by the problems
faced by freely interacting groups [50]. An integration
of JAD with NGT (NJAD) is expected to improve the
generation of requirements as well as consensus and
agreement on them and restore the group process loss
with JAD alone. Our study is an extension of previous
research that was geared toward improving JAD’s
performance. However, unlike some of those studies,
it does not require the acquisition of group support
systems (GSSs) technology.
2. Background, research model and hypotheses
2.1. JAD—strengths and weaknesses
JAD is superior to the interviewing technique and
takes significantly less time [7]. During a JAD session a trained facilitator helps system developers and
users pool their knowledge to establish system
requirements. The facilitator stimulates effective
user–developer interactions to generate ideas about
system features and helps to speed up decisionmaking [23].
JAD and its many derivatives have become increasingly popular in systems development and other organizational, decision-making contexts [5,41]. It is
considered best practice for fostering user commitment [72]. JAD is often used with rapid application
development (RAD) [67], an iterative and incremental
approach for accelerating ISs delivery, and the
dynamic systems development method (DSDM), the
RAD-based technique used extensively in the UK
[9,12]. It is considered a risk-reducing investment in
ISs development [48].
JAD, however, relies heavily on facilitation to guide
the meeting toward the attainment of its objectives [1].
Effective JAD facilitation is essential to encourage
group productivity, resolve conflict, and minimize
problems that are usually associated with freely interacting groups [73]. Kettelhut [44] identified several
potential group problems that have affected JAD sessions. These overlap with similar difficulties that
Nunamaker et al. [60] claimed contributed to process
loss, resulting in the failure of groups to realize their
potential.
The JAD literature reminds facilitators of the
importance of ‘‘chilling the dominator,’’ minimizing
disruptive group behavior, and resolving conflict [6]. It
contains several prescriptions for encouraging constructive interactions that are necessary for effective
outcomes [68]. These suggestions typically do not
include recommendations for how they should be
accommodated in JAD; the facilitator is expected to
employ a variety of techniques to implement them.
2.2. Incorporating NGT
The NGT originated by Delbecq et al. [27] for
conducting potentially problematic group meetings,
embodies a set of procedures for conducting group
sessions. It is a popular technique and one of the most
successful processes for structuring group meetings
[31]. NGT consists of the following five steps [53]:
1. Participants independently and silently generate a
list of ideas.
2. The facilitator records one idea at a time from
group members in a round-robin format until all
participants have completed their list.
E.W. Duggan, C.S. Thachenkary / Information & Management 41 (2004) 399–411
3. Participants discuss each idea for clarification
only, without critical evaluation or lobbying.
4. Participants independently rate and rank the ideas.
5. The group decides the priority ordering of the
alternatives based on voting and mathematical
pooling of the individual rankings.
The partitioning of activities into creative thinking
and idea-generation, evaluation, and decision-making
prevents negative group dynamics from inhibiting the
free flow of information, views, and opinions; this
contributes to more and better ideas [28,56]. These
benefits help to produce solutions that reflect the
combined judgment of the group and may lead to
synergy (process gain) [36].
Because of its communication structure, NGT
requires less facilitator intervention than JAD [78].
Poole and DeSanctis’ [63] adaptive structuration
theory (AST) claims that meeting outcomes reflect
both the impact of the structuring mechanisms and
the manner in which the structures are used as
intended (or modified). The role of facilitation is
primarily to inspire faithful adoption of the meeting
structure [14].
Although Lederer [49] proposed the use of NGT
alone for determining systems requirements, it may
not be entirely applicable to all aspects of SRD.
Phase I
Phase II
Phase III
401
JAD involves five stages; only the workshop is
affected by NGT intervention. Furthermore, many
desirable features of JAD [18] should be preserved. Our concept, therefore, does not suggest
the replacement of JAD by NGT but their integration during the JAD workshop to influence SRD
process effectiveness.
Fig. 1 illustrates how NGT may be interspersed into
the JAD workshop at appropriate intervals to stimulate
ideas about major systems components, identify the
decisions that these components support and the
sources of information needed to make those decisions. It may also be used to bring about consensus on
competing alternatives, and to categorize and prioritize system features. Conventional JAD approaches
are still required for activities such as discussing
assumptions, agreeing on guidelines and ground rules,
accommodating knowledge sharing by domain
experts, constructing models, and rationalizing and
synthesizing the ideas generated.
NGT is easy-to-use and is easily adaptable to the
problem domain [21]. However, facilitators must
develop a sense for the appropriate timing of each
NGT intervention. For example, it does not enhance
group process goals such as building team rapport
[66], nor is it particularly effective for synthesizing ideas [15,52]. Similarly, some facilitators may
Phase IV:
The Workshop
Phase V
Open Interactions…
P
R
O
J
E
C
T
D
E
F
I
N
I
T
I
O
N
R
E
S
E
A
R
C
H
P
R
E
P
A
R
A
T
I
O
N
F W
O O
R R
K
T S
H H
E O
P
NGT
Open Interactions…
NGT
Open Interactions…
…..
…..
Fig. 1. JAD/NGT integration.
F
I
N
A
L
D
O
C
U
M
E
N
T
402
E.W. Duggan, C.S. Thachenkary / Information & Management 41 (2004) 399–411
misapply the participation objective of NGT and aim
for equal participation instead of participation commensurate with knowledge and expertise.
Group
Task
2.3. Summary of past research
Purvis and Sambamurthy [64] confirmed that users
perceived that JAD contributed to superior, designer–
user interactions in comparison to traditional techniques. Other studies that have investigated the effects
of supporting JAD with electronic meeting systems
during SRD [19,51] have been largely inconclusive.
Davidson [25] found that organizational adaptations
and inappropriate uses of JAD have curtailed its
effectiveness, while Dean et al. [26] concluded that
JAD’s (single-user) methods restrict participation and
productivity.
Researchers have validated the claim of NGTs
effectiveness in a variety of decision-making environments. These include claims of its superiority over
freely interacting techniques used for problem finding
and problem solving [75]; the confirmation of its
effectiveness with heterogeneous groups, and for solving complex problems [69]; and its usefulness in
generating diverse ideas for the solution of multidimensional problems [33]. Gresham reported [47]
that participants were highly satisfied with NGT process, and it has been used successfully in combination
with other techniques [70,71,79].
In the largest sample size used in a group study to
date, Watson et al. [76] investigated structured support
for group discussions under conditions of no support,
pencil and paper support similar to that provided by
NGT, and with a group decision support system
(GDSS). They found that while GDSS support produced superior results when compared to no support,
there was no significant difference in the results
obtained from groups using a GDSS and those using
pencil and paper support.
2.4. The research model
In our study, we have borrowed extensively from
frameworks used in group studies—particularly from
GSS, which can be viewed as an alternative meeting
arrangement. We agree that group decision-making is
similar to the transformation that occurs in work
systems, which involves input, process, and output
Process
Outcome
Context
Communication
Structure
Fig. 2. Transformation model for group interaction.
[42]. The model adapted form Nunamaker et al. [59]
for testing group decision effectiveness (Fig. 2)
denotes this transformation. We also borrowed from
two other frameworks [61,62] used to test structured
support for group deliberations. The former specifically addressed software requirements development
teams.
The model indicates that when groups interact,
several input factors (for example, group attributes,
the nature of the task, contextual characteristics, and
the adopted communication structure) act simultaneously on the deliberation process. These interactions determine the nature of the process conditions
achieved. Inputs may either contribute to effective
group interactions (in which process gain occurs) or
poor deliberations (which result in process loss); the
outcome of the process experience is the aggregate of
process gains and losses.
This relationship is represented in Huber’s equation
for the effectiveness of decision groups [40]:
Actual effectiveness ¼ potential effectiveness
þ process gain process loss
(1)
Process gain results from effective communication, the
ability of group members to be stimulated by the ideas
of others to create new and useful ideas themselves,
and group learning, which may result in synergy.
Process loss is caused by non-task factors that reduce
decision-making capability; these include excessive
socialization, free riding, groupthink, conformance
pressure, domination, and conflict avoidance.
The research model (Fig. 3) incorporates the following input (independent) variables that affect group
E.W. Duggan, C.S. Thachenkary / Information & Management 41 (2004) 399–411
403
Input
Group Factors
Dynamics
Size
Effort
Process
Output
Task
Nature
Complexity
Facilitation
Skill Level
Control of Structure
Interaction
- Effectiveness
- Efficiency
- User
Satisfaction
Outcome
- Quality
- User
Satisfaction
Structure
JAD
NGT
Fig. 3. The research model.
process and the meeting outcome. Group communication structure and facilitation were manipulated during
the study and the others controlled:
Group communication structure (JAD or NJAD)—
the procedure that governs the pattern of interaction—is the major mechanism under review. It is the
vehicle through which the other input factors are
focused on the meeting objectives.
Group facilitation is also an important contextual
characteristic. The facilitator (either through competence or assistance permitted by the meeting
protocol) is the guardian of faithful adoption of
the communication structure. Group interaction and
decision quality are improved when facilitators
successfully use meeting procedure as intended.
Other group characteristics (dynamics, composition, and size) also bear directly on the process.
Similarly, the level of effort the group devotes to the
task, the extent of the group’s engagement
(reflected in the degree of socialization and free
riding), and its knowledge and experience are
important characteristics that may determine process loss or gain.
The structure and complexity of the task are also
influential factors.
The process and outcome variables of interest are
also indicated. Process effectiveness is a well-used
gauge of successful group interaction. It is reflected in
the quality of communication, idea generation, conflict resolution, and consensus building that the group
achieves. Efficiency is also an indicator of process
‘‘goodness.’’ It is apparent in the rate of flow, quality,
and relevance of the ideas generated during the meeting. The effects of group dynamics, facilitator behavior, and the structuring mechanism for the meeting
influence process efficiency.
The effectiveness of the process helps to determine
the quality of the outcome and participants’ satisfaction with it. Group sessions may be considered successful when the participants develop consensus about
the outcome of interest and commitment to, and
support for, the agreement (i.e., they are satisfied with
it). The outcome criteria also include the decision
quality—the extent to which the meeting produces
complete, correct, and feasible decisions.
2.5. Research hypotheses
Six hypotheses were constructed to evaluate
the relationships of our research model. The first
five expressed the expectation that NJAD would
404
E.W. Duggan, C.S. Thachenkary / Information & Management 41 (2004) 399–411
Table 1
List of hypotheses
Hypothesis
Hypothesis
Hypothesis
Hypothesis
Hypothesis
Hypothesis
1
2
3
4
5
6
Groups using NJAD will experience greater process effectiveness than those using JAD alone
Groups using NJAD will experience the same degree of process efficiency as those using JAD alone
Groups using NJAD will experience a higher level of satisfaction with the process than those using JAD
Groups using NJAD will identify more with the decision than those using JAD
Groups using NJAD will produce higher quality systems requirements than those using JAD alone
Novice facilitators will experience greater improvement in process effectiveness as they move from using JAD to NJAD
outperform JAD in creating more desirable process
conditions, which would help to generate higher quality requirements and greater satisfaction with the
outcome. The sixth related to the impact of the communication structure on facilitation; whether NJAD
would reduce JAD’s critical dependence on facilitation excellence (Table 1).
The effectiveness of JAD or NJAD sessions may
be reflected in the extent to which the input factors
generate favorable process conditions for effective
communication. Increased effectiveness results from
a higher level of participation and conflict resolution
and a reduction in the degree of destructive dominance. NJAD was expected to outperform JAD in
creating these conditions by channeling group
efforts more toward the attainment of its mission
(Hypothesis 1).
Process efficiency, which is one of JAD’s strengths
[45], describes the extent to which the input factors
permit the productive use of available meeting time. In
comparison to freely interacting techniques, NJAD
could increase process time because of the increased
rigor of the NGT procedure [58], but it was expected to
provide greater process effectiveness. We predicted
that the additional NJAD overhead would be balanced
by the gain in process effectiveness (Hypothesis 2).
Group behavior that contributes to process loss is
sometimes a source of disharmony [38]. On the other
hand, there is a direct relationship between effective
interactions and participants’ affinity with the process
[13]. We proposed that NJAD, which promotes more
effective interactions, would also contribute to a
higher level of satisfaction than JAD (Hypothesis
3), and to a greater level of satisfaction with the final
decision about the requirements and the participants’
identification with them (Hypothesis 4).
The quality of systems requirements is defined as
the degree to which they identify and incorporate
desirable features for overall system success.
High-quality specifications are internally consistent,
accurate, complete, unambiguous, relevant, and feasible [32,65,77]. NJAD was expected to contribute to
more effective idea-generation and decision-making
than JAD, and this would result in higher quality
systems requirements (Hypothesis 5).
The reason excellent facilitation is such a crucial
determinant of JAD’s success is that the JAD facilitator is largely responsible for inducing effective
group behavior [2]. NGT, however, provides such
assistance within its protocol. This helps to reduce
the facilitator’s burden and reduces the need for
excellent facilitation to obtain effective process outcomes. Because of the technique, novice facilitators
should reduce the performance gap between themselves and their more experienced counterparts
(Hypothesis 6).
3. Research method
In the completely randomized design we used for
this study, communication structure (JAD and NJAD)
was crossed with two levels of facilitation (expert and
novice). Table 2 depicts the 2 2 factorial design with
the four experimental conditions. In the experiment,
we varied the impact of the communication structure
and the effects of facilitation to determine their influence on the deliberations and on the resulting decisions, and controlled the effects of other potentially
influential variables.
Table 2
The 2 2 factorial design
Group structure
JAD
NJAD
Facilitator skill level
Level I (high)
Level II (low)
6 groups ( 6 members)
6 groups 6
6 groups 6
6 groups 6
E.W. Duggan, C.S. Thachenkary / Information & Management 41 (2004) 399–411
3.1. Subjects
One-hundred-and-seventy-four persons volunteered
for the experiments, which took place in three US
cities. The volunteers included 12 active JAD facilitators, 18 note takers, who did not participate in the
discussion, and 144 role-players. Four organizations–
three facilitators’ associations and a training company–
supplied the facilitators. Role-playing volunteers and
scribes came from four enterprises, two educational
institutions, and the general business community. The
diverse group of role-players included non-facilitator
members of the volunteer organizations, information
technology workers, graduate and undergraduate business students, and business professionals. The participants had an interest in the outcome of the experiments
and were promised copies of the results.
3.2. The task
The simulated case selected for the experiment was
designed to demonstrate the processes and problems
of systems analysis and provide an appreciation for the
human factors involved [54]. The groups were
required to determine high-level systems requirements
for an integrated order processing, inventory management, accounts payable and receivable, and distribution management system to support a chain of owned
and franchised deli-style sandwich shops.
405
Facilitators were extensively debriefed for approximately 1 h before each experiment, which consisted of
at least two group sessions.
On the day of the session, the facilitators were given
the freedom to select the order in which they applied the
two meeting structures. Immediately before the start of a
session, which lasted for approximately 2 h, participants
completed the pre-session survey, reviewed the rules for
the session, and read the case. During the (audio) taped
session, they produced the solution set of system features. A scribe (selected from the available volunteers)
supported one or two sessions and documented the final
solutions. Immediately after the session, the participants
completed the post-session questionnaire and the facilitators recorded their observations.
An independent volunteer that was both treatmentand hypothesis-blind later coded the data into the
database and typed all the hand-written requirements.
A second volunteer compared the original and typedup versions of the requirements and verified their
accuracy. The typed documents and the rating guidelines were then sent to the three judges.
3.4. Instruments
Four questionnaires were used to collect data
(Table 3). They were all adapted from instruments
developed by researchers in previous studies.
3.5. Variables and measures
3.3. Procedures
Twenty-four, six-member groups played user and
developer roles in generating the requirements for the
case. The participants were randomly assigned to a
group, and groups were randomly assigned to one of
the four experimental conditions. Each group was
supported by a scribe and directed by a facilitator,
who was already familiar with the details of the case.
The participating organizations classified their volunteer facilitators into the two skill categories.
Twelve facilitators (six from each category) were
randomly pre-selected to conduct the 24 sessions.
Each facilitator conducted two sessions—one with
each meeting structure. Facilitators’ packets containing the description of the task, sample questionnaires,
data collection instructions, and a script to conduct the
NJAD sessions were delivered to them beforehand.
The input variables from our research model were
either manipulated in the experiment (group structure
and facilitation skill level) or controlled experimentally (e.g. all groups were of the same size and used the
same task) or statistically (group experience, level of
effort, and experience level of the group). The dependent variables and their operationalization are shown
in Table 4.
3.6. Analytical procedures
The analysis included a pre-examination of the
data to gauge the reliability of the measurement
instruments, establish construct validity, and determine whether there was a need for co-variation
analyses to control influential variables that were
not controlled experimentally. Multiple analysis of
406
E.W. Duggan, C.S. Thachenkary / Information & Management 41 (2004) 399–411
Table 3
Instruments used in the study
Instrument
Completed by
Comments
Source
Pre-session
questionnaire (PI1)
Post-session
questionnaire (PI2)
Facilitators survey (FI)
Participants
[3]
Facilitator
Rating sheet (RI)
Expert judges
Objective background information on participants’ profile
(nine questions: six on a seven-point Likert scale, three single-answer)
Four multi-item scales to capture perceptual data about the
process and outcome (24 items, seven-point Likert scale)
Facilitator report on session: mix of objective and perceptual
data (five items)
10 items including an eight-item quality rating sheet (scores of 0–5)
Participants
[3,34,35]
[30]
[8]
Table 4
Operationalization of constructs
Construct
Measure
Operationalization
Process effectiveness
Participation
Self-reported
Number of non-participants or minimal contributors (FI)
Two-item scale from PI1
Three-item facilitator instrument
Two-item scale (PI1)
Number of unresolved items at the end
(reported by the facilitator)
Two-item scale (PI1)
Seven-item scale (PI1)
Process efficiency
No. of unique features divided
by meeting duration
Meeting duration (FI)
Number of unique features determined by expert judges
Satisfaction with the process
Self-reported
Four-item scale (PFI)
Satisfaction with the outcome
Self-reported
Five-item scale (PFI)
Requirements definition quality
Quality points
Three judges’ scores (on accuracy, precision, completeness,
conciseness, relevance, creativity, consistency, and feasibility)
Domination
Conflict resolution
variance (MANOVA) and analysis of variance
(ANOVA) were the main statistical procedures used
to test the hypotheses.
Factor analysis was conducted to establish construct
validity and reliability analysis to investigate the
internal consistency of the measurement scales. The
resulting factor loadings suggested a high degree of
construct validity. Other statistical techniques used in
support of the main procedures included discriminant
analysis to assess facilitator classification accuracy
and Kendall’s coefficient of concordance for testing
agreement among raters.
4. Summary of results
The tests conducted on the control variables
indicated no need for co-variation analyses. No
significant difference among the groups was found
for the level of effort expended for either group
structure (P ¼ 0:093) or facilitation (P ¼ 0:254).
Similar results were obtained for group experience
in a professional business environment and with SRD
(P ¼ 0:758 and 0.290 for group structure and facilitation, respectively).
Cronbach’s reliability test was used to revalidate the
instruments and produced the following Alpha ratings:
the post-session instrument, 0.91; attitude to group
work (PI2), 0.79; destructive dominance (FI), 0.83;
and the expert judges’ rating instrument 0.98. Kendall’s coefficient of concordance indicated reasonable
evidence of congruence among the expert judges (at a
significance level of 0.058), and that they applied
similar standards of assessment.
The two-group discriminant analysis to determine
whether facilitators were accurately categorized also
E.W. Duggan, C.S. Thachenkary / Information & Management 41 (2004) 399–411
407
Table 5
Summary of results of the tests of hypotheses
Hypothesis
Hypothesis
Hypothesis
Hypothesis
Hypothesis
Hypothesis
Hypothesis
1:
2:
3:
4:
5:
6:
Process effectiveness
Process efficiency
Satisfaction with process
Satisfaction with outcome
SRD quality
Difference in process effectiveness
Statistical technique
Significant
Hypothesis supported
ANOVA
MANOVA
ANOVA
ANOVA
ANOVA
MANOVA
Yes ðP < 0:003Þ
No ðP < 0:101Þ
Yes ðP < 0:034Þ
Yes ðP < 0:017Þ
Yes ðP < 0:011Þ
Yes ðP < 0:032Þ
Yes
Yes
Yes
Yes
Yes
Yes
failed to reject the equality hypothesis (P ¼ 0:092).
Based on participants’ evaluation of the competence
of the facilitators, the discriminant analysis indicated
classification accuracy of 66.7%.
The empirical tests corroborated our prediction that
NJAD would outperform JAD with respect to the
quality of the requirements, and confirmed our expectation that the integrated approach would be at least as
efficient as JAD. The results also supported our contention that participants using NJAD would be more
satisfied with the process and identify more with the
requirements than those using JAD. The findings also
indicated that the integrated structure helped to reduce
the need for highly competent facilitators to obtain
successful JAD outcomes. Table 5 summarizes
these results.
5. Discussion of the results
The main premise of our theory that the integration
of JAD and NGT would contribute to greater process
effectiveness than JAD alone was corroborated in the
experiment. This was an important result for the
consistency of the model because the expectations
for user satisfaction with the process and outcome
and the quality of the requirements (which were also
supported by the findings) were all based on the
confirmation of this proposition.
The empirical support for the proposition that
NJAD was as least as efficient (time conserving) as
JAD is important for ISs practitioners. It reduces the
risk of lengthy NJAD meetings jeopardizing the
release of busy managers and key personnel whose
participation are required for effective results. It
should be reassuring to practitioners if NJAD, despite
its more rigorous structure, could preserve the benefit
of reduced SRD cycle time, which is one of JAD’s
acknowledged strengths.
In systems development, satisfaction with the SRD
process may be necessary for achieving psychological
identification with, and ownership of the project,
which increases the probability of acceptance of the
final system. Satisfaction with the process that generates the requirements may also be a prerequisite for
users’ satisfaction and identification with the requirements generated by the process. Dissatisfied users are
often obstacles to system acceptance and deployment.
Newman and Robey [57] maintained that systems
success is not always determined by technical validity
but also by the ‘‘affective and behavioral responses
of users.’’
The results support our proposition that NJAD
would produce higher quality requirements than
JAD. This is promising for practitioners who have
struggled with poor systems requirements. SRD,
though not an end in itself, is a critical determinant
of system success. High-quality requirements have a
ripple effect that contributes to better results in later
development phases and eventually to technically
sound, and maintainable systems.
We believed that excellent facilitation was required
for excellent JAD outcomes but we also proposed the
reciprocal relationship (for NJAD) where the communication structure assisted facilitators in reducing
problematic group behaviors. Our final result indicated that NJAD reduced the performance gap
between competent and unskilled group leaders; it
appeared to reduce JAD’s critical dependence on
excellent facilitation for high-quality results. This
finding could pave the way for applying NJAD in
those cases where the unavailability of competent
facilitators would otherwise have precluded the use
of JAD.
408
E.W. Duggan, C.S. Thachenkary / Information & Management 41 (2004) 399–411
These results do not signal the end of JAD; our
proposal advocates that JAD be integrated with NGT.
Also, the results are only applicable to user–developer
communication problems and their effects on the
quality and acceptance of the resulting requirements.
Several other activities (e.g., project management,
systems design, testing, and implementation) have
their independent influences on the delivered system.
6. Conclusions
Our research can be considered a follow up to
studies that (1) found no significant difference in
results produced by groups using GSS and NGT
and (2) examined the effectiveness of using JAD in
a GSS environment for SRD without formally investigating the effects of facilitation. Anson et al. [3]
examined the impact of facilitation on meeting outcomes, with and without GSS, but did not include
either JAD or SRD. Our study considered group
structure and facilitation effects simultaneously with
SRD.
Two limitations should be mentioned. The unrealistic setting and the restrictive experimental conditions necessary to establish internal validity reduce our
ability to generalize the results. It is not clear whether
their full effects can be reproduced in a natural SRD
setting characterized by emotional issues, political turf
wars, vested interests, and especially, the differences
in the rank and knowledge of participants. These are
the basis of the difficult problems that facilitators are
likely to encounter.
Another limitation was the difficulty in specifying
suitable criteria for classifying the facilitators. The
measures of facilitation experience and the number of
sessions actually conducted were used, but organizations that participated applied other (undisclosed)
means of classification. However, the likelihood of
misclassification was reduced, because the skills
required in this experiment were at opposite ends of
the facilitator competence scale. In any event, the
discriminant analysis provided evidence that the classification was reasonable.
Despite the increasing sophistication of information
technology, systems builders continue to grapple with
poor systems development processes and low-quality
systems. NJAD may be able to complement other
techniques (e.g., prototyping) that have been used to
increase the quality of user participation in SRD and,
consequently, the effectiveness and efficiency of the
process and the quality of the results. These benefits
should help to reduce systems failures and, eventually,
the direct and opportunity costs of the usually high
allocation of IS resources to systems operations and
maintenance.
The results of our study promise some assistance to
this effort. The proposed integrated structure appears
to counteract the problems that inhibited JAD’s effectiveness and preserve its benefits. These effects may
help to enhance the deliverables from other phases of
SDLC that depend on systems requirements, and
improve overall system quality. Ultimately, higher
quality systems will allow organizations to reassign
resources usually allocated to systems maintenance
for other useful organizational activities.
Because NJAD helps to offset the effects of facilitation skills and reduces the reliance on facilitator
competence for outstanding process outcomes, this
removes one of the potential dilemmas when a skillful
facilitator is unavailable: not to use JAD at all or to
make do with an unskilled facilitator. Superb facilitation, though desirable, is not a critical success factor
under NJAD; this makes the technique more widely
available to solve pervasive SRD problems.
References
[1] D.C. Andrews, JAD: a crucial dimension for rapid applications development, Journal of Systems Management 42 (3)
(1991) 23–27, 31.
[2] D.C. Andrews, N.S. Leventhal, Fusion: Integrating IE,
CASE, and JAD, Prentice-Hall, Englewood Cliffs, NJ, 1993.
[3] R. Anson, R. Bostrom, B. Wynne, An experiment assessing
group support system and facilitator effects on meeting
outcomes, Management Science 41 (2), 1995, pp. 189–208.
[4] P. Antunes, On the design of group decision processes for
electronic meeting rooms, CLEI Electronic Journal (special
issue) 2 (1) 1999.
[5] P.M. Asaro, Transforming society by transforming technology: the science and politics of participatory design, Accounting Management and Information Technologies 10 (4), 2000,
pp. 257–290.
[6] J.H. August, Joint application design: the group session
approach to systems design, Prentice-Hall, Englewood Cliffs,
NJ, 1991.
[7] S.A. Becker, E. Carmel, A.R. Hevner, Integrating joint
application development (JAD) into cleanroom development
E.W. Duggan, C.S. Thachenkary / Information & Management 41 (2004) 399–411
[8]
[9]
[10]
[11]
[12]
[13]
[14]
[15]
[16]
[17]
[18]
[19]
[20]
[21]
[22]
[23]
[24]
with ICASE, in: Proceedings of the Hawaii International
Conference on Systems Sciences, Hawaii, 1993, pp. 13–21.
J.E. Bailey, S.W. Pearson, Development of a tool for
measuring and analyzing computer user satisfaction, Management Science 29 (5), 1983, pp. 530–545.
P.D.M. Barrow, P.J. Mayhew, Investigating principles of
stakeholder evaluation in a modern IS development approach,
Journal of Systems and Software 52 (2–3), 2000, pp. 95–103.
J.M. Bartunek, J.K. Murninghan, The nominal group
technique: expanding the basis procedure and underlying
assumptions, Group and Organization Studies 9 (3), 1984, pp.
417–432.
M.G. Beruvides, Group decision support systems and
consensus building issues electronic media, Computers in
Industrial Engineering 29 (1–4), 1995, pp. 601–605.
P. Beynon-Davies, H. Mackay, D. Tudhope, Its lots of bits of
paper and ticks and post-it notes and things: a case study of a
rapid application development project, Information Systems
Journal 10 (3), 2000, pp. 195–216.
Borovits, S. Ellis, O. Yeheskel, Group processes and the
development of information systems, Information and
Management 19 (2), 1990, pp. 65–72.
R.B Bostrom, R. Anson, V.K. Clawson, Group facilitation
and group support systems, in: L.M. Jessup, J.S. Valacich
(Eds.), Group Support Systems: New Perspectives, Macmillan, New York, 1993, pp. 146–168.
B. Broome, The role of facilitated group process in
community-based planning and design, in: L.R. Frey (Ed.),
Innovations in Group Facilitation: Applications in Natural
Settings, Hampton Press, Inc., New Jersey, 1994, pp. 27–52.
G.J. Browne, V. Ramesh, Improving information requirements determination: a cognitive perspective, Information and
Management 39 (8), 2002, pp. 625–635.
T.A. Byrd, K.L. Cossick, R.W. Zmud, A synthesis of research
on requirements analysis and knowledge acquisition techniques, MIS Quarterly 16 (3), 1992, pp. 117–138.
E. Carmel, J.F. George, J.F. Nunamaker, Supporting joint
application development (JAD) with electronic meeting
systems: a field study, in: Proceedings of the International
Conference on Information Systems, vol. 12, USA, 1992,
pp. 223–232.
E. Carmel, J.F. George, J.F. Nunamaker, Examining the
process of electronic-JAD, Journal of End User Computing 7
(1), 1995, pp. 13–22.
E. Carmel, R.D. Whitaker, J.F. George, PD and joint
application design: a transatlantic comparison, Communications of the ACM 36 (6), 1993, pp. 40–48.
R.J. Chapman, The effectiveness of working group risk
identification and assessment techniques, International Journal of Project Management 16 (6), 1998, pp. 333–343.
K.E. Cheng, A requirements definition and assessment
framework for SDL tools, Computer Networks and ISDN
Systems 28, 1996, pp. 1703–1715.
A. Crawford, Advancing Business Concepts in a JAD
Workshop Setting, Prentice-Hall, Englewood Cliffs, NJ, 1994.
E.J. Davidson, An exploratory study of joint application
design (JAD) in information systems delivery, in: Proceedings
[25]
[26]
[27]
[28]
[29]
[30]
[31]
[32]
[33]
[34]
[35]
[36]
[37]
[38]
[39]
[40]
[41]
[42]
409
of the XIV International Conference on Information Systems,
vol. 12, Nos. 5–8, 1993, pp. 271–278.
E.J. Davidson, Joint application design (JAD) in practice,
Journal of Systems and Software 45 (3), 1999, pp.
215–223.
D.L. Dean, J.D. Lee, M.O. Pendergast, A.M. Hickey, J.F.
Nunamaker, Enabling the effective involvement of multiple
users: methods and tools for effective software engineering,
Journal of Management Information Systems, 14 (3) (1997–
1998) 179–222.
A.L. Delbecq, A.H. Van de Ven, D.H. Gustafson, Group
Techniques for Program Planning: A Guide to Nominal Group
and Delphi Processes, Scott. Foresman, Glenview, IL, 1975.
A.L. Delbecq, A.H. Van de Ven, D.H. Gustafson, Group
Techniques for Program Planning: A Guide to Nominal Group
and Delphi Processes, Greenbriar, Middleton, WI, 1986.
A.R. Dennis, G.S. Hayes, R.M. Daniels Jr., Business process
modeling with group support systems, Journal of Management Information Systems 15 (4), 1999, pp. 115–142.
M. Dieckmann, Making new technology investments pay off,
Managing Office Technology 41 (7), 1996, pp. 14–16.
K.L. Dowling, R.D. St. Louis, Asynchronous implementation
of the nominal group technique: Is it effective? Decision
Support Systems 29 (3), 2000, pp. 229–248.
B. Fazlollahi, M.R. Tanniru, Selecting a requirement
determination methodology—contingency approach revisited,
Information and Management 21 (5), 1991, pp. 291–303.
S. Frankel, NGT þ MDS: an adaptation of the nominal group
technique for ill-structured problems, Journal of Applied
Behavioral Science 23 (4), 1987, pp. 543–551.
D.S. Gouran, C. Brown, D.R. Henry, Behavioral correlates of
perceptions of quality in decision-making discussions, Communication Monographs 45, 1978, pp. 51–63.
S.G. Green, T.D. Taber, The effects of three social decision
schemes on decision group process, Organizational Behavior
and Human Performance 25, 1980, pp. 97–106.
T.R. Henrich, T.J. Greene, Using the nominal group
technique to elicit roadblocks to MRP II implementation,
Computers and Industrial Engineering 21 (1–4), 1991, pp.
335–338.
R.A. Hirschheim, User experience with and assessment of
participative systems design, MIS Quarterly 9 (4), 1985, pp.
295–304.
E.S. Ho, Y.J. Lai, S.I. Chang, An integrated group decisionmaking approach to quality function deployment, IIE
Transactions 31 (6), 1999, pp. 553–567.
K. Holtzblatt, H.R. Beyer, Requirements gathering: the
human factor, Communications of the ACM 38 (5), 1995,
pp. 30–32.
G.P. Huber, Managerial Decision Making, Scott. Foresman,
Glenview, IL, 1980.
R.B. Jackson, D.W. Embley, Using joint application design to
develop readable formal specifications, Information and
Software Technology 38 (10), 1996, pp. 615–631.
J. Jiang, G. Klein, Software development risks to project
effectiveness, Journal of Systems and Software 52 (1), 2000,
pp. 3–10.
410
E.W. Duggan, C.S. Thachenkary / Information & Management 41 (2004) 399–411
[43] M.C. Kettelhut, Avoiding group-induced errors in systems
development, Journal of Systems Management 42 (3), 1991,
pp. 13–17.
[44] M.C. Kettelhut, JAD methodology and group dynamics, Information Systems Management 10 (1), 1993, pp.
46–53.
[45] M.C. Kettelhut, Using JAD for strategic initiatives, Information Systems Management 14 (3), 1997, pp. 29–36.
[46] R. Kling, The organizational context of user centered designs,
MIS Quarterly 1 (4), 1977, pp. 1–52.
[47] L.J. Korhonen, Nominal group technique, in: M.W. Galbraith
(Ed.), Adult Learning Methods, Krieger Publishing Company,
Florida, 1980, pp. 247–259.
[48] R.L. Kumar, Managing risks in IT projects: an options
perspective, Information and management 40 (1), 2001, pp.
63–74.
[49] A.L. Lederer, Information requirements analysis, Journal of
Systems Management 32 (12), 1981, pp. 15–19.
[50] N.S. Leventhal, Using groupware tools to automate joint
application development, Journal of Systems Management 46
(5), 1995, pp. 16–22.
[51] Y.I. Liou, M. Chen, Using group support systems in joint
application development for requirements specifications,
Journal of Management Information Systems 8 (10), 1993–
1994, pp. 805–815.
[52] A. MacPhail, Nominal group technique: a useful method for
working with young people, British Educational Research
Journal 27 (2), 2001, pp. 161–170.
[53] J.G. Mahler, Structured decision making in public organizations, Public Administration Review 47 (4), 1987, pp.
336–342.
[54] R. Marble, Casebook for systems analysis and design: FSS
Inc., Mitchell-McGraw Hill, 1992.
[55] M.L. Markus, Power, politics and MIS implementation,
Communications of the ACM 7, 1983, pp. 430–614.
[56] C.M. Moore, Group Techniques for Idea Building, Sage
Publishing, Newbury Park, CA, 1987.
[57] M. Newman, D. Robey, A social process model of user–
analyst relationships, MIS Quarterly 16 (2), 1992, pp.
249–266.
[58] F. Niederman, C.M. Beise, P.M. Beranek, Issues and concerns
about computer-supported meetings: the facilitator’s perspective, MIS Quarterly 20 (1), 1996, pp. 1–22.
[59] J.F. Nunamaker, A.R. Dennis, J.S. Valacich, D.R. Vogel, J.F.
George, Group support systems research: experience from the
lab and field, in: L.M. Jessup, J.S. Valacich (Eds.), Group
Support Systems: New Perspectives, Macmillan, New York,
1993, pp. 125–145.
[60] J.F. Nunamaker, A.R. Dennis, J.F. George, J.S. Valacich, D.R.
Vogel, Electronic meeting systems to support group work:
theory and practice at Arizona, Communications of the ACM
34 (7), 1991, pp. 40–61.
[61] R. Ocker, S.R. Hiltz, M. Turoff, J. Fjermestad, The effects of
distributed group support and process structuring on software
requirements development teams: results on creativity and
quality, Journal of Management Information Systems 12 (3)
(1995–1996) 127–153.
[62] Pinsonneault, K.L. Kraemer, The impact of technological
support on groups: an assessment of the empirical research,
Decision Support Systems 5 (2) (1989) 197–216.
[63] M.S. Poole, G. DeSanctis, Understanding the use of group
decision support systems: the theory of adaptive structuration,
in: C. Steinfield, J. Fulk (Eds.), Organization and Communication Technology, Sage Publications, Beverly Hills, 1990,
pp. 173–179.
[64] R. Purvis, V. Sambamurthy, An examination of designer and
user perceptions of JAD and the traditional IS design
methodology, Information and Management 32 (3), 1997,
pp. 123–135.
[65] S. Raghaven, G. Zelesnik, G. Ford, Lecture Notes in
Requirements Elicitation, SEI-94-EM-010, Carnegie Mellon
University, Pittsburgh, 1994.
[66] E.B. Richmond, M.T. McKnelly, Alternative user survey and
group process methods–nominal group technique applied to
US depository libraries, Journal of Government Information
23 (2), 1996, pp. 137–149.
[67] O. Rist, Reinventing E-Business, Network Computing 12
(20), 2001, pp. 41–47.
[68] S.P. Schuman, What to look for in a group facilitator, Quality
Progress 29 (6), 1996, pp. 69–72.
[69] B.Y. Stephenson, L.K. Michaelsen, S.G. Franklin, An
empirical test of the nominal group technique in state solar
energy planning, Group and Organization Studies 79 (3),
1982, pp. 320–334.
[70] Teltumbde, A framework for evaluating ERP projects,
International Journal of Production Research 38 (16) (2000)
4507–4520.
[71] J.B. Thomas, R.R. McDaniel Jr., M.J. Dooris, Strategic issue
analysis: NGT þ decision analysis for resolving strategic
issues, Journal of Applied Behavioral Science 25 (2), 1989,
pp. 189–200.
[72] L. Uden, Design and evaluation of human centered CIM
systems, Computer Integrated Manufacturing Systems 8 (2),
1995, pp. 83–92.
[73] J. van Maurik, Facilitating excellence: styles and processes of
facilitation, Leadership and Organizational Development
Journal 15 (8), 1994, pp. 30–34.
[74] A.H. Van de Ven, A.L. Delbecq, Nominal versus interacting
group processes for committee decision-making effectiveness,
Academy of Management Journal 14, 1971, pp. 203–212.
[75] A.H. Van de Ven, A.L. Delbecq, The effectiveness of
Nominal, Delphi, and interacting group decision making
processes, Academy of Management Journal 17 (4), 1974, pp.
605–621.
[76] R.T. Watson, G. DeSantis, M.S. Poole, Using a GDSS to
facilitate group consensus: some intended and unintended
consequences, MIS Quarterly 12 (3), 1988, pp. 463–478.
[77] J.C. Wetherbe, Executive information requirements: getting it
right, MIS Quarterly 15 (1), 1991, pp. 51–65.
[78] J. Wood, D. Silver, Joint Application Development, Wiley,
New York, 1995.
[79] N. Zuech, Identifying and ranking opportunities for
machine vision in a facility, Industrial Engineering 24,
1992, pp. 42–61.
E.W. Duggan, C.S. Thachenkary / Information & Management 41 (2004) 399–411
Evan W. Duggan is an assistant professor
of MIS in the Department of Information
Systems, Statistics, and Management
Science at the University of Alabama.
He received his PhD and MBA from
Georgia State University and BSc from
the University of the West Indies, Jamaica. He has over 25 years of experience in
IT with a major multinational organization. His research interests involve the
management of ISs in corporations including IS success factors, human and group issues in systems
development and implementation, and group support systems and
facilitation. He has research publications (or articles forthcoming)
in the International Journal of Industrial Engineering, Journal of
International Technology and Information Management, Information Technology and Management, Journal of End User Computing, Information Resources Management Journal, and Human–
Computer Interactions. He has taught MIS and Decision Sciences
courses at the graduate and undergraduate levels in several US and
international institutions, including executive MBA programs.
411
Cherian S. Thachenkary is an associate
professor of Management in the J. Mack
Robinson College of Business at Georgia
State University in Atlanta, GA, USA. He
obtained his PhD and MSc degrees in
Management Sciences from the Faculty
of Engineering at the University of
Waterloo in Canada, and a BSc from the
University of Toronto. He conducts teaching and research in the area of Management of Technology. His current focus is
on the digital economy—the cost/benefits of high bandwidth
networks and electronic services/applications. His research has
been published in the Journal of Telemedicine and e-Health, OR/
MS Today, Computer Networks and ISDN Systems, IEEE
Transactions on Engineering Management, Office: Technology
and People, and Computerworld. He has served as a Senior Editor
of Computer Networks and ISDN Systems, and as a Departmental
Editor of DATABASE. He has also held faculty appointments at the
University of Waterloo in Canada, The University of Waikato in
New Zealand, and at Cairo University in Egypt.