AN ALTERNATIVE DESIGN
MODEL FOR BUILDING
ELECTRONIC PERFORMANCE
SUPPORT SYSTEMS
Kursat Cagiltay
Instructional Systems Technology and
Cognitive Science
Indiana University
July 15, 2002
What is EPSS?
• EPSS is
– a computer based (electronic),
– user controlled and easy to use system that provides
support at the moment it is needed (right time),
– presents relevant (right type) and
– context-focused (right amount) information that the
performer needs,
– in the real work environment (right place)
• Components of EPSS
–
–
–
–
Database
Tools
Instructional component
Expert system
Why do we need a D/D model?
• EPSSs are significantly different from instructional
systems.
– Main focus is performing rather than learning
– EPSS is used while doing the actual job, not beforehand
– No need to follow a predetermined sequence
• Especially the focus on the "work process" makes
the design and development of an EPSS quite
different than the traditional models of instructional
design. (Witt & Wager, 1994; Cole et al.,1997; Cote, 2000)
• There is a broad consensus that there is a lack of
EPSS design and development models (Gustafson, 1993;
Gustafson, 2000; Milheim, 1997; Laffey, 1995; Rosenberg, Coscarelli & Hutchison, 1999)
Research Questions
• Given that the traditional models of instructional
development are inappropriate for EPSS
development, what are the primary distinctive
characteristics of an effective and efficient EPSS
design and development process?
• What should an effective and efficient design
model incorporate for the development of an
effective and efficient EPSS?
Methodology
• Multiple case studies
• Formative research methodology (Reigeluth & Frick, 1999)
– Naturalistic case study
– Cases are not specifically designed according to
the theory
– Cases are analyzed to see the failing points and
valuable elements
• Two major types of naturalistic formative research
procedures
– “In vivo” (during the process) and
– “Post facto” (after the process)
• Research process
Select
several
cases

collect and
analyze data
on the cases

fully develop a
tentative
model
Methodology
• Cases
– A higher educational setting (in-vivo, postfacto)
– A car manufacturing company (post-facto and
in-vivo)
– A telecommunications company (in-vivo)
Methodology
• Data collection
– Observations
• In-vivo only
– Interviews
• D/D team members, end-users, managers
– Document Analysis
• Web-sites, reports, e-mail logs, proposals
• Data Analysis
– Continuous and iterative
– Explored similarities and differences among the cases
– Looked for: social, political and technical aspects of the
EPSS design/development, and process related issues
Results: The Underlying Principles of the Model
•
Principles related to social issues
1. Identify and develop awareness and need
for EPSS
2. Understand political/cultural characteristics
and history of the organization and the
stakeholders
3. For active involvement establish good
communication strategies
4. Explore and create mechanisms to increase
motivation for active involvement
Results: The Underlying Principles of the Model
•
Principles related to technical issues
5. Form a team with technically competent
members
6. Focus on compatibility, portability and
standardization of the EPSS
7. Make early decisions about the best technology
needs
8. Design a seamless Performer-EPSS-Interaction
environment (Human-Computer-Interaction)
9. Manage content and information overflow
10. Get help from the Information Systems discipline
Results: The Underlying Principles of the Model
•
Principles related to process issues
11. Follow a dynamic EPSS design/development
process
12. Form a multi-skilled/multi-disciplined team
13. Create shared vision and make sure everybody
believes it
14. Forecast risks in advance and develop a risk
management mechanism
Two general patterns were observed
1. Cases were developed for complex sociotechnical organizational environments, and the
D/D processes have shown a dynamic nature.
2. Throughout the entire D/D process, the team
members had to make changes and revisions
of their plans to overcome problems: adaptive
The proposed EPSS D/D model is based on the 14
principles. It reflects these two patterns and
received its name from them:
Adaptive Dynamic EPSS Model:
ADEM
• ADEM includes
– people/organization,
– the technology that supports them, and
– the processes that tie them all together with an
emphasis on peopleware (soft) issues
• The focus was more on the generic design
process, with the major components that reflect
the patterns of design activities carried out by
EPSS designers in the field.
• The ADEM is a framework for portraying overall
EPSS design and development. Shows more
descriptive model characteristics than a step-bystep, prescriptive one
Two Foundations of the ADEM
• The development of the ADEM was helped by
Chaos Theory.
– Kellert (1993) defines Chaos Theory as, “qualitative
study of unstable aperiodic behavior in deterministic
nonlinear dynamical systems” (p. 2)
• Information System (IS) development and EPSS
development have many similarities, so if one
side of the ADEM stands on Chaos Theory, the
other side is based on the IS discipline
Conceptual Elements of Chaos
• Sensitive dependence on initial conditions
• Fractals: Fractal implies recursion,
patterns inside of patterns (feedback
between parts or levels of the system )
• Strange Attractors (choices of behavior)
• It (and ADEM) originates from nature
• How?
Visual representation of the ADEM
inspired from fractal geometry
Mandelbrot-Julia set fractal
Recursion: Patterns inside patterns
Mandelbrot-Julia set fractal
ADEM: General Process Layout
ADEM: Feedback Mechanism
• Classical negative feedback mechanisms
makes the system resistant to change
• ADEM has a positive feedback
mechanism or the learning loop approach
– Such loops make a system open to change
– The growth of knowledge during the overall
process makes learning loops or positive
feedback possible
The Process of the ADEM
• Through increments of
several iterations
• Increments occur
concurrently
• Change is seen to be
normal
• Time or schedulebased phases:
time-boxing
Task-1
Task-2
Task-3
Task-4
Task-n
Time
Adapted form Highsmith (2000)
Time-Boxes
• ADEM approaches the D/D process with a
timeline lifecycle policy (time-boxing).
• Major time-boxes are:
– Pre-project time-box: Feasibility, speculation
– Take-off time-box: Planning & Design
– Rising time-box: Development
– Destination time-box: Implementation &
maintenance
Pre-project time-box
• Feasibility – Mindset study – Speculation
– Methodological decisions
– Is EPSS the right solution? Go, no-go.
• Major Outcomes
– Vision statement
– Risk management plan
• Major issues
– Organization type, support, motivation
mechanisms, communication, vision, risks,
technology, information resources
Take-off time-box
• Analysis & Design
– Functional & structural requirements of the
EPSS are determined
– Analysis framework uses Checkland’s SSM
CATWOE analysis
– Based on the analysis, activities and timeboxes are determined. Time mgmt. policy!
• Major issues
– Organizational environment, iterative process,
participatory design, technical decisions,
multiple perspectives, positive feedback
Rising time-box
• Development
– Based on the blueprint of the previous stage
– “Concurrent component engineering”
– Incorporates multiple perspectives (social,
technical, organizational)
– Results driving approach (goal: quality products)
– Development is not finished - it evolves
• Major issues
– Multidisciplinary collaboration, short cycles,
learning from mistakes, active participation
Destination time-box
• Implementation – Moving to real-life environment
– Input from more and more users
– Analysis and design still continue
– “As soon as the design is implemented, its
consequences indicate the need for redesign” Cherns (1976)
– Weakest part of the ADEM – No cases reached it
• Major issues
– Political and cultural issues, communication, reward
mechanisms
Model Summary
Methodology
Adaptive & Dynamic, Socio-Technical
People
All of the actively participating stakeholders
Team
Multidisciplinary
Environment
Complex
Process
Product-oriented, concurrent, iterations with
increments
Change
Continuous, learning-based
Evaluation
Parallel to activities, continuous testing, positive
feedback
Management
Timeboxing, Risk management, Vision-based
Technology
Standard, compatible, suitable
Discussion
• Is it safe to use the ADEM?
– Answer is Yes: Needs good planning and careful work
• Does it need more work?
– Answer is Yes: Needs to be critically evaluated and
improved
• Hard to use it?
– Yes, but simple solutions work for simple problems
• The model itself needs to be improved in dynamic
and adaptive way
Limitations
• Three cases were investigated
• Identifying the underlying detailed steps of
the ADEM was beyond the scope of this
study. They can be addressed in future
studies
• The principles are tentative, guidelines are
not widely tested, and the model has not
been applied in any real case
Future Research
• Further research on Chaos Theory is
needed for EPSS D/D.
• A strong bridge definitely should be
established between IS and EPSS
research.
• Detailed guidelines need to be developed
for the phases and the implementation
phase needs to be made clearer
Questions?