Educ 533 - Instruction Design, Implementation, and Evaluation
Instruction Design, Implementation, and Evaluation
Module 1
Introduction
Designing and rolling out a successful training program involves a variety of people coordinating their efforts to achieve the desired outcome. Clients and end users of the program
may be interviewed to determine the real training requirements. Instructional designers will
design and develop the program. Administrators will organize the training schedules and venue
logistics. If the program incorporates e-learning components, information technology
professionals will be involved at a number of points. Ensuring that each piece of work is
performed at the right time, using the right amount of resources and to the right standard is no
small feat. Add to this the only to be expected changes to requirements, unanticipated issues
threatening the success of the project and the usual mix of politics and personalities and you
can see why many programs fail to deliver the anticipated organizational benefits.
Experience with both successful and failed programs indicate that the chances of success
improve significantly if this complex undertaking is treated as a project, using recognized
project management principles and methods. Each discipline has its own set of accepted
project phases. For example, software development follows a Software Development Life Cycle
(SDLC) with five phases; Analyze, Design, Build, Test and Implement.
For training program development, a phased systems approach is often referred to as the
Instructional Systems Design (ISD) model. The benefits in using a structured phased approach is
that the end product is more likely to meet the genuine needs of the client and other
stakeholder groups. This approach also helps to ensure that no development activity is started
before a necessary pre-activity is completed. For example, development of participant
materials is not begun before the program design is completed. In this way, rework is
minimized, saving costs and much frustration.
The most popular rendering of the specific phases using the ISD approach is the ADDIE model.
ADDIE represents the five phases of the project, being Analyze, Design, Develop, Implement
and Evaluate. The Business Performance Pty Ltd set of project templates is based on these five
phases. Each phase is characterized by a set of activities and a project output in the form of a
tangible deliverable. The deliverable for one phase is the input for the next. Each phase also
culminates in a review, which forms a go/no-go point for deciding whether to proceed to the
next phase.
The five phases of the ADDIE model are preceded by a project initiation and planning phase.
This initial phase determines the costs associated with undertaking the project and the
expected organizational benefits resulting from it. This forms the overarching rationale for
proceeding with the project. A project definition that outlines basic project parameters such as
objectives, scope, milestones and resource requirements is then drawn up to hand over to the
1
Educ 533 - Instruction Design, Implementation, and Evaluation
Project Manager. To complete this phase, the Project Manager develops the project plan that is
then used to guide and manage the project.
Based on module, Instructional System Design (ISD) models differ from Instructional Design (ID)
models in that ISD models have a broad scope and typically divide the instruction design process into five
phases:





Analysis
Design
Development
Implementation or Delivery
Evaluation
Analysis














Analysis phase is the building block of a training program.
Analysis phase is often called a Front-End Analysis.
Analysis is performed to determine and articulate the business unit or customer's
learning, training, and performance needs
Clarify organizational and training program objectives.
Agree the scope of the training program.
Articulate training administration requirements.
Determine strategies for transferring learned skills to the workplace.
Detail project risks, opportunities and assumptions.
Investigate constraints in implementing the program, including technological, budget,
timing and duration.
List training vendor/trainer selection criteria.
Determine the target participants, program entry requirements, participant
characteristics and special needs.
Determine extent of training participant knowledge/skill assessment required.
Determine the tasks currently performed by target participants and level of
performance required following the training.
Estimate program design, development, implementation and evaluation costs, effort
required and schedule.
This is also divided into four categories, the Business Needs are linked to results, job Performance
Needs are linked to behaviour, Training Needs are linked to learning, and Individual Needs are
linked to reaction.
2
Educ 533 - Instruction Design, Implementation, and Evaluation
3
Design Phase










This phase insures the systematic development of the training program.
This process is driven by the products of the analysis phase and ends in a model or
blueprint of the training process for future development.
Translate the program objectives into terminal and enabling learning objectives.
Quantify program development, implementation and evaluation costs and effort
required.
Determine program structure and sequence.
Determine program duration and pace.
Decide program format and mode of delivery.
Specify type of participant assessments and assessment conditions.
Determine program evaluation methodology, data collection methods, timing and
reporting formats.
Articulate transfer of learning methods and workplace support.
The model or blueprint should contain at least five key outputs:
o Learning objectives
o Performance test
o Learning steps (performance steps)
o Entry behaviors
o Structure and sequence the instructional outline
Development Phase
Educ 533 - Instruction Design, Implementation, and Evaluation










This phase elaborates and builds on the Learning Objectives and Learning Steps that
were produced in the design phase
Develop communication packs for program stakeholders.
Develop session plans, trainer guides, learner guides and trainer and participant
resources.
Develop trainer and on-the-job aids.
Develop coaching/mentoring guides and resources.
Develop technology infrastructure and software.
Develop participant assessments.
Develop project and program evaluation instruments.
Conduct pilot program to test that program meets client requirements.
Review implementation and evaluation costs, effort required and schedule.
Implement













Rollout program communications to stakeholders.
Produce program materials and aids.
Install technology infrastructure and services.
Set up administrative databases and systems.
Install on-the-job aids.
Prepare coaches/mentors.
Book venue, accommodation and travel arrangements.
Set up venue and accommodation.
Schedule participants.
Conduct training sessions.
Implement training transfer strategies.
Conduct participant assessments.
Collect participant feedback.
Evaluation in Instructional Design






Evaluation is the systematic determination of merit, worth, and significance of a
learning or training process by using criteria against a set of standards
Collect training program evaluation data.
Collect project evaluation data.
Review training program performance (number of employees trained, percent
participants passed, participant satisfaction).
Review project performance (cost, schedule, scope, stakeholder satisfaction, project
team satisfaction).
Report program and project performance results
Evaluations help to measure Reich's gap by determining the value and effectiveness of a
learning program. It uses assessment and validation tools to provide data for the evaluation.
4
Educ 533 - Instruction Design, Implementation, and Evaluation
Assessment is the measurement of the practical results of the training in the work
environment; while validation determines if the objectives of the training goal were met.
The main purpose of the evaluation was to focus on the process of implementation rather than
on its impact, since this would be minimal after such a short time, accessing in particular the
participatory approaches used to identify project beneficiaries and the communities’ role in
implementing and monitoring the project.
Systems & Processes in Instructional System Design
System is defined as a set of concepts or parts that must work together to perform a particular
function.
In ISD, the system is composed of:
o People: The instructional designers, SMEs, trainers, learners, etc.
o Material: The content used to help the learners become better performers.
o Technology: The learning methodologies, media, etc. It also includes the
technology that the learners are trying to master.
o Time: The time invested in creating the learning platform and the time used to
train the learners.
The three basic functions are:
o Input: The instructional designers, SMEs, trainers, learners, content, learning
methodologies, media, etc.
o Process: Aid the learners with gaining new knowledge and skills in a new
technology.
o Output: Performers.
The system in ISD is almost the same with computer technology that has a set of concept or
instruction for particular task or function, and has three basic functions which are inputs,
process and output.
Processes
A process is a planned series of actions within a system that advances material or procedures
from one stage of completion to the next.
Some of the processes that take place in a learning department include:
5
Educ 533 - Instruction Design, Implementation, and Evaluation
o Registration: people who want to learn —> registration forms completed —> people
who are now registered for class.
o Development: training need —> develop courseware —> a learning program.
Computer Training Class: students who need to learn MS Word —> learning platform —>
employees who can now perform
ISD as Design Science
ISD is more a part of the design sciences, which offer ways to perform certain human-defined
tasks.ISD is a model to aid in the design, development, and delivery of a training program. It is a
valuable toolbox that provides proven methods for building viable training programs. This
means the people within the organization, to include learners, designers, & managers must
control the ISD model, rather that it controlling them. Immediate problems often arise that
require rapid solutions
The steps in each phase should not be thought of as concrete in nature but rather dynamic
(Merrienboer, 1997). Going back to the dynamic nature of ISD , one step does not necessarily
have to be completed before the next one is started, hence the dashed-arrows in the diagram
below that show the processes or phases can be iterated:
6
Educ 533 - Instruction Design, Implementation, and Evaluation
For example, some designers will have to complete part of the work in the design phase before
they can complete the estimate step in the analysis phase. In the development phase, the first
three steps, list learner activity, select delivery system, and review existing material, might be
combined into one step by some designers. In addition, once a learning process is created, it
might go through one or more iterations until it works as intended.
Every training process will develop its own rhythm, which means designers need to find the
natural flow of the steps required to produce a successful learning platform. Although the ISD
process is normally considered a formal one in that the five phases should be performed in a
designated work-flow (click button below to view the flow), it requires both art and science in
its implementation in that not all flows will be performed and new ones may be required as
needed.
Critiques and Comparison of ISD

The ISD model has been criticized because it is frequently presented in flowchart form,
leaving the impression that it is mechanistic and linear in its approach:

The ISD model is a management tool that makes the creation of learning platforms or
learning processes more efficient

It is this heuristic design that has been criticized by others because it tells training
designers what to do, but not how to do it.

Some have been calling ADDIE (ISD) a process model that can be used for any number of
outcomes, such as building a house, strip mall, or a web site. A true ISD model, on the
other hand, is limited to instructional outcomes

ISD is that it is too time-consuming
Learning Activities: Computer Software development
Analysis: This is the process of deciding exactly what a computer
system is to do. The traditional approach to analysis is to thoroughly
document the existing system that the new system is intended to
replace, even if the existing system is entirely manual and rife with inefficiency
and error. Then, a specification for a new system to replace the
old system is created. This specification defines exactly what the new
7
Educ 533 - Instruction Design, Implementation, and Evaluation
system will do, but not necessarily how it will do it.
Design: This process creates a plan for implementing the specification
for a new system that results from the analysis step. It focuses on how
the new system will work.
Develop: The actual creation of the software base on what is in design phase.
Implementation: Here’s where the programs that make up the new system
are coded and tested, the hardware required to support the system is purchased
and installed, and the databases required for the system are
defined and loaded.
Testing/maintenance/evaluate: In this phase, all pieces of the system are checked
out to make sure that the system works the way it should. The moment the computer system
goes into production, itneeds maintenance. In this dreaded phase, errors — hopefully minor —
that weren’t caught during the implementation and acceptance phases arecorrected. As the
users work with the system, they invariably realize that
what they really need isn’t what they said they wanted, so they requestenhancements — which
are gradually incorporated into the system.The biggest challenge of this phase is making sure
that corrections andenhancements don’t create more problems than they solve.
Summarized by:
Abelardo T. Bucad
8