PRESS the F5 key to begin the slideshow.
Evaluating the Effectiveness
of Your Training:
Kirkpatrick’s 4 Levels
Left Click to move to next slide and begin next audio.
Is the training program a quality program?
Does it follow...
The Principles and Process of Good
Training Development & Delivery
1.
2.
3.
4.
5.
Training Needs Assessment
Establish Learning Objectives
Content
Delivery
Evaluation
National Institute of Corrections Model:
Instructional Theory Into Practice (ITIP)
Training Cycle
Evaluation
Needs
Assessment
Set
Performance
Objectives
Deliver
Training
Design
Training
Train
Instructors
Revise
Pilot
Test
Evaluation
Assessment of Training Needs
• In order to be effective, a training must
be based on the needs of the trainees,
and the “needs” or goals of the
organization.
• Identify “The Gap” -- between what skills
employees actually have --and what they
should be
• Who should determine need for training?
Management, trainees themselves, coworkers, specialists in the field,…
Needs Assessment Sample
Questionnaire
Rate your competence in each of the following
areas giving a 1 to the lowest and a 5 to the
highest. Give a different number to each.
Rating
5
3
1
2
Juvenile Careworker Skill
Establishing clear expectations
Balancing positive & negative feedback
Giving immediate and appropriate
consequences
How to use your facility’s behavior
management plan
Establishing Learning
Objectives
• Skills-based or competency-based
objectives
• Knowledge-Based objectives: more
likely to include what the learner will
understand or be able to explain
• Attitudes: individual attitudes needed to
fully perform
Learning Objectives
Competency goals should be:
•
•
•
•
•
Behaviorally anchored
Observable
Forward-thinking
Discrete
User-friendly
Example: Supervisor frequently gives accurate
and specific feedback to subordinates about
their performance.
Hackett
Training Content
• Does content meet best practices
established by leading organizations in
the field of juvenile confinement? (NIC,
NJDA, CJCA, NCCHC)
• Does the content cover the needs of
staff?
• Does it accurately reflect local policies
and procedures?
• Is the content timely? Does it reflect
current research and theory?
Training Delivery
• Meets the needs of all learning styles –
active/kinetic, visual, social, audio
• Covers all parts of the learning cycle –
perceiving, reflecting, applying, creating
• Organized presentation
• Positive learning environment
• Empathetic, knowledgeable,
professional trainers
• Et cetera
Training Evaluation: Does the
training actually work?
• Does the training leave your staff feel
more motivated, more confident,
more knowledgeable?
• Does the training affect the way staff
do their job? How are youth affected
by the change?
• Is it causing the change in staff’s
performance that we want? To the
degree that we want?
Evaluating Training Effectiveness:
Kirkpatrick’s 4 levels
Level 4: Results
Level 3: Behavior Change
Level 2: Learning
Level 1: Reaction
Level 1 Evaluation: Reaction
•
•
•
•
•
•
Training Evaluation
Did you like this training?
Was the material relevant to your job?
Was the trainer knowledgeable?
What do you think was the most
important thing you learned?
Are the handouts helpful?
Comments: _______________
A positive reaction does not guarantee
learning, but a negative reaction almost
certainly reduces its possibility.
Level 2 Evaluation: Learning
Comparison of Test
Performance
75
70
65
60
55
Pre-Test
Post-Test
Other methods can be less formal, such as
trainer observations during practice, team
assessments, or self-assessments.
Level 3 Evaluation – Behavior
Change on the Job
Guidelines:
1. Use a control group if practical
2. Evaluate both before and after the training if
possible
3. Survey and/or interview people who can
observe the behavior
4. Get 100 percent response or a random
sampling
5. Repeat the evaluation at appropriate times
including after allowing a time lapse
Level 3 Evaluation continued:
Example questions
For trainee: How much has the program improved the
working relationship between you and your co-workers?
Large
extent
Some
extent

No
change
Made
worse
For supervisor: Since the training, how well does the
trainee do in making decisions?
Much
better
little
better

The
same
Little
worse
Much
worse
For subordinates & trainees: Since the training, how
much much does she/he ask for your opinion or support
to your ideas?
More
often
15 trainees
5 staff
Little
more
5 trainees
15 staff
The 0 trainees
same
10 staff
Less
often
0
0
Level 4 Evaluation:
Organizational Impact
• Measures broad-based changes in areas
that are important to the agency as a
whole, such as:
– decreased legal costs
– Safety improvements
– Increased arrests
• Compare measures before and after
training.
• Complicated and Expensive
Sample results of a Level 4 Evaluation
Change in use of physical restraints as
measure of Behavior Management
training
Number of incidents
$60
$40
$20
$0
Jan
Tra
t
b
Fe
en
v
E
g
n
ini
ril
p
A
Ma
y
e
Jun
y
Jul