Trademark and Copyright Warning
© Copyright 2010-2012, Kirkpatrick Partners, LLC.
All rights reserved.
This presentation remains the exclusive property
of Kirkpatrick Partners, LLC. Distribution or
duplication of these slides and content, in
whole or in part, in any way is strictly
prohibited. These are for the personal use of
program attendees only.
The following marks are the property of Kirkpatrick
Partners, LLC as related to educational / training
materials, programs and books:
Kirkpatrick®
Kirkpatrick Four LevelsTM
The One and Only Kirkpatrick®
© 2010-2011 Kirkpatrick Partners, LLC. All rights reserved.
Marching Orders for U.S. Military Training: 2012
and Beyond
Part 1: Jim Kirkpatrick, Ph.D. and Wendy Kayser Kirkpatrick
Audience Poll
What brings you to the webinar?
© 2010-2011 Kirkpatrick Partners, LLC. All rights reserved.
Session Objectives
• Identify the levels currently being utilized
in your training initiatives
• Describe how to cost-effectively measure
training at all four Kirkpatrick levels
• Set specific goals for increasing the
organizational value of training
© 2010-2011 Kirkpatrick Partners, LLC. All rights reserved.
1998
1944
Dr. Don Kirkpatrick
© 2010-2011 Kirkpatrick Partners, LLC. All rights reserved.
Quick Pre-test
In sequence from 1-4, what are the names
of the Kirkpatrick levels?
A.
B.
C.
D.
Reactions, Transfer, Knowledge, Impact
Reaction, Knowledge, Behavior, Result
Response, Learning, Transfer, Impact
Reaction, Learning, Behavior, Results
© 2010-2011 Kirkpatrick Partners, LLC. All rights reserved.
The Kirkpatrick Model
Level 1: REACTION
To what degree participants react favorably to the
training
© 2010-2011 Kirkpatrick Partners, LLC. All rights reserved.
The Kirkpatrick Model
Level 2: LEARNING
To what degree the participants acquire the
intended knowledge, skills, and attitudes
© 2010-2011 Kirkpatrick Partners, LLC. All rights reserved.
The Kirkpatrick Model
Level 3: BEHAVIOR
To what degree participants apply what they
learned during training when they are back on
the job
© 2010-2011 Kirkpatrick Partners, LLC. All rights reserved.
The Kirkpatrick Model
Level 4: RESULTS
To what degree targeted outcomes occur as a
result of the learning event(s) and subsequent
reinforcement
© 2010-2011 Kirkpatrick Partners, LLC. All rights reserved.
Audience Poll
What is the highest level to which you have
ever measured a program or initiative?
A.
B.
C.
D.
Level 1 Reaction
Level 2 Learning
Level 3 Behavior
Level 4 Results
Why Evaluate?
© 2010-2011 Kirkpatrick Partners, LLC. All rights reserved.
© 2010-2011 Kirkpatrick Partners, LLC. All rights reserved.
© 2010-2011 Kirkpatrick Partners, LLC. All rights reserved.
Level 4: Results
Return on Expectations (ROE)
What a successful training initiative delivers
to key business stakeholders,
demonstrating the degree to which their
expectations have been satisfied
© 2010-2011 Kirkpatrick Partners, LLC. All rights reserved.
Planning
Execution
© 2010-2011 Kirkpatrick Partners, LLC. All rights reserved.
Demonstration
of Value
The End is the Beginning
Reaction
Learning
Behavior
Results
© 2010-2011 Kirkpatrick Partners, LLC. All rights reserved.
The End is the Beginning
Level 4: Results
Level 3: Behavior
Level 2: Learning
Level 1: Reaction
© 2010-2011 Kirkpatrick Partners, LLC. All rights reserved.
Audience Poll
What is the highest level result / mission of
the U.S. military, in general?
© 2010-2011 Kirkpatrick Partners, LLC. All rights reserved.
Your Marching Orders
• Cost-effective training, reinforcement and
evaluation
• Utilize current resources as much as
possible
• Demonstrate mission value
© 2010-2011 Kirkpatrick Partners, LLC. All rights reserved.
Planning
Execution
© 2010-2011 Kirkpatrick Partners, LLC. All rights reserved.
Demonstration
of Value
THE NEW WORLD KIRKPATRICK MODEL
LEVEL 1
REACTION
MONITOR & ADJUST
• Engagement
• Relevance
• Customer
satisfaction
LEVEL 2
LEARNING
• Knowledge
• Skills
• Attitude
• Confidence
• Commitment
LEVEL 3
BEHAVIOR
LEVEL 4
RESULTS
• Leading
indicators
© 2010-2011 Kirkpatrick Partners, LLC. All rights reserved.
• Desired
outcomes
Level 3: Behavior
Required Drivers
Processes and systems that reinforce, monitor,
encourage, and reward performance of critical
behaviors on the job
© 2010-2011 Kirkpatrick Partners, LLC. All rights reserved.
Level 3: Behavior
Methods, tools and techniques (Required Drivers)
© 2010-2011 Kirkpatrick Partners, LLC. All rights reserved.
Level 4: Results
Leading Indicators
Short term observations and measurements
that suggest that critical behaviors are on
track to create a positive impact on
desired results
© 2010-2011 Kirkpatrick Partners, LLC. All rights reserved.
Kirkpatrick Evaluation Plan
Background:
U.S. Navy Yeoman A. School
• Trained to run Navy offices, both on ship
and shore
• Plans for them to take on more
responsibility in coming years
• History of varied training approaches,
none sufficient. A new approach is
required.
© 2010-2011 Kirkpatrick Partners, LLC. All rights reserved.
U.S. Navy Yeoman A School
Stakeholder Expectations
To efficiently run a Navy office to meet the mission of the assigned
command (including but not limited to correspondence, leave,
security, legal matters, and reports)
Level 4 Results
Cost reduction while accomplishing specific mission objectives
Level 3 Behavior
1.
2.
3.
Follow respective procedures for completing written
documentation
Maintain confidentiality and integrity in dealing with personnel
issues
Present positive customer service-oriented attitudes at all times
© 2010-2011 Kirkpatrick Partners, LLC. All rights reserved.
Audience Poll
Identify a leading indicator that would affirm
that the critical behavior is on track to
contribute to the targeted results:
“Follow respective procedures for
completing written documents”
© 2010-2011 Kirkpatrick Partners, LLC. All rights reserved.
U.S. Navy Yeoman A School
New World Additions
Level 4 Results
Cost reduction while accomplishing specific mission objectives
Leading Indicators
• Decreased errors in written documentation
• Reduced number of critical incidents
• Reduced number of kicked back reports
• Fewer complaints from executive office
Level 3 Behavior
1.
2.
3.
Follow respective procedures for completing written
documentation
Maintain confidentiality and integrity in dealing with personnel
issues
Present positive customer service-oriented attitudes at all times
© 2010-2011 Kirkpatrick Partners, LLC. All rights reserved.
U.S. Navy Yeoman A School
New World Additions
Level 3 Behavior / Required Drivers
1.
Follow respective procedures for completing written documents
Monitoring: Spot checks, observation, self- and peer-audits,
formal inspections
Reinforcing: On-going training, self-directed learning, job aids
2.
Maintain confidentiality and integrity in dealing with personnel
issues
Monitoring: Tracking security breaches, desk audits, checklist
audits
Encouraging: Coaching by commanders and peers, mentoring
(each Yeoman has one)
© 2010-2011 Kirkpatrick Partners, LLC. All rights reserved.
U.S. Navy Yeoman A School
New World Additions
Level 2 Learning
Attitude (Will this be worthwhile?)
Evaluation methods: Post-training interviews with trainers and
mentors
Confidence (I can do this on the job)
Evaluation methods: ‘Formative discussion, hybrid survey,
immediate post training interview with mentor
Commitment (I intend to do this on the job)
Evaluation methods: Action planning, hybrid survey, immediate
post training interview with mentor
© 2010-2011 Kirkpatrick Partners, LLC. All rights reserved.
U.S. Navy Yeoman A School
New World Additions
Level 1 Reaction
Engagement
Evaluation methods: Pulse checks during training, trainer
observation, hybrid course evaluation
Relevance
Evaluation methods: Group discussion, hybrid course evaluation
© 2010-2011 Kirkpatrick Partners, LLC. All rights reserved.
Planning
Execution
© 2010-2011 Kirkpatrick Partners, LLC. All rights reserved.
Demonstration
of Value
A compelling chain of evidence
demonstrates your bottom line value.
© 2010-2011 Kirkpatrick Partners, LLC. All rights reserved.
Chain of Evidence
© 2010-2011 Kirkpatrick Partners, LLC. All rights reserved.
Audience Poll
Please provide one specific example of how
you will apply these principles in your
work.
Register for FREE Resources
Links to:
•
•
•
Online resource library with 50+ items
White paper and article
LinkedIn Kirkpatrick Evaluation discussion group
Subscription to:
•
Weekly e-newsletter
–
–
–
Kirkpatrick Quick Tips
Articles
News, specials and events
Kirkpatrickpartners.com
© 2010-2011 Kirkpatrick Partners, LLC. All rights reserved.
Thank you!
kirkpatrickpartners.com
information@kirkpatrickpartners.com
(443) 856-4500