Certainty Principle: How do We Know Training Improves
Performance?
Author: Veronica Townsend, Solutions Developer, Center for Workforce Solutions, Anne Arundel Community College
Many organizations ask for feedback at the end of a training class – was the training relevant, was the
instructor interesting and what information will you apply on the job? However, few organizations
measure the impact of training on employee performance. This is usually because it takes too much
time or is too expensive. As knowledge leader in employee development, our staff at Anne Arundel
Community College’s Center for Workforce Solutions (CWS) challenged ourselves to respond to this
market need by creating a cost-effective, easy-to-use learning outcomes tool which could be
contextualized for any organization or development need.
There are four levels of learning measurement that professional training standards include:
1.
2.
3.
4.
Participant reaction (did I like it?)
Knowledge gain (what new knowledge was acquired?)
Behavior (what changed on the job?)
Results (what was the impact on organizational measurements?)
Since we recognize the level of impact that accurately measuring on-the-job behavior change can
have on training programs and participants, our staff worked to develop a simple process that
provides standardized scoring methods while taking minimal client time. Clients spend thousands of
dollars on training, and CWS wanted to show the value of this investment by comparing participant
performance before and after the training.
Our formula for training success used a three-step approach:
1. Use focus groups of participants, managers and clients to define the current performance and
desired future performance of the participants.
2. Provide a series of training sessions spaced over time
3. Involve the supervisors of the participants in reinforcing the new desired behaviors
The Pilot
In the pilot program, our staff partnered with Goodwill Industries of the Chesapeake, using about 30
participants, to introduce this process in a real-world environment. Industries of the Chesapeake,
using about 30 participants, to introduce this process in a real-world environment. Research indicates
achieving the greatest change in behavior depends on reinforcement by supervisors. We therefore
made sure our plan provided training highlight sessions for supervisors of the participants as well as
tips to help supervisors lead on-the-job reinforcement activities, follow-up session for participants to
share their stories of achievement and rubric identifying beginning, developing, accomplished and
exemplary performance to measure the results. A section of the rubric is shown below.
Behaviors to Achieve
Success
(Identified in Focus Group)
Beginning
1
Developing
2
Accomplished
3
Exemplary
4
% of time behavior
occurs:
Less than
20%
26-50%
51-75%
>75%
Listening:
Listens to understand
what the resident/
employee says they need
as well as their underlying
needs. Asks questions to
clarify.
Open to Feedback &
Change:
Accepts feedback and
demonstrated ability to
change.
To fine-tune the rubric and automate the survey tool, we turned to Nassim Ebrahimi, Ph.D., Director of
Learning Outcomes Assessment, who integrated the CWS approach into AACC’s college-wide emphasis on
measuring learning outcomes.
“Learning is at the heart of AACC’s mission,” says Ebrahimi. “Documenting proof of learning and using the
information to enhance our offerings improves learning opportunities and better prepares your employees for
success in the workplace.”
Participants completed the rubric as a self-evaluation, while supervisors of each participant evaluated
observations over three stages: before the training, three months after the last training activity, and again five
months later. All the participants and their supervisors needed to do was check the appropriate box.
CWS staff then delivered both an in-depth data analysis and a trend analysis to Goodwill’s management team,
along with practical and objective feedback. In the pilot program, the average ratings for each of the seven
performance competencies improved between 26% and 53%. The normal bell-shaped curve of the overall
“before training” rating - about 50% in the lower “beginning” and “developing” ratings - was transferred to 90%
of the participants in the top two rating of “accomplished” and “exemplary” in the “after training” ratings. The
improved ratings were sustained between the three- and five-month measurements, indicating sustained
behavior change was observed by the supervisors on-the-job.
Our staff was excited to work with Goodwill in providing one of our key learning philosophies - that training
success is not just about having one great learning event in the classroom. We have found that employee
performance improves through a series of trainings, reinforcements and evaluations. When this learning is
linked to observable change, the value of the investment in employee training is apparent to all.
As a competitive differentiator, rubrics are simple and effective way to visually demonstrate the impact that
training has on performance improvement. Following the success of this learning outcomes pilot, our CWS staff
developed an online survey process to accommodate larger groups of participants. As a knowledge leader both
locally and nationally, CWS is committed to providing employee development that delivers measureable results
and will continue to reach out to other organizations to develop a similar program, custom-designed to achieve
exceptional performance improvement in any competency in any organization.