Running head: PROGRAM EVALUATION RUBRIC
Program Evaluation Rubric
Alison Chateauneuf
AET 570
February 23, 2015
Randy Howell
1
Running head: PROGRAM EVALUATION RUBRIC
2
Program Evaluation Rubric
This program evaluation will take a look at measuring the success of a current training in
a call center environment. The program focused on for this evaluation will be homeowner’s
insurance training. This module style training runs for a three day timespan and walks
participants through cost evaluators and homeowners insurance products with the overall goals
of making them proficient in navigating the systems we use to quote, and accurately assess
replacement value on a property that our policyholder is looking to insure.
This program will be deemed successful based on whether it can meet certain concepts of
Kirkpatrick’s four levels of evaluation. Kirkpatrick (2008) describes the four levels of evaluation
as follows:

Reaction: A measure of the satisfaction of the participants who attended the
program (p. 486).

Learning: The extent to which participants increased their knowledge, learned or
improved present skills, or changed their attitudes (p. 486).

Behavior: The extent to which participants applied what they learned when they
returned to their jobs (p. 486).

Results: The improvement of morale, the increase in sales or production, the
reduction in turnover, the increase in customer satisfaction, the return-oninvestment (ROI), and any other benefits that came from attending the program
(p. 486).
In keeping with this method of thought, the following rubric has been designed to give an
overall evaluation of this program.
PROGRAM EVALUATION RUBRIC
Level 1:
Reaction
Training
Design
Instructor
Exercises
3
Unsatisfactory
Satisfactory
Exemplary
Comments
When provided a
formal level 1
evaluation the
average
participant score
on the level 1
evaluation ranged
between 0-3.4
when asked about
the course
objectives, topic,
pace and level of
difficulty
When provided a
formal level 1
evaluation the
average
participant score
on the level 1
evaluation ranged
between 0-3.4
when asked about
the instructors
overall
performance,
knowledge, time
management, and
responsiveness
When provided a
formal level 1
evaluation the
average
participant score
on the level 1
evaluation ranged
between 0-3.4
when asked if the
exercises were
helpful and
meaningful
When provided a
formal level 1
evaluation the
average
participant score
on the level 1
evaluation ranged
between 3.5-4.4
when asked about
the course
objectives, topic,
pace and level of
difficulty
When provided a
formal level 1
evaluation the
average
participant score
on the level 1
evaluation ranged
between 3.5-4.4
when asked about
the instructors
overall
performance,
knowledge, time
management, and
responsiveness
When provided a
formal level 1
evaluation the
average
participant score
on the level 1
evaluation ranged
between 3.5-4.4
when asked if the
exercises were
helpful and
meaningful
When provided a
formal level 1
evaluation the
average
participant score
on the level 1
evaluation ranged
between 4.5-5
when asked about
the course
objectives, topic,
pace and level of
difficulty
When provided a
formal level 1
evaluation the
average
participant score
on the level 1
evaluation ranged
between 4.5-5
when asked about
the instructors
overall
performance,
knowledge, time
management, and
responsiveness
When provided a
formal level 1
evaluation the
average
participant score
on the level 1
evaluation ranged
between 4.5-5
when asked if the
exercises were
helpful and
meaningful
The level 1
evaluation is sent
directly to
participants and
consists of
several questions
in each category
that will average
together in order
to provide the
overall results.
The rating scale
is 0-5.
The level 1
evaluation is sent
directly to
participants and
consists of
several questions
in each category
that will average
together in order
to provide the
overall results.
The rating scale
is 0-5.
The level 1
evaluation is sent
directly to
participants and
consists of
several questions
in each category
that will average
together in order
to provide the
overall results.
The rating scale
is 0-5.
PROGRAM EVALUATION RUBRIC
Application
Logistics
When provided a
formal level 1
evaluation the
average
participant score
on the level 1
evaluation ranged
between 0-3.4
when asked if the
information
provided during
this training
program was
relevant to their
job
When provided a
formal level 1
evaluation the
participant gave
all did not meet
expectations
ratings when
asked about
facilities, seating,
and lighting
during the
training
Level 2:
Learning
Formal Level 2
Evaluation
4
When provided a
formal level 1
evaluation the
average
participant score
on the level 1
evaluation ranged
between 3.5-4.4
when asked if the
information
provided during
this training
program was
relevant to their
job
When provided a
formal level 1
evaluation the
participant gave
all met
expectations
ratings when
asked about
facilities, seating,
and lighting
during the
training
When provided a
formal level 1
evaluation the
average
participant score
on the level 1
evaluation ranged
between 4.5-5
when asked if the
information
provided during
this training
program was
relevant to their
job
When provided a
formal level 1
evaluation the
participant gave
all exceeded
expectations
ratings when
asked about
facilities, seating,
and lighting
during the
training
The level 1
evaluation is sent
directly to
participants and
consists of
several questions
in each category
that will average
together in order
to provide the
overall results.
The rating scale
is 0-5.
The level 1
evaluation is sent
directly to
participants and
consists of
several questions
in each category
that will average
together in order
to provide the
overall results.
The rating scale
is 0-5.
Unsatisfactory
Satisfactory
Exemplary
Comments
Average
participant score
on the 100
question criterionreferenced level 2
learning
evaluation ranges
from 0-79.9%
Average
participant score
on the 100
question
criterionreferenced level
2 learning
evaluation
ranges from 8089.9%
Average
Passing score
participant score on this CRT is
on the 100
80%
question
criterionreferenced level
2 learning
evaluation
ranges from 90100%
PROGRAM EVALUATION RUBRIC
5
Level
3:Behavior
Skills
Application
Unsatisfactory
Satisfactory
Exemplary
Comments
When asked to
provide level 3
feedback a
majority of
managers advise
that skills learned
in the training are
not being applied
in the workplace
When asked to
provide level 3
feedback a
majority of
managers advise
that the training
has not improved
employee
performance or
the workplace
environment
Direct
Observation
Through process
confirmations, it
has been
observed that
employees who
have attended the
training are not
using what they
were taught
When asked to
provide level 3
feedback a
majority of
managers advise
that skills
learned in the
training are
being applied in
the workplace
consistently
When asked to
provide level 3
feedback a
majority of
managers advise
that the training
has improved
employee
performance or
the workplace
environment
consistently.
Through process
confirmations, it
has been
observed that
employees who
have attended
the training are
consistently
using what they
were taught
Based on
questionnaire
handed to
management
and site
leadership
Performance
Environment
When asked to
provide level 3
feedback a
majority of
managers advise
that skills
learned in the
training are
being applied in
the workplace
most of the time
When asked to
provide level 3
feedback a
majority of
managers advise
that the training
has improved
employee
performance or
the workplace
environment in
some ways
Through process
confirmations, it
has been
observed that
employees who
have attended
the training are
using what they
were taught
some of the time
Based on
questionnaire
handed to
management
and site
leadership
Feedback
observed by site
and call center
management as
well as trainers
PROGRAM EVALUATION RUBRIC
Level 4:
Results
Business
Metrics
Unsatisfactory
Trained employees
are not meeting
the new call
handle time goals
of 6:58 within 30
days for this call
type on the job
Satisfactory
Trained
employees are
meeting the new
call handle time
goals of 6:58
within 30 days
for this call type
on the job
Trained employees Trained
are not meeting
employees are
the new after call
meeting the new
work goal of 1:13 after call work
within 30 days for goal of 1:13
this call type on
within 30 days
the job
for this call type
on the job
6
Exemplary
Comments
Trained
employees are
exceeding the
new call handle
time goals of
6:58 within 30
days for this call
type on the job
Trained
employees are
exceeding the
new after call
work goal of
1:13 within 30
days for this call
type on the job
Metric
information
based upon call
center
daily/weekly
metric report for
this category
Metric
information
based upon call
center
daily/weekly
metric report for
this category
PROGRAM EVALUATION RUBRIC
7
References
Brinkerhoff, R. O., & Mooney, T. P. (2008). Chapter 30: Level 3: Evaluation ASTD handbook
for workplace learning professionals. Alexandria, VA: ASTD Press.
Coscarelli, W., & Shrock, S. (2008). Chapter 29: Level 2: Learning-Five Essential Steps for
Creating Your Tests and Two Cautionary Tales ASTD handbook for workplace learning
professionals. Alexandria, VA: ASTD Press.
Kirkpatrick, D. L. (2008). Section VI: Measuring and Evaluating Impact - Luminary Perspective:
Evaluating Training Programs ASTD handbook for workplace learning professionals.
Alexandria, VA: ASTD Press.
Kristiansen, N. (2008). Chapter 28: Level 1: Reaction Evaluation ASTD handbook for workplace
learning professionals. Alexandria, VA: ASTD Press.
McCain, D. V. (2008). Chapter 31: Level 4: Results ASTD handbook for workplace learning
professionals. Alexandria, VA: ASTD Press.