Why is Learning Measurement important?
Training is an investment to achieve organizational objectives and it is important to determine whether the
investment in the training event was worthwhile.
Training costs can be significant in any business. Most employers are prepared to incur these costs because
they expect their business to benefit from employees' development and progress. The extent to which
businesses have benefited can be assessed by evaluating training appropriately. There are also other good
reasons for evaluating training. It helps you to:

track the development of staff knowledge and skills

find out if the learning is being applied in the workplace

identify training gaps and future training needs

establish if the investment was worthwhile

inform future training plans and strategy

ensure training continuously improves
What is good measurement?
Good measurement for learning interventions should include the following:
Identifying SMART objectives that are aligned to the success criteria

Aiming to address all levels of a Learning Measurement Framework

Identifying and gaining agreement on the measurement ahead of developing the learning intervention
and selection of the measurement tool in the Measurement Plan

Identifying the right data and tool for measurement
When to take measurement?
Most programs should include data collection immediately following a program. Depending on the
measurement objectives, follow-up data collection may be needed. Typical questions that may be answered
include:
Immediately after training…

To what extent were the identified training objectives achieved by the program?

What specifically did the learners learn?

What commitment have the learners made about the learning they are going to implement on their return
to work?
Follow up…

How successful were the trainees in implementing their action plans?

To what extent were they supported in this by their line managers?
Business Outcomes: ROI (Return on Investment), ROE (Return on
Expectations) and Risk Avoidance
There is often the temptation to look at the impact of learning as a financial figure based on the cost of the
programme looking at measurable business benefit associated with the cost of that programme. In some
cases this can be quantified succinctly (such as learning investment associated with learning a new process or
IT system), but longer term ROI may be difficult to prove for behavioral learning programmes. For this reason,
it is often worthwhile to express learning measurement in terms of risk reduction or avoidance (the risk of NOT
learning towards achieving a strategic objective) or return on expectation (ROE). Given that ROI is difficult to
prove through direct links to having learned something (and many CEO’s are realizing that it is not a reliable
figure for measuring learning), only consider its use where there is a real and identified need to quantify
learning measures in these terms. The investment in the systems and statistical analysis may outweigh the
investment costs associated with its calculation when the outcome the organization is seeking is quite obvious
and the cost of trying to prove the direct link is not worth the effort.
Benchmarks
Sometime it is appropriate to establish benchmarks. These help to show the improvement in the identified
measures. Benchmark data should be collected prior to the training event. There are various ways to set
benchmarks:

Existing internal or external data

Estimates of baseline data by participants or others

What is the current state?
Examples of benchmarks are:

Identifying the average score of a Pre-Assessment test

Viewing metrics (sales, performance) for a group of trainees prior to their training event
How to Measure
Defining SMART Learning Objectives
Critical to good measurement is defining the right SMART learning objectives that are aligned to what you are
trying to achieve with the learning intervention. Subsequently, all measurement of the impact and
effectiveness of a learning intervention should be aligned with the objectives of that intervention. Consider,
“What would success look like?” ensure you get the input from the appropriate stakeholders in the business in
defining success as this will tie into the measures to demonstrate results (level 4 Level of the Learning
Measurement framework.
For example, success would be a reduction in the number of accidents on the manufacturing line. The SMART
learning objectives for a training program on health and safety standard operating practice (H&S SOPs) on the
manufacturing line may include a range of objectives not just covering demonstration of knowledge of the
SOPs, but also applying them. There may also be some behavioral practices that help emphasize how an
employee would act in ambiguous situations, such as reporting faults, or choosing the right path to resolve a
problem (choices A, B, or C).
2 | Page
The Learning Measurement Framework
Research from empirical data and surveys has shown that 70% of workplace learning is obtained from tough
jobs, 20% from people (mainly the boss) and 10% from formal courses or reading (Lombardo & Eichinger. 1996)If most learning takes place in the workplace, doing the job, this means that in designing any formal training
other actions need to be considered before and after the training. Before training it is important to get clear
on the success criteria i.e. change to on the job skills, knowledge, attitude or behaviours, as well as how to
measure those criteria after the training. Historically Learning/Training groups have concentrated on
measuring the effect of the training itself (known as Level 1 Reaction & Level 2 Learning).
As the 70:20:10 rule is more accepted, it is imperative that all learning interventions/training are designed and
measured to deliver impact in the workplace (known as Level 3 Behaviours) as well as meet the success criteria
agreed by the stakeholder of the business (know as Level 4: Results).
These four levels of effective learning measurement mentioned above are the Kirkpatrick Learning
Measurement Framework. This framework is considerd one of the best models in training evaluation and
measuring the business value of the leanring programs. It is commonly used framenwork to measure
learning. Dr. Don Kirkpatrick created the elegant and simple Kirkpatrick Model in 1954. It is considered a
world-wide standard for evaluating the effectiveness of training is commonly .
The most effective evaluation of learning/training begins at the very start of desinging the program.
“Trainers must begin with desired results and then determine what behaviour is needed to accomplish them.
Then trainers must determine the attitudes, knowledge, and skills that are necessary to bring about the
desired behaviour(s). The final challenge is to present the training program in a way that enables the
participants not only to learn what they need to know but also to react favourably to the program.” Donald
Kirkpatrick, 1993.
The results are defined in measurable terms, so that all involved can see the ultimate destination of the
initiative. Clearly defined results will increase the likelihood that resources will be most effectively and
efficiently used to accomplish the mission.
Attempting to apply the four levels after a program has been developed and delivered makes it difficult, if not
impossible, to create significant training value. All four levels need to be considered at every step in the
program design, execution, and measurement.
3 | Page
Kirkpatrick identified 4 levels of measurement. The 4 levels are described below:
Level
Description and characteristics
1. Reaction
Reaction evaluation is how the delegates felt, and their personal reactions to the training or
learning experience, for example:
2. Learning
3. Behavior
4. Results

Did the trainees like and enjoy the training?

Did they consider the training relevant?

Was it a good use of their time?

Did they like the venue, the style, timing, domestics, etc?

Level of participation.

Ease and comfort of experience.

Level of effort required to make the most of the learning.

Perceived practicability and potential for applying the learning.
Learning evaluation is the measurement of the increase in knowledge or intellectual
capability from before to after the learning experience:

Did the trainees learn what was intended to be taught?

Did the trainee experience what was intended for them to experience?

What is the extent of advancement or change in the trainees after the training, in the
direction or area that was intended?
Behavior evaluation is the extent to which the trainees applied the learning and changed
their behavior, which can be seen/measured immediately following and/or several months
after the training, depending on the situation:

Did the trainees put their learning into effect when back on the job?

Were the relevant skills and knowledge used?

Was there noticeable and measurable change in the activity and performance of the
trainees when back in their roles?

Was the change in behavior and new level of knowledge sustained?

Would the trainee be able to transfer their learning to another person?

Is the trainee aware of their change in behavior, knowledge, or skill level?
Results evaluation is the effect on the business or environment resulting from the improved
performance of the trainee
Measures would typically be business or organizational key performance indicators, such as:
Volumes, values, percentages, timescales, return on investment, and other quantifiable
aspects of organizational performance, for instance; numbers of complaints, staff turnover,
attrition, failures, waste, non-compliance, quality ratings, achievement of standards and
accreditations, growth, retention, etc.
It is important that the level of measurement for a program should be linked to the course objectives.
More information can be found on the Kirkpatrick website : http://www.kirkpatrickpartners.com/
4 | Page
Creating a Measurement Plan
Measurement plans should be created early in the training design process and certainly before developing
evaluation tools (e.g. questionnaires etc). This should be your Measurement Plan. This ensures time for any
preliminary data collection as well as gaining agreement on the measurement plan before the training event.
If you are clear on the outcome you are aiming for through the learning intervention then early identify the
measurement needed to show the extent the objectives and hence outcome has been achieved. This enables
you to identify the right tool and craft the right questions based on your planned metrics.
Establishing your Measurement Plan
It is important to document and gain agreement on your measurement strategy prior to data collection. This
ensures that you measurement work meets the needs/expectations of the project owners/sponsors. A sample
Data Collection plan is displayed below.
Business Outcomes: Measuring Return on Investment, Return on Expecations
or Risk Avoidance
Identifying how training provides a good business outcome is always difficult because there are several factors
involved. Simply it requires high quality appropriate training, the learner learning and applying on the job,
while being enabled to apply their new skills in their business through the support of their management and
organisation. Being able to measure this can only be attempted if the business partners are engaged early to
help identify what success will look like and to work cooperatively to design the learning intervention aligned
to the performance processes..
Before training, learning professionals need to partner with supervisors and managers to prepare participants
for training. Even more critical is the role of the supervisor or manager after the training. They are the key
people to reinforce newly learned knowledge and skills through support and accountability
Caution: Measuring the impact of learning can become very complex, resulting in expensive surveys, multiple
tests, and the use of other assessment tools. Be careful not to make ‘measurement’ THE goal, and weigh the
benefits of embarking on complex interventions to prove the learning programmes effectiveness at the cost of
good instructional design. That is, if your measurement costs more than the learning programme itself, you
have cause for concern and you should rethink the measurement plan to ensure that you are putting the
appropriate effort and balance into learning. Think of existing measures and how they are captured and see if
you can easily tap into existing means of measuring business outcomes rather than inventing new ones.
5 | Page
6 | Page