Evaluating HRD Programs
HUMAN RESOURCE DEVELOPMENT
EVALUATING HRD PROGRAMS
XAVIER INSTITUTE OF SOCIAL SERVICE
RANCHI
Submitted by:
Submitted to:
Group: VI
Personnel Management-II
2010-2012
Prof. Sajeet Lakra
Page 1
Evaluating HRD Programs
GROUP CONSTITUTION
Members
Ashutosh Sharma
Aditi Gupta
Kumar Pratik
Divyendu Shekhar
Utkarsh
Shashank Shekhar Rai
Roll No.
06
15
28
38
52
66
Page 2
Evaluating HRD Programs
ACKNOWLEDGEMENT
Success of a project depends on the endeavor put behind it. Encouragement
and patronization perfect an endeavor.
We take this opportunity to express our heartfelt gratitude to our faculty
Prof. Sajeet Lakra, for providing us with the opportunity of preparing the
study material for the of the HRD curriculum and give presentation in the
class. The group’s involvement and team work helped us understand the
chapter in a lucid manner. The group’s cohesiveness and team work helped us
to make this task a success. We thank all the team members and also all our
colleagues for their unconditional support.
Thank You
Group VI
Page 3
Evaluating HRD Programs
CONTENTS
S.No.
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
Topics
Introduction
Purpose of HRD Evaluation
Principles of Evaluation
Importance of Training Evaluation
Benefits of Evaluation
Process of Training Evaluation
Models and Framework of Evaluation
Data Collection for HRD Evaluation
Linking Training and development with
Organization’s Business Strategy
Case Study
Conclusion
12.
Bibliography
Pg. No.
05
06
07
08
10
11
15
21
22
26
28
29
Page 4
Evaluating HRD Programs
1. INTRODUCTION TO HUMAN RESOURCE
DEVELOPMENT EVALUATION PROCESS
A comprehensive and effective evaluation plan is a critical component of any
successful training program. It should be structured to generate information
of the impact of training on the reaction; on the amount of learning that has
taken place; on the trainees behaviour; and its contribution to the job/
organization. Therefore evaluation of training is a measure of how well it has
meet the need of its human resources.
Any attempt to obtain information (feedback) on the effect of training
program and to assess the value of training in the light of that information for
improving further training.”- Hamblin
“The systematic collection of descriptive and judgmental information
necessary to make effective training decisions related to the selection,
adoption, value, and modification of various instructional activities.”
“Any attempt to obtain information (feedback) on the effect of training
program and to assess the value of training in the light of that information for
improving further training.”
Assessing the effectiveness of the training program in terms of the benefits to
the trainees and the company must look into the following aspects:
– process of collecting outcomes to determine if the training program was
effective
– from whom, what, when, and how information should be collected
Page 5
Evaluating HRD Programs
2. PURPOSE OF EVALUATION
• Determining accomplishment of specific training objectives.
• Determining cost effectiveness of training.
• Explain program failure, if it occurs.
• Enhancement of the credibility of training and development if tangible
benefits to an organization are proved.
• Determine whether the program is meeting the intended objectives
• Identify strengths and weaknesses
• Determine cost-benefit ratio
• Identify who benefited most or least
• Determine future participants
• Provide information for improving HRD programs
• Reinforce major points to be made
• Gather marketing information
• Determine if training program is appropriate
• Establish management database
Page 6
Evaluating HRD Programs
3. PRINCIPLES OF EVALUATION
• Clarity about the goals and purposes of evaluation.
• Evaluation must be continuous.
• Evaluation must be specific.
• Evaluation must be based on objective methods and standards.
• Realistic target dates must be set for each phase of the evaluation
process.
• Evaluation has to be accepted as a process than an end product of
training.
Page 7
Evaluating HRD Programs
4. IMPORTANCE OF TRAINING EVALUATION
An organization that invests in training programs for employees should also
conduct some form of training evaluation. This can be done in a number of
different ways. The purpose of evaluation is to determine whether or not the
training achieved the desired objective. The objective might have been very
precise, or extremely loose, but there would have been one. This is usually
set as a result of assessing trainining needs. Evaluating training will help show
how effective it was at meeting the objective. All organizations, regardless of
size or type, should use some process of assessing the effectiveness of training
undertaken by employees. Education in any form is an investment of time and
money, which makes it important that the organization establishes, as far as it
can, the return on that investment.
Evaluating training effectiveness is important because it sheds light on four
aspects. They include:




How well the training program met the learner’s needs and objectives
What knowledge and skills it has imparted to learners
What desirable change it has brought in the learners’ performance
What organizational benefits it has yielded
However, unfortunately, in today’s organizations, this mechanism of training
evaluation is in short supply. Organizations are unwilling to spend their
resources for a comprehensive after-training evaluation. They end it by
serving a ceremonious feedback form to learners and getting their opinions.
But you must realize that it will not work. It will not make your training
programs effective and yield the results you want.
In this absence of a proactive follow-up, the following undesirable
consequences will have to be faced:
Lack of application of skills by learners:
You had analyzed the performance gaps in your employees and devised a
suitable training program. At the end of the training program, your
participants said that the training program was quite useful and they had learnt
Page 8
Evaluating HRD Programs
many useful skills. You are happy about that. But at the end of your annual
auditing, you are horrified to find no progress in the performance in spite of
the training. And your employees have not used the skills you intend them to
after the training. This is quite possible. You can avert it by putting in place a
proper evaluation mechanism.
Inability to measure ROI:
Financial assessment of a training program is crucial in that it provides useful
insights into the Returns on Investment (ROI). This assessment depends on
evaluating how well the employees implemented the skills they have learnt in
the training program.
Inability to make future training programs contextual:
In today’s dynamic marketplace, the need for consistent skill up gradation
never loses its importance. What was said to be an effective training program
in the past may not be so now. You need to continuously adapt it to suit your
current needs. This is possible when you know how effective your previous
training program was.
There are methods like Kirkpatrick’s Training Evaluation Model, Bloom’s
Taxonomy of Learning Domains and Keller’s ARCS Model. Use the models
suggested here depending on the available resources, size of the training
program and your corporate culture. When you have complex and critical
training programs, you need to choose elaborate tools and have a wider
approach to evaluate each program at various stages. Only then can the
evaluation become effective and accurate.
Page 9
Evaluating HRD Programs
5. BENEFITS OF TRAINING EVALUATION
 Improved quality of training activities.
 Helps employees to monitor their own improvement.
 Builds morale, by demonstrating an interest in staff development.
 To maximize the training ROI (return on investment).
 Helps to determine the form of future training programs.
 Assists with identifying the effectiveness of different forms of teaching
(such as classroom based or web based).
 Improved ability of the trainers to relate inputs to outputs
 Better discrimination of training activities between those that are
worthy of support and those that should be dropped
 Better integration of training offered and on-the job development
 Better co-operation between trainers and line-managers in the
development of staff
 Evidence of the contribution that training and development are making
to the organization
 Reduction in preventable accidents measured
 Reduction in scrap/rework measured in cost of labor and materials
Page 10
Evaluating HRD Programs
6. PROCESS OF TRAINING EVALUATION
There are two considerations involved in evaluation:
• Who is responsible for the validation and evaluation processes?
• What resources of time, people and money are available for
validation/evaluation purposes?
Thus the process of evaluation involves 3 stages:
• Before training
• During training
• After training
BEFORE TRAINING:
The learner's skills and knowledge are assessed before the training
program. During the start of training, candidates generally perceive it as a
waste of resources because at most of the times candidates are unaware of the
objectives and learning outcomes of the program. Once aware, they are asked
to give their opinions on the methods used and whether those methods
conform to the candidate’s preferences and learning style.
DURING TRAINING:
It is the phase at which instruction is started. This phase usually consists of
short tests at regular intervals.
AFTER TRAINING:
It is the phase when learner’s skills and knowledge are assessed again to
measure the effectiveness of the training. This phase is designed to determine
whether training has had the desired effect at individual department and
organizational levels. There are various evaluation techniques for this phase.
Page 11
Evaluating HRD Programs
The Training Evaluation should involve:
 senior management
 the trainer
 line management
 the training manager
 the trainee
SENIOR MANAGEMENT:
• Awareness of need & value of training
•
Involving Training Manager in senior management meetings
•
Knowledge & support of training plans
•
Active participation in events
•
Evaluation to be done & regular summary reports
•
Policy & strategic decisions based on results & return on investment
data.
THE TRAINER:
• Provision of any necessary pre-program work
• Identifying knowledge & skills level
• Training & learning resources to enable the learners to learn within
objectives of program
• Monitoring the learning as the program progresses
• Assessment of & receipt of reports.
Page 12
Evaluating HRD Programs
THE LINE MANAGER:
• Work-needs & people identification
• Involvement in training programme & evaluation development
• Pre-event preparation & holding briefing meetings
• Giving ongoing & practical support to the training programme
• Reviewing the progress of learning implementation
• Final review of implementation success and assessment.
THE TRAINING MANAGER:
• Management of the training department & agreeing to the training
needs
• Maintenance of interest & support in the planning & implementation of
the programs including practical involvement where required
• The introduction & maintenance of evaluation systems
• Production of regular reports for senior management
• Contact with senior management
• Liaison with the learners' line managers and arrangement of learning
implementation.
Page 13
Evaluating HRD Programs
THE TRAINEE:
• Plan & design the training programme
• Contribute towards the evaluation process
•
Take interest & performing an active part in the training programme or
activity
•
Complete a personal action plan during & at the end of the training
•
Support the evaluation processes.
Page 14
Evaluating HRD Programs
7. MODELS AND FRAMEWORK OF EVALUATION
• There are six frameworks for evaluation
• Kirkpatrick
• CIPP
• Brinkerhoff
• Kraiger, Ford & Salas
• Holton
• Philips
Two approaches are mainly used:
 Goal-based ApproachThe Kirkpatrick Model
 System-based Approach:
(i)CIPP Model
(ii) TVS Approach
(iii) IPO Approach
Page 15
Evaluating HRD Programs
Kirkpatrick (1959)
CIPP Model (1987)
IPO Model (1990)
TVS Model
(1994)
1. Reaction: to gather data on 1. Context: obtaining information 1. Input: evaluation of system
participants reactions at the end about the situation to decide on performance indicators such
of a training program
educational needs and to
as trainee qualifications,
establish program objectives
availability of materials,
appropriateness of training,
etc.
1. Situation:
collecting pretraining data to
ascertain current
levels of performance
within the
organization and
defining a desirable
level of future
performance
2. Learning: to assess whether 2. Input: identifying educational 2. Process: embraces
the learning objectives for the strategies most likely to achieve planning, design,
program are met
the desired result
development, and delivery of
training programs
2. Intervention:
identifying the reason
for the existence of
the gap between the
present and desirable
performance to find
out if training is the
solution to the
problem
3. Behavior: to assess whether 3. Process: assessing the
job performance changes as a implementation of the
result of training
educational program
3. Impact: evaluating
the difference
between the pre- and
post-training data
4. Results: to assess costs vs.
benefits of training programs,
i.e., organizational impact in
terms of reduced costs,
improved quality of work,
increased quantity of work, etc.
3. Output: Gathering data
resulting from the training
interventions
4. Product: gathering information 4. Outcomes: longer-term
regarding the results of the
results associated with
educational intervention to
improvement in the
interpret its worth and merit
corporation’s bottom line- its
profitability, competitiveness,
etc.
4. Value: measuring
differences in quality,
productivity, service,
or sales, all of which
can be expressed in
terms of dollars
Page 16
Evaluating HRD Programs
GOAL-BASED
SYSTEM-BASED
These models do not define the steps
necessary to achieve purposes and do not
address the ways of utilizing results to
improve training.
CIPP, IPO, and TVS) seem to be more
useful in terms of thinking about the
overall context and situation but they
may not provide sufficient granularity.
The difficulty for practitioners following
such models is in selecting and
implementing appropriate evaluation
methods (quantitative, qualitative, or
mixed).
Systems-based models may not
represent the dynamic interactions
between the design and the evaluation
Because of their apparent simplicity, trainers
jump feet first into using such models
without taking the time to assess their needs
and resources or to determine how they’ll
apply the model.
Few of these models provide detailed
descriptions of the processes involved
in each steps. None provide tools for
evaluation.
Many organizations do not use the entire
model, and training ends up being evaluated
only at the reaction, or at best, at the
These models do not address the
collaborative process of evaluation, that
is, the different roles and
responsibilities that people may play
during an evaluation process
learning level
.
of training
.
Page 17
Evaluating HRD Programs
THE KIRKPATRICK MODEL:
Page 18
Evaluating HRD Programs
Level 1 : Reaction
Reaction evaluation is how the delegates felt about the training or learning
experience.
Evaluation tools and methods
• 'Happy sheets', feedback forms.
• Verbal reaction, post-training surveys or questionnaires
Relevance and practicability
• Quick and very easy to obtain.
• Not expensive to gather or to analyse
Level 2 : Learning
Learning evaluation is the measurement of the increase in knowledge before
and after.
Evaluation tools and methods
• Typically assessments or tests before and after the training
• Interview or observation can also be used
Relevance and practicability
• Relatively simple to set up; clear-cut for quantifiable skills
• Less easy for complex learning
Level 3 : Behavior
Behaviour evaluation is the extent of applied learning back on the job implementation.
Page 19
Evaluating HRD Programs
Evaluation tools and methods
• Observation and interview over time are required to assess change,
relevance of change, and sustainability of change
Relevance and practicability
• Measurement of behaviour change typically requires cooperation and
skill of line-managers
Level 4 : Results
Results evaluation is the effect on the business or environment by the trainee.
Evaluation tools and methods
• Measures are already in place via normal management systems and
reporting - the challenge is to relate to the trainee
Relevance and practicability
• Individually not difficult; unlike whole organisation
Process must attribute clear accountabilities
Page 20
Evaluating HRD Programs
8. DATA COLLECTION FOR HRD EVALUATION
There are several techniques of data collection. They are as follows:
 Observation
 Questionnaire
 Interview
 Written Test
 Simulation/ Performance test
Advantages and Disadvantages of Techniques :
METHODS
1. Interview
ADVANTAGES
• Flexible
• Opportunity for
clarification
• Depth Possible
• Personal Contact
DISADVANTAGES
• High Reactive
effects
• High cost
• Face-to-Face
Threat Potential
• Trained
Observers
Needed
•
2.Questionnaire
• Low Cost
• Anonymity Possible
• Respondent sets
pace
• Variety of Options
• Possibly
Inaccurate Data
• Return Rate
Beyond Control
Observation
• Non- Threatening
• Excellent way to
• Reactive Effect
• Probable
Page 21
Evaluating HRD Programs
Measure Behaviour
Change
Written Test
• Low Purchase Cost
• Readily Scored and
Quickly Processed
• Easily Administered
• Wide Sampling
Possible
Simulation/
Performance Test
• Reliable
• Objective
• Close Relation to
Job Performance
• Trained
Observers
Needed
• Possible low
Relation to Job
Performance
• Reliance on
Norms may
Distort Individual
Performance
• Cultural Bias
• Time consuming
• Simulation often
Difficult and
Costly
Page 22
Evaluating HRD Programs
9. LINKING TRAINING AND DEVELOPMENT WITH
BUSINESS STRATEGY
• Training and Development strategy is a comprehensive master plan
which ensures that investment in learning is assessed and evaluated.
• Business Strategy is the determination of the long term goals and
objectives of an organization, and allocation of resources necessary for
carrying out these goals.
• Training functions lie at the heart of how businesses access and develop
human resources required to maintain a sustainable competitive
advantage over rivals.
• Training should result in creating skills that are appropriate for
achieving the strategic objectives of a business.
The Factors governing the starting point for formulating an effective training
and development strategy are as follows:
• Understand the industry and the business’ unique offer and what gives
it competitive advantage.
• Consider the changes predicted in the environment- the rate of
growth/decline, the competition and degree of technological change.
• Assess the need to change and adapt to economic circumstances.
• Assess the level of knowledge and professional capability of staff now
and that needed in the future.
• Understand the importance of customer service and reflect the
community served- what sort of customers are served and the nature of
their expectations.
Training and development strategy should also align with the cultural beliefs
about what learning may achieve and the way it is regarded. Assumptions
include:
Page 23
Evaluating HRD Programs
• All work is a process of continual learning and improvement. This can
be the starting point for creating a ‘learning organization’.
• Learning should be encouraged only where there is measurable benefit
to the organization.
• Learning should be targeted on key performance skills.
• Learning may cover other aims than just a narrow definition of business
benefit- it can support the organization’s duty to the community or
wider social responsibility.
• Learning is a cost to the business and should be delivered effectively and
efficiently.
• Learning and talent development is an investment for the future.
• Personal career development is critical for retention and engagement
and should ne encouraged.
According to Rothwell & Kazanas the ten Approaches to link T&D strategy
with Business Strategy are:
1. The Top Down Approach: The Organizational strategy determines
and dictates the HRD offerings.
2. The Market Driven Approach: The HRD function or department
identifies future learning on the basis of future market conditions
confronting the organization.
3. The Career Planning Approach: The HRD effort helps individuals
prepare for the future against the backdrop of organization’s strategic
plan.
4. The Futuring Approach: The HRD function formally and directly
assists top managers as they formulate the organizational strategy.
5. The Artificial Experience Approach: The HRD function simulates
the conditions the organization may face in the future.
Page 24
Evaluating HRD Programs
6. The Pulse Taking Approach: Since HRD function is at the
forefront of communicating, assessing needs, design and develop
instructions and evaluate results-they maintain a high profile. This
unique position in organization’s communication chain is utilized to the
best advantage in this approach. HRD people take pulse of organization
regarding implementation of business strategy.
7. The Performance Diagnosis Approach: While assessing the
training needs, HRD professionals often uncover performance
problems or opportunities of strategic importance. They relay that
information to the top management for their use in strategic planning.
8. The Education Approach: HRD practitioners try to link HRD with
the organizational strategy by teaching people how to think strategically
as they do their work. By applying strategic planning model people can
anticipate the future problems and opportunities.
9. The Inter Personal Approach: It is informal and may involve
socializing with strategists to know what they are seeking.
10. The Rifle Approach: It takes its name from its laser like focus on
achieving pinpointed results. The HRD practitioners aim their efforts to
the areas of greatest need or the business’ most pressing problem.
Page 25
Evaluating HRD Programs
10. TRAINING EVALUATION IN BRITANNIA
For each training programme conducted in Britannia, a training brochure is
developed.
The brochure developed consists of the following information:
• Programme objectives: Need of the training and what are the objectives
that this training aims to achieve, what likely outcomes are expected to
come out of impact of this training.
• Programme content: Topics being covered during the training.
• Methodology adopted.
• Programme faculty.
• Personnel to be covered.
• Training methodology.
• Training effectiveness criteria and scale.
• The training effectiveness is measured by measurement of the
achievement of the objectives.
•
A person gets nominated for a training programme which flows from
the training needs of that individual
• The programs are divided into three broad categories:
 Functional
 Behavioural
 General/ Omnibus programmes
Page 26
Evaluating HRD Programs
Functional programs:
The outcome of the training is measured by comparing the data pre-training
and post-training. A scale is developed for measuring the effectiveness of
training based on the % achievement of the objectives.
Behavioural programs:
The effectiveness of the training of this nature is measured annually. This is
seen through the training need identification for the coming year for the
employee. If the training need is repeated there, then the training provided is
taken as ineffective. If the training need is repeated but with focus on a part of
the need, then the training is partially effective. If not repeated, then the
training is effective.
General programs:
These are the training needs flowing directly from the organizational needs.
Examples of these can be ISO 9000 training, ISO 14000 training and any
awareness training. These are omnibus training programmes, which are run
for a large number of employees.
The effectiveness of the training is measured by:
 Achievement of those organizational objectives within the time lines.
 Number of audit issues raised on the areas covered in the training.
 Any other such thing as defined in the training brochure
Page 27
Evaluating HRD Programs
11. CONCLUSION
Evaluation of HRD Program helps in gauging the degree to which a training
(or other HRD program) achieves its intended purpose.
In other words it measures the effectiveness of the HRD programThe
importance of training programs must be understood by HRD professionals.
Not every program needs to be evaluated to the same extent. The key is to
have a well planned evaluation strategy that sets the stage for how and to what
extent each program will be evaluated.
The value of the training evaluation forms is that they give immediate
feedback and they are inexpensive to set up and process. However, it is
possible for someone to leave a course having enjoyed it, and believing they
learned a great deal. It is only when they come to apply this knowledge that
they discover whether the training was as effective as they thought.
A more thorough evaluation can involve a subsequent round of training
evaluation forms, or an assessment interview, which are conducted a few days
or weeks after the training. This allows the learner time to reflect on what
they have learned and to have put at least some of it into practice. Their
opinion of the training may have changed based on their experience in the
workplace.
Methods for the effective evaluation of training have been studied for some
time. The Kirkpatrick model is now considered to be the industry standard,
and is frequently used by HR departments when determining their own
systems.
Organizations that train but do not evaluate that training cannot be certain of
its value, either to themselves or their employees.
Page 28
Evaluating HRD Programs
12. BIBLIOGRAPHY
•
Human Resource Development, Jon M. Werner & Randy L. Desimone, Pg. 151-201,Eighth
Edition, Cengage Learning India Pvt. Ltd.
•
Human Resource Development- Wikipedia, The Free encyclopedia
http://en.wikipedia.org/wiki/Human_resource_development
[Accessed on 27th November, 2011]
•
http://managementhelp.org/trng_dev/evaluate/evaluate.htm
[Accessed on 27th November, 2011]
•
http://www.ifets.info/journals/5_2/eseryel.html
[Accessed on 25th November, 2011]
Page 29