Return on Investment: Training and Development Session 1 ROI and Evaluation Objectives • At the completion of this unit, students will be able to: > > > Discuss the importance of determining ROI in training environments. Identify stakeholders and develop questions to determine their views and needs. Write training goals and observable objectives based on stakeholders’ needs. Objectives > > > > > Describe the importance of the assessment of learning for ROI. Recognize the differences between short-term and long-term assessment. Identify items that belong on a training budget sheet to determine ROI. Calculate expenses and ROI. Write a report on training that includes all relevant data. Return on Investment • Rooted in manufacturing. • Advanced to banking, health care, nonprofit, public and education sectors. • Part of quality and efficiency methodologies. ROI Is Used To: > Quantify the effectiveness of training. > Manage the training budget. > Provide evidence to management > > > > > and other stakeholders. Build trust and respect for ourselves and our unit. Earn the ears of senior management. Identify areas for improvement. Provide data requested by senior management. Keep our jobs. Models • Benefit/Cost Ratio Program Benefits BCR Program Costs • ROI (%) Benefit Cost ROI (%) x100 Cost Why is ROI important to you? Evaluation Levels 1. Reaction and Planned Action 2. Learning 3. Application and Implementation 4. Business Impact 5. Return on Investment Stakeholders • Who are they? > Anyone who pays for, participates in, benefits from, or has decision-making responsibility for HRD and training. • What should they know? > The need for training. > The results from training. > The costs of training. Needs Assessment and Task Analysis • Needs assessment: identifies gaps between what is and what should be in the organization. • Task analysis: investigates the specific skill(s), knowledge or attitudes and at which point there is a breakdown in use or performance. If the analysis uncovers a lack of knowledge, skills or attitude, then training is required. If it reveals faulty equipment, poor work conditions or lack of incentive, then another solution is needed. Return on Investment: Training and Development Session 2 Goals, Objectives, Assessment Goals • Organizational Goals These goals are typically targeted toward the organizational level and include such things as productivity, net income, inventory and cash cycles, and customer satisfaction. • Learning Goals These goals typically focus on the individuals in the organization and their performance, knowledge and skills. Writing Objectives Easy as A, B, C, D Audience: Who? Behavior: What do “they” do? Condition: What is the setting and method of evaluation? Degree: Measurement to be met. Example Objective At the completion of the course the learner will be able to score 85% or better on a written multiple choice 25 question test Condition Behavior Degree Audience Example Verbs • Verbs to Use: • Verbs to Avoid > Discuss > Understand > Explain > Comprehend > Demonstrate > Know > Identify > Do > List > Perform > Compare/Contrast > Score (on an assessment) Objectives in ROI • Course objectives: What is it that the learners will be able to do upon course completion? • Application objectives: What goals do the HRD/training practitioner have for the application of the skills gained in training? • Impact objectives: What effect will accomplishing the application objectives have on the organization? • Course objective: Learners will be able to make 15 entries in a customer database in 15 minutes with no more than 1 error. • Application objective: Learners will be able to reduce the data entry error rate by 50 percent over the next 6 months. • Impact objective: Employee time spent correcting database errors is reduced by 25 percent from last year’s rate. Increasing Scope Example Objectives Learning Assessment • Alignment between assessment and objective is paramount. > Easy to do with well-written objectives. • Cognitive, psychomotor and affective domains are assessed in different ways. > Cognitive: written and oral tests; discussion; compare/contrast exercises. > Psychomotor: demonstration; performance. > Affective: longitudinal observation and/or discussion. • Be sure to use the right type of assessment for each objective of the learning event. Return on Investment: Training and Development Session 3 Costs, Budgets, Accounting Costs, Budgets, Accounting • Quantifying ROI means accounting for all the costs of the program. > Fixed costs: independent of the number of participants. > Variable costs: Dependent on the number of participants. • There are costs at every step – make sure to account for them all. Output Measures Outputs are product-driven and include data that are not performance-based but rather outcomes of changes in performance or investment in equipment. Examples include: > Units built. > Time it takes to build the units. > Income from sale of units. Benefits and Soft Skills • Change in: > Attitude, work climate, leadership, teamwork. • We desire these changes because they ultimately effect productivity. > Allow time for change in attitude or behavior, then measure these changes and report qualitatively. > Allow time for change in productivity, then measure for data and report quantitatively. Models • Benefit/Cost Ratio Program Benefits BCR Program Costs • ROI (%) Benefit Cost ROI (%) x100 Cost • Phillips – See Figure 2-2 in text. Karen Kaminski Tobin P. Lopes Benefit/Cost Ratio Example • Data entry clerks’ average wage: $9.50/hr. • Five hours per week were spent correcting errors before training. • 20 percent less time correcting errors saves one hour each week. • 40 clerks. BCR = .076 for one week. What about 13 weeks? 26 weeks? 9.50x40x1 hr BCR .076 $5,000 Now with ROI% • Data entry clerks’ average wage: $9.50/hr. • 20 percent less time correcting errors saves one hour each week. • 40 clerks. • When clerks use their skills for 26 weeks, ROI% is almost 100%!! 380 x1 5000 4620 ROI (%) x100 x100 92.4% 5000 5000 (380 x 26) 5000 4880 ROI (%) x100 x100 97.6% 5000 5000 Create a Data Collection Plan • What? • New information that needs to be recorded? • Who? • When? • How? Additional Thoughts • Access to learners after training: > Do you have access to learners to follow up on application and use of the knowledge and skills developed during the training? • Isolating training effects: > Consider what else may have occurred at the same time as the training that may have made a difference. Trend Line Example Trend Line 35 Y axis: Number of Customer Complaints 30 Projected Value Before Training 20 25 15 Training Intervention 10 New Projected Value After Training J F M A M J J X axis: Months A S O N D Return on Investment: Training and Development Session 4 Reporting Your Findings Review of ROI > Justify the training budget. > Quantify effectiveness of training. > Provide evidence to management and > > > > > other stakeholders. Build trust and respect for ourselves and our unit. Earn the ears of senior management. Identify areas for improvement. Respond to management directive. Keep our jobs. Stakeholders: Review • • • • • • • Customer Employee Supervisor Subordinate President Board of Directors Stockholders Reporting • Needs to be: > Timely, targeted and unbiased. • The purpose is to: > Illustrate success. > Secure approval. > Gain acknowledgement/agreement on solutions. > Build credibility. > Enhance marketing. Reporting • Who needs to know? • What do they need to know? > How will you inform them? • Presentation. • Briefing paper. • Detailed report. • Letter. • When and where? • Who will present? • Feedback? Items to Include in a Report • Executive summary. • Title of training. > Goals and objectives. > Who participated, when/where, length. • Assessment and outcomes. • Evaluation. • Budget. • Recommendations. Summary • Know your audience: > Who, existing and desired skills, knowledge and attitude (audience analysis). • Expectations of/from organization: > Goals and objectives, stakeholders. • Assessment and evaluation: > Budget, data collection. • Reporting. Finally Take action on the data collected. Use it to enhance your training and therefore your organization.