Welcome to the Indianapolis SUN!
The ABC’s of ROI
(Return on Investment)
John Gillespie
Laerdal Medical
Traditional ROI is:
“…performance measure used to evaluate the
efficiency of
an investment or to compare the efficiency of a
number of
different investments.” (“Investopedia,” n.d.)
Investopedia. (n.d.). Return on investment - ROI. Retrieved from http://www.investopedia.com/terms/r/returnoninvestment.asp
Traditional ROI is:
“…performance measure used to evaluate the efficiency of
an investment or to compare the efficiency of a number of
different investments.” (“Investopedia,” n.d.)
“…if an investment does not have a positive ROI, or if
there are other opportunities with a higher ROI, then the
investment should be not be undertaken.” (“Investopedia,”
n.d.)
Investopedia. (n.d.). Return on investment - ROI. Retrieved from http://www.investopedia.com/terms/r/returnoninvestment.asp
Why is ROI important???
• Sustainability of your simulation program
• Help insure that the training dollars are
providing value to the organization
• Help insure that the learner is receiving
value for time spent in training
Was/is this a:
• Thoughtful investment?
Or
• Thoughtless investment?
Solution Matrix. (2011). Return on investment. Retrieved from http://www.solutionmatrix.com/return-on-investment.html
Remember this:
“…hospitals typically measure ROI from a business
perspective—cost, revenues or operating efficiencies
—but many benefits of clinical applications fall into
quality and safety realms that do not easily translate
into dollars.” (Page, 2010)
Page, D. (2010). IT’s return on investments is tricky to pin down. Hospital & Health Networks. Retrieved from http://www.hhnmag.com/hhnmag_app
/articledisplay.jsp?dcrpath=HHNMAG/Article/data/06JUN2010/1006HHN_Fea_MostWired&domain=HHNMAG
Four areas of focus:
• Organizational Buy In
• Faculty/Personnel
Development
• Overcoming Barriers to
Simulation
• Curriculum Integration
Organizational Buy In
• ROI (Return on Investment)
• ROE (Return on Expectation)
• Utilization Data Collection
• Evaluation Data Collection
– Leading indicators are usually pretty close to accurate
• Policy and Procedures that are specific to your organization
• Care about what keeps your organizational leaders up at
night
• Create a value statement
– Insure that the stakeholder gets a regularly scheduled
report
But what about ROE?
Main principles
• “Business/[organizational] objectives are seen as a starting point.”
(“Trainingcheck.com,” n.d.)
• “’Return on Expectation’ (ROE) is key” (“Trainingcheck.com,” n.d.)
• “Collective efforts are needed throughout an [organization] to achieve
success.” (“Trainingcheck.com,” n.d.)
Trainingcheck.com. (n.d.). Kirkpatrick ROE model of training evaluation. Retrieved from http://trainingcheck.com/training-evaluation
/kirkpatrick-ROE-model-of-training-evaluation/
Return on Expectation
• “Proactive, business partnership approach that unified teams.”
• “Defines training as a contributor to key business results.”
• “Value defined by business stakeholders in cooperation with training.”
• “Focus on comprehensive evidence and a compelling story of value.”
• “Easy to understand, flexible, and cost-effective.”
Kirpatrick, JD, Kirkpatrick, WK. (2010). ROE’s rising star: Why return on expectations is getting so much attention. Training + Development. Retriev
http://www.astd.org/TD/Archives/2010/Aug/Free/1008_ROEs_Rising_Star.htm
Data Collection
• To prove either ROI or ROE, you must have data to base your return
on.
Types of data collection to support ROI & ROE
• ROI in Healthcare- Using data to demonstrate a positive impact on the
organization through the use of simulation
– This can include unexpected results
• ROI in operations- Use, or utilization
– Tool for determining when you are under utilizing/over utilizing
your resources
• ROI in customer service- Were our customers satisfied with the
simulation experience?
– Can become a tool for measuring the effectiveness of your
instructors/simulations
• ROI in academic learning- Assessment
– Process driven checklist type of scoring
Measuring Outcomes
• The effectiveness of your program is only as
relevant as the effectiveness of the tool(s) that you
are using to measure it by.
• Using the Kirkpatrick model, as one example, you
can and should measure the effectiveness of the
learning objectives met right after the simulation
and then again 6 months later.
• Is the behavior reflecting what was learned during
simulation (Debriefing)? Report this back to the
stakeholder. Are you indicating a beneficial return
on investment?
Monthly Log
WakeMed-Amar Patel
Monthly Log
WakeMed-Amar Patel
Monthly Log
WakeMed-Amar Patel
Monthly Log
WakeMed-Amar Patel
Month End Data
WakeMed-Amar Patel
First Quarter - Raw
WakeMed-Amar Patel
First Quarter - Graphed
WakeMed-Amar Patel
Targets
WakeMed-Amar Patel
How does this data help?
WakeMed-Amar Patel
An actual data collection tool……
Back to the ROI…..
How is simulation use valuable?
• Value 1- Simulation is only valuable if you use it.
– A simulator that is unused/under utilized has no
value.
• Worse, it is a cost center and a financial
drain on the organization.
How is simulation use valuable?
• Value 2.- Educationally effective simulation not
only allows the learners to “connect the dots”, but
it also can change behaviors in an organization.
– The result can allow you to focus your training
dollars to effect
– Reduce your malpractice costs
– Reduce time spent in training
– Improve patient outcomes
How is simulation use valuable?
• Value 3.- Use as a recruiting tool to draw in
candidates that increase the overall value of your
organization.
How is simulation use valuable?
• Value 4.- Repeatability regardless of the patient
census at any given moment.
– Allows for a consistent product to be delivered
to the learners.
References
Best Practice Committee. (2002). The Value of IT investments: It’s not just return on investments. Retrieved from http://www.cio.gov/documents
/thevalueof_it_investments.pdf
Big Dog & Little Dog’s Performance Juxtaposition. (n.d.). Kirkpatrick’s four-level training evaluation model. Retrieved from http://www.nwlink.com/~donclark/hrd
/isd/kirkpatrick.html
Cooke, M., Irby, D., Sullivan, W., & Ludmerer, K. (2006). American medical education 100 years after the Flexner report. The New England Journal of Medicine,
355(13), 1339-1344.
Investopedia. (n.d.). Return on investment - ROI. Retrieved from http://www.investopedia.com/terms/r/returnoninvestment.asp
Kirpatrick, JD, Kirkpatrick, WK. (2010). ROE’s rising star: Why return on expectations is getting so much attention. Training + Development. Retrieved from
http://www.astd.org/TD/Archives/2010/Aug/Free/1008_ROEs_Rising_Star.htm
Markow, M. (2011). Business terms glossary. BusinessWings. Retrieved from http://www.businesswings.co.uk/articles/Business-terms-glossary
Page, D. (2010). IT’s return on investments is tricky to pin down. Hospital & Health Networks. Retrieved from http://www.hhnmag.com/hhnmag_app/jsp
/articledisplay.jsp?dcrpath=HHNMAG/Article/data/06JUN2010/1006HHN_Fea_MostWired&domain=HHNMAG
Robert Wood Johnson Foundation. (2011). Glossary of healthcare terms. Quality/Equality. Retrieved from http://www.rwjf.org/qualityequality/glossary.jsp
SBA.gov. (2011). Essential elements of a good business plan. Retrieved from http://www.sba.gov/category/navigation-structure/starting-managing-business
/starting-business/writing-business-plan/essential-elements-good-busines
Smith, R. (2009). Embracing game technology for medical education. Retrieved from http://www.peostri.army.mil/CTO/FILES 2009_RSmith_MT3.pdf
Solution Matrix. (2011). Return on investment. Retrieved from http://www.solutionmatrix.com/return-on-investment.html
Trainingcheck.com. (n.d.). Kirkpatrick ROE model of training evaluation. Retrieved from http://trainingcheck.com/training-evaluation
/kirkpatrick-ROE-model-of-training-evaluation/
Research in Simulation: Using Data to Help Improve a Return on Investment Amar P. Patel, MS, NREMT-P, CFC Laerdal SUN Conference Foxwoods 2012