V07.10.12
METRICS & MEASUREMENT
Understanding the components of an effective measurement process, including the identification of key
metrics and the use of data.
Summary of Content Suggestions
 Listening to patients/family - Focus Groups, Mystery Shopping, etc.
 Understanding, managing and coordinating patient survey administration - survey methodologies/vendors.
 Data analysis & validity – segmentation.
 Goal-setting.
 Results analysis, reporting and action planning – providing data in a way that is accessible, usable, and actionable.
 Market analysis.
 Quantitative and qualitative research methods & data analysis.
 Data management.
 Determining ROI.
 Methods for keeping staff informed of Patient Experience scores on ongoing basis.
 Real time data opportunities such as Patient and Family Advisory Councils, Patients on hospital committees, Patient family shadowing, etc.
 Use of Patient and Family Stories to drive change.
Additional Suggestions - Worksheet
Process measurement vs goal measurement
Using Patient and Family Advisory Councils for deeper discussions of areas of concern or opportunity raised in patient satisfaction surveys, ie. Press Ganey surveys.
Use of written comments to understand numeric data
Include action plan implementaton and measurement
Limitations of and improvement of the quality and rigor of measurement of these concepts. In depth interviewing, narrative, phenomenological analysis in addition to surveys
and statistical quantitative methods.
Software available to manage qualitative data.
IRB vs. quality improvement process for management of data
* great!
* I'm not sure that I understand how this is different than an introduction into continuous process improvement cultural elements?
* consider emphasizing that patient experience is the goal, not measurement targets. Perhaps "effectively communicating vision, strategy, and measurements" -measurements just help us "take the pulse", help us identify/diagnose problems, and assist us to gauge efficacy of improvement efforts -- in my experience, patients, families,
and professionals "tune out", disengage, and may miss the point when patient experience survey results are emphasized (i.e. "everyone wants great ratings, who wouldn't", "it's
the same story: ratings, improve"
Compare patient care satisfaction with clinical care outcomes.
Focus on data as well as verbatim patient comments – not all about the scores, how they relate together
Bring all data sources together to make assessments and understand the big picture; how do you package this information so it is useful for the receiver
Relationship between patient experience data and other key performance indicators – one stop shop for all of organizations top priorities
quantitative and qualitative data used in conjunction
peer groups
Patient Experience Body Of Knowledge – DOMAIN DEVELOPMENT SUMMARIES
www.theberylinstitute.org
Copyright 2012 The Beryl Institute. All rights reserved. Federal copyright law prohibits unauthorized reproduction by any means.
1
V07.10.12
local, national, international benchmarking
developing survey tools (validation, etc)
assessing psychometrics of tools during selection process
ROI and metrics are ever-increasing in importance as hospitals invest in new tools to drive outcomes. There are typically MANY factors that impact certain results, so the
measurements and metrics need to consider these pieces. It is also important that we measure the correct data points. For instance measuring the percentage of patients
receiving information on the impacts of smoking, but not measuring whether the patient/family actually read/viewed the materials becomes a measure without value.
Using my galaxy example, this would be Jupiter. It's big. Again, well designed dashboards that are easy to read and interpret. We often make this too wierd. Keep it simple.
Keep it focused. Keep it real. Keep it meaningful. Act on issues that pop out in the data.
If you cannot measure a process, you will never effectiviely manage a process.
DON'T BASE YOUR DECISIONS SOLEY ON THIS COMPONENT
TARP's work on quantifying the cost of negative word of mouth
Using patient comments for patient satisfaction surveys - voice of the patient/family
ensuring 'analysis paralysis' is avoided by spending all of the time drilling down and showing the data 100 different ways insted of fixing the underlying issues.
Patient Experience can be percieved as touchy-feely and an optional investment unless clearly defined measurements are connected to your organization's progress.
Integrated systems of VOC
PE efforts are doomed if you can't show tangible progress. Like any effort to bring about change, if you don't measure it, it won't change.
Identify point at which ROI becomes the cost-of-doing-business
Make an intentional effort to keep it simple. Time and effort put into metrics and measurement is often for not when data is to complex, to old and not real.
New measures for Accountable Care Organizations and other care innovations
Organizational transparency in data (e.g., dashboards)
Return on relationship (physicians, families)
Measurement plans
Experience outcomes: More than Would recommend and Overall satisfaction with experience
The Net Promoter Score (NPS) was developed by Fred Reichheld of Bain & Company, Reichheld, F. The Ultimate Question, Driving Good Profits and Growth. 2006, Boston,
Harvard Business Press.
NPS depends on one simple question for customers: How likely are you, on a scale of 0 (not at all likely) to 10 (extremely likely), to recommend this product or service to a
colleague or friend? “Detractors” answer from 0 to 6, “passives” answer 7 or 8, and “promoters” answer 9 or 10. NPS is calculated by subtracting the 0-8 scores from the 9 and
10 scores. For example, if 50% answer 9-10 (promoters), 40% answer 7-8 (passive), and 10% answer 0-6 (detractors), the NPS would be 0%.
Consumer Oriented Research: The Patient-Centered Outcomes Research Institute (PCORI) is an independent organization created to help people make informed health care
decisions and improve health care delivery. PCORI will commission research that is guided by patients, caregivers and the broader health care community and will produce high
integrity, evidence-based information.
Reports will be critical to measuring and improvements. Qualitative and quantitative results will both be important. A challenge will be to better under the "silent" moment of
truths. SOmetimes end users will tell you something is important but the real critical opportunity o make the experience better is not clearly stated. Reading between the lines
may be as important as the survey results.
I think this section is critical to help integrate a PE effort into larger system goals. I see very little of the things listed here actually used in hospitals. Are people doing ROI on PE?
Absolutely required of this role.......this is central to expert status.
HCAHPS, CG-CAHPS, move survey questions to experience questions like those defined in CAHPS surveys. Add quality outcome metrics.
Provide results weekly to all staff
Publicly report results -- both good and bad
Patient Experience Body Of Knowledge – DOMAIN DEVELOPMENT SUMMARIES
www.theberylinstitute.org
Copyright 2012 The Beryl Institute. All rights reserved. Federal copyright law prohibits unauthorized reproduction by any means.
2
V07.10.12
This is an essential skill set to have - most healthcare staff are trained as scientists and data is very important to establish a baseline and track for improvement. Need in depth
understanding of the metrics including comparative data. Many organizations don't exploit the competitive nature of especially physicians. In addition, ways to provide
provider/staff specific data is powerful.
Measurement of good as well as not so good feedback.
Would add "methods to keep staff and patients informed"...not just staff. I am concerned that throughout each domain, there is an absence of the language of partnership in
having patients/families included in each domain. While good, it is not written in a way to get at the essence of the power of fully engaging and including patients in all we do.
I am wondering if you had patients and families sit as full members on your team to draft this or, if you are assuming leaders int he field know what patients would say? If the
later, we have missed a big opportunity here.
Transparency in sharing employee engagement scores and employee satisfaction scores with all employees.
it would be nice to see someone other than the bedside caregiver collect data
Additional Suggestions – Focus Group






I’m fairly active in health information technology, and we’ve just recently had the meaningful use criteria come out for stage 2. One of the things that happened in that
stage is that we are seeing much more aggressive move towards patient engagement and engaging the patient and their families with the technology. Starting with the
electronic health record but also there’s other technologies. So I think that there’s a need to make sure that we’re incorporating the data and the analytics that can come
from those technologies because it’s important that we get the data out of those technologies and then personalize it for patients. (LED TO NEW DOMAIN)
I think when we’re talking about quantitative and qualitative reporting one thing I think would be interesting and important to add is sort of quantifying the patient
experience and patient advocacy and service recovery, and how that affects the bottom line of an organization. What I mean by that is like I’m the patient advocate, it’s
not a quote-unquote productive position, however we all know that happy patients come back and refer to others and don’t pass on negative feedback so there is a
bottom line that if you have happy customers you have more income from that, and a really good way to quantify that somehow, to show the organization the benefit of
having a person or a department that focuses on that service recovery and that complaint management.
If I could dovetail on that, having been a patient advocate in the past and then been in industry in that role I can tell you that that is one of the areas where I think we can
turn to industry, that industry has models for determining how those services contribute to the ROI, and that as organizations we need to begin to ask industry to help us
figure out how to do that.
Net promoter is an industry-wide metric and methodology looking at customer engagement and loyalty and then translating that into return on investment. But frankly I
think that this is a leadership issue, that if you don’t have leadership _____ that has this in their gut, it’s a real challenge. It’s a challenge for me in that respect but I think
it’s clear. The thing that I do like, I want to say that I like about how this is written up is that it’s looking at data, it’s looking at stories, it’s looking at _______, I like that. I
think that those are all really important and the challenge is putting those together with sort of a combination QI marketing head, which is really where it goes. But I think
we collect way more information than we have the capacity to use and respond to, and I think that’s sort of a pain in the butt for the people we survey.
Somebody just brought up a few minutes ago like gathering so much information and surveying people and annoying them, and I’ll say that we’ve actually had some
feedback from our customers that it’s a very laborious survey process after they leave. It is random, it’s voluntary, but they kind of get sucked into this process and now
they had a good experience with us and we’re ticking them off after they leave or that survey company is when they call and they kind of read through this really long
survey when the people are home and not feeling well necessarily yet and that’s a fine line and a fine balance of gathering that information but making sure that your
method while you’re doing it is not turning this into a negative experience, because this is now their last experience with our hospital.
And that’s why it’s so important to pull as much of that information out as you can from the information systems that the facility already has. So we leverage the
information systems, get as much of the information out of what we’re already collecting while the person is in the hospital, and then you can reduce maybe some of
those follow-up surveys, or shorten them or do something a little bit different. You can learn a lot about people by capturing the information that you already have.
Patient Experience Body Of Knowledge – DOMAIN DEVELOPMENT SUMMARIES
www.theberylinstitute.org
Copyright 2012 The Beryl Institute. All rights reserved. Federal copyright law prohibits unauthorized reproduction by any means.
3
V07.10.12

It seems like everybody wants to tag on something to these surveys, like hey when we’ve got them on the phone let’s get the most bang for our buck, so that’s just one
more question, well what do you guys want to focus on, we’ll ask them that too. And by the time you get done they’ve thought, I’ve listened in, basically they plugged me
in to the calls and I got to monitor the survey calls and it made me a little anxious listening because it was such a painful process. You listened and it was long and they
repeated the same thing over and over again. It was really eye-opening and you’re right, instead of adding as much as we can to get the most bang for our buck we need
to be pulling out as much as we can so that it’s a really easy painless process for them.
ROUND I FEEDBACK
METRICS & MEASUREMENT Key components of an effective measurement process, including the identification of key metrics and the use of data.
INITIAL CONTENT SUGGESTIONS
 Know how to listen to patients
 Data analysis & validity
o Patient Focus Groups
o Segmentation
o Capture experiences online
 Providing data in a way that is usable/actionable
o Mystery shopping techniques
 Market Analysis
 Patient survey administration
 Quantitative Research Methods & Data Analysis
 Survey methodologies/vendors
 Data Management
 Goal-setting
COMMENTS/SUGGESTIONS for DOMAIN CONTENT/TOPICS
 This entire domain is vital. Formal training in these methods key to success. No need to be a statistician but knowledge of what to do with info when its gathered is vital.
 Add Family to and of 1st bullet. The first bullet point: Is it about listening or is it more about knowing how to capture the data appropriately?
 Yes to – focus groups, mystery shopping and no to patient survey admin and below ‐ those items we get from the survey vendors, who are very good about walking their
clients through survey set‐up, methodology, validity, stratifying data, reading reports, identifying priorities.
 Understand the difference between satisfaction surveys and behavior based surveys (could fit under survey methodologies)
 Add ROI?
 Data management. Discuss how organizations can get so deep in data that they forget what to do to improve the data they’re looking at.
 Partnering with your survey vendor - what support should you expect?
 Goal setting: I think it should have its own category. Knowing how to set appropriate goals should be separate from figuring out how to measure. So often we set the
measurement and try to use that as a goal. It fails, because people don’t rally around a 2% or 10% increase in a score. They rally around a mission-related goal. Should
include specifics regarding setting realistic targets and use the SMART model.
 Survey methods – both quantitative and qualitative results use. Methods of collection, electronic, phone, paper, etc.
 Skills to present the results. Interpreting/analyzing quantitative and qualitative data. Measurement should be specific to feedback process, e.g. surveys, focus groups, etc.
Make it understandable for all levels of organization. Case studies on how to turn data into info. Trending results over time, defining format, content of reports, etc.
 As part of the discussion on survey methodologies, there needs to be a short topic on data bias of real-time survey results vs. post-visit survey results.
 Consider adding accessibility of data – could be added to “Providing data in a way that is usable/actionable”
 Use data to support/link efforts. Data integration from the multiple sources described in the content ideas.
 HCAHPS and CG-CAHPS - Public reported domains
 Link Quantitative and Qualitative data when possible.
 Expectations for management: are they running their reports and reviewing the data (feedback) with staff?
 Identify and empower PE champion or patient advocate as point person for data reviewed in PI and other meetings. Would this person be the main vendor contact?
Patient Experience Body Of Knowledge – DOMAIN DEVELOPMENT SUMMARIES
www.theberylinstitute.org
Copyright 2012 The Beryl Institute. All rights reserved. Federal copyright law prohibits unauthorized reproduction by any means.
4
V07.10.12







Include metrics to track effectiveness of service recovery program.
Technology Gap Analysis – understanding limitations of technology, and understanding where technology can assist in improving patient experience.
Determine method for keeping staff informed of Patient Experience scores on ongoing basis.
Patient and Family Advisory Councils. Patients on hospital committees. Patient family shadowing
Patient and Family Stories - Use of narratives to drive change
Social media protocols for monitoring and determining when and/or how to respond to comments. Monitor unsolicited feedback
Patient experience mapping
Patient Experience Body Of Knowledge – DOMAIN DEVELOPMENT SUMMARIES
www.theberylinstitute.org
Copyright 2012 The Beryl Institute. All rights reserved. Federal copyright law prohibits unauthorized reproduction by any means.
5