Summary1 of Performance Measurement Workgroup
Conference Call on
Monday, December 12, 2005 3:00 pm-4:30 pm EST
Participants
Kevin Weiss, ACP, (chair)
Bruce Bagley, ACP
Michael Barr, ACP
Carmella Bocchino/ Rebecca Hayes/ Christina Dabkowski, AHIP
John Bott, Employer Health Care Alliance
Niall Brennan, MedPAC
Cindy Brown, ACS
Khatereh Calleja, ASPS
Kathy Coltin, Measures to Market
Jennifer Curtis, WellPoint
Trudie Cushing, ACR
Lou Diamond, Medstat
Andrea Gelzer, Cigna
Anders Gilberg, AMA
Claudia Hart, APA
Trent Haywood, CMS
David Hopkins, PBGH
Sheldon Horowitz, ABMS
Robin Hudson, AUA
Chris Izui, BCBSA
Michelle Johnson, MGMA
Karen Kmetik, AMA
Stacey Jones, AAOHNS
Sandra Lewis, ACCP
Carol McCall, Humana
Sharon McGill, AOA
Crystal Mottur Pilson, ACP
Michael Mustille, Kaiser Permanente
Eileen O’Conner, Health Net
Jennifer Padberg, ASTRO
Greg Pawlson, NCQA
Virginia Proestakes, GE
Mark Rattray, United Health Care
Debbie Robin, AGA
Peter Sawires, PRCH
David Scrase, PHP
Gerry Shea, AFL-CIO
Jonathan Scheff, Health Net
1
Please note that to better organize this summary; some of the comments have been taken out of
chronological order.
Samantha Sheridan, Westat
Richard Snow, AOA
Maureen Stanley, AAO
Donald Storey, Aetna
Howard Underwood, Aetna
Tom Valuck, CMS
Nancy Wilson, AHRQ
Timothy Zeddies, Independence Blue Cross
Presenters on Patient Experience of Care Survey
Julie Brown, RAND
Chuck Darby, AHRQ
Marc Elliot, RAND
Dana Safran, New England Medical Center
Introduction
Kevin Weiss welcomed participants to the call and reviewed the objective of the call – to
discuss the revised efficiency principles document and the patient experience of care
survey developed by AHRQ.
Discussion of Principles of Efficiency Document
The following revisions were made to the principles document
 In definition c, strike “and appropriate.”
 In principle number 1, replace “consistent with” to “according to.”
 In principle 2, strike the word “both” in the first line and strike “so as to evaluate
performance relative to the appropriate peer or reference group.”
 Number 3, change “things” to “elements.”
 Michael Barr, Carol McCall, Lou Diamond and Greg Pawlson were tasked with
adding underuse language to principle number 4.
Discussion of Patient Experience of Care Survey, Led by Chuck Darby, AHRQ
Chuck Darby and others provided background and details on the Consumer Assessment
of Health Plans Survey (CAHPS) Clinician Group Survey.
Chuck Darby, AHRQ
 The survey has gone through extensive field testing and AHRQ is currently analyzing
a final round of field testing data. The data analysis will continue through February
2006.
 The Advisory Board for the survey is working on overarching issues, such as burden,
costs to implement, data collection methodologies, and inclusion of the most
appropriate quality improvement items.
 The final report is being readied for submission to the National Quality Forum in
March 2006.
Julie Brown, RAND
 Julie Brown gave an overview of the questionnaire’s content.
 The survey has been field tested with different supplemental questions, along with
core questions.
 It has also undergone cognitive testing. It is at approximately 6th grade reading level
to ensure ease of use by consumers.
Dana Safran, New England Medical Center
 Dana Safran gave a technical overview of the survey’s reliability testing.
 Sample size: data shows high reliability (0.7) in the 40-45 completed surveys range.
 The survey is undergoing analysis to determine if the reliability changes by going
from 6-point scales to 4-point scales on some questions (i.e., strongly agree – strongly
disagree).
Marc Elliot, RAND
 Further analyses will include psychometric analyses, case mix/ patient mix, mode
effects, and non-response analysis.
Question and Answer Discussion on CAHPS Clinician Group Survey
 The survey includes 26 core items. The survey has been tested with various
supplemental items that would bring the total to 30-35.
 The main focus of the survey is primary care, including items for pediatrics and
specialty care. Field testing sites2 have used it in other ways, including:
o PBGH, in medical, surgical and OB/GYN areas.
o Large specialty groups in Massachusetts.
o American Board of Medical Specialties is pilot testing the survey into
some maintenance of certification modules.
 Methods of data collection tested include by mail, phone, or distribution in the
physician’s office.
o Some surveys are sent out to consumers without knowledge of an office
visit, while other field tests have relied on medical group files to identify
patients who have had a visit in the last twelve months.
o In the TriCare field test, surveys were mailed out in two month cycles
(patients received surveys either 60 days post-visit or 90-120 days postvisit).
 There is a pediatric version of the survey in development; the current survey does not
address mental health services and would not be appropriate for psychiatric patients.
It was suggested that the ECHO (Experience of Care and Health Outcomes) survey
could be put forward to address that gap.
 Implementing the survey in a cost-efficient manner is a concern. The Advisory Board
is very conscious of this issue. AHRQ is working on making the survey available online.
2
An attachment detailing the field testing sites was provided.
Next Steps
Patient Experience Survey:
 Kevin Weiss would like the workgroup to endorse the survey on the next call,
January 6, 2006 and forward the survey to the AQA meeting at the January 12, 2006
for endorsement.
 If workgroup members have questions, concerns, or comments, they should email
Rebecca Hayes.
Other Measurement Activities:
 Kevin solicited participants interested in working on composite measures and registry
data.
 Other work for the workgroup in 2006 will include subspecialty measurement sets.