Measuring Organizational
Readiness for Quality Improvement
Perspectives
Measuring organizational readiness for change
– Traditional approaches (attitudes, beliefs, culture)
– Expanding to clinical structure and care processes
Elizabeth Yano, PhD
VA Greater Los Angeles HSR&D Center for the
Study of Healthcare Provider Behavior
UCLA School of Public Health
Examples of studies of organizational influences
on quality
– National VA reorganization (emphasizing PC)
– VA quality improvement interventions (QUERI)
PrePre-implementation organizational assessment
PostPost-implementation predictors of sustainability/spread
Review key informant surveys as one method
Organizational Readiness
Organizational Readiness
“Readiness represents a state of mind about the
need for an innovation and the capacity to
undertake change...”
change...”
“Readiness consists of people’
people’s beliefs,
attitudes, and intentions about the desirability of
changes, and perceptions about the ability of
individuals and the organization to successfully
make those changes.”
changes.”
“Readiness represents the predisposition to
unfreeze established patterns of behavior.”
behavior.”
Commonly measured as:
Sources: Beer 1990, 2004; Beer & Eisenstat 1996, 2004; METRIC 2005
Sources: Beer 1990, 2004; Beer & Eisenstat 1996, 2004; METRIC 2005
Contribution of Organizational
Readiness Measures to VA Quality
Culture (staff surveys)
– QI orientation associated with ↑ tobacco counseling rates
Investments of resources/competition (PC leader survey)
– Sufficiency of clinical support arrangements accounts for
substantial variation in prevention performance
Past contracts/agreements (PC leader survey)
– Stringent PC patient assignment associated with lower breast &
cervical cancer screening
Control over decisions, resources (PC leader survey)
– PC practice autonomy over internal operations associated with
higher CRC screening rates and better diabetic control
Sources: Yano et al., JGIM, 2002; Soban & Yano, JACM, 2005; Yano HCOC, 2005;
Goldzweig et al, AJMC, 2004; Jackson, et al., AJMC, 2005.
– Culture (innovativeness, flexibility, climate)
– Definition of roles
– Investment of resources (including competition)
– Past contracts or agreements (can limit
changes in behavior)
– Threats to power—
power—control over decisions,
resources
Organizational Readiness:
Expanding Beyond Attitudes, Beliefs, Culture
Interventions associated with changes in quality
–
–
–
–
Organizational change (biggest
(biggest effects)
effects)
Reminders
Audit and feedback/profiling
Incentives
Organizational change interventions focus on
structures and processes of care (org “enablers”
enablers”)
– Not attitudes, beliefs, culture – hard to change
– Clinical processes of care, management strategies,
clinic structure, provider training, decision support
Source: Stone, et al., Ann Intern Med, 2002.
1
Major Reorganization of VA Health
Care System (1996(1996-present)
VA Health Care System as the
Organizational Context
VA health care system largest in U.S.
Historically, VA was…
was…
– IndividuallyIndividually-managed hospitals focused on specialty care
– Funded through priorprior-year cost
cost reimbursement
– Extremely poor quality of care reputation
– 163 hospitals, >750 freestanding communitycommunity-based
outpatient clinics, >130 nursing homes, >200 vet ctrs
– Served over 5 million patients in past year
Congressionally approved VA reorganization…
reorganization…
–
–
–
–
–
–
–
Veterans’
Veterans’ integrated service networks (VISNs
(VISNs)) (n=22)
NetworkNetwork-level budget control and performance agreements
Incentivized auditaudit-andand-feedback on quality/satisfaction
Funded by population served (capitated
(capitated))
Eligibility reform changed VA to health plan and payor
Computerized patient record system (CPRS) w/decision support
Primary care as platform for restructuring care delivery
Affiliated with over 1,200 academic institutions
– Including 107 medical schools, 55 dental schools
– 110,000 students and trainees in >45 disciplines/year
> ½ of all U.S. MDs received part of their training in VA
VA market penetration growing rapidly
– 20% of those <65 and 29% of those >65
VA health care budget $25.9 billion (2003)
VA Improves 12 of 13 Leading Quality
Indicators (1995(1995-2000)
1995
1996
1998
1999
2000
100
90
80
70
60
50
40
30
20
10
0
HbA1c
Flu shots
CRC Screen
ASA MI
VA Beats Medicare 1212-1 in 2000
Medicare
VA
100
90
80
70
60
50
40
30
20
10
0
Mammo
Flu shots
Pneumovac
Primary Care Organizational Changes
HbA1c
DM eye
DM lipid
Jha NEJM 348:22 2003
Jha NEJM 348:22 2003
Changes in PC Physician Volume/Mix
Mean FTEEs
10
Percent of VAs w/PC Program
100
90
80
9
PC budget
PC-based QI
Pt-PCP assignment
PC teams
PC Teams
8
GIM MDs
Geriatrics MDs
IM Subspecialists
Psychiatrists
7
Pt assignment
70
6
5
60
PC-based QI
50
4
3
40
Separate PC budget
30
20
10
2
1
0
1993
0
1993
1996
1996
1999
1999
2
Staff Alignment to Primary Care
Changes in PC Resource Sufficiency
Percent of VAs with PC Staff Reporting Only to Primary Care
Percent of VAs Reporting Always/Mostly Sufficient
90
90
MDs
NPs
PAs
MSWs
RNs
80
70
60
50
*
80
70
*
*
*
60
50
40
40
30
30
20
20
10
10
0
1993
1996
*
0
1993
1996
Admin
offices
1999
Organizational Readiness:
Implications for QI Research
Basic
Science
VA intramural
research program
Clinical
Research
Health Services
Research
TRIP over the
proverbial
“brick wall”
TRANSLATION
“new discoveries”
Examining
Rooms
Treatment
rooms
PCs
Patient
education
space
Organizational Readiness:
Implications for QI Research
Health Services
Research
EFFICACY to
EFFECTIVENESS
“new treatments”
“new cures”
“improved access”
“better quality of care”
• Measure organizational readiness
• Use to select promising sites
• Locally tailor QI intervention(s)
• Fixed characteristics (eg, urban/rural)
• Mutable characteristics (change/adapt)
IMPLEMENTATION
“barriers”
“barriers”
“barriers”
Routine
Care
Routine
Care
Organizational Readiness for QI:
Organizational Readiness for QI:
Preparing to Implement Depression Collaborative Care
Preparing to Implement HIV QI Intervention
Depression collaborative care model in 7 VAs
–
–
–
–
Depression care manager
PCPC-MH collaboration
Informatics/registry (screening, reminders)
Leadership support
PrePre-intervention semisemi-structured telephone
interviews of all PC and MH leaders
– Assess current screening and referral processes
– Assess local barriers (eg
(eg,, turf, staffing gaps, history)
– Fed back into planning calls, adapted protocols
Sources: Parker LE, Yano EM, Rubenstein LV, 2003; Ficket et al, in prep.
HIV quality improvement intervention trial (16 VAs)
VAs)
(Asch et al)
– GroupGroup-based QI, auditaudit-andand-feedback, reminders
Used national HIV provider survey (n=118 VAs)
VAs)
to examine how organizational factors affected
adoption of HIV QI activities before starting trial
–
–
–
–
Assessed local QI activities, HIV guideline use
Measured attitudes toward proposed QI modalities
Evaluated regional, facility and practice variations
Fed back to team (site selection, adapted protocols)
Sources: Anaya, Am J Med Qual 2004; Korthuis et al, JAIDS, 2003; Yano, et al., Mil Med 2005.
3
Organizational Readiness for QI:
Implementing/Sustaining Depression Collaborative Care
Onsite inin-person stakeholder interviews
– Network, medical center, clinic site
– PC and MH leaders, PC and MH providers, nurses,
care managers, patients, consumer reps (n=106)
SemiSemi-structured interviews exploring
implementation of each care model component
– Leadership support/opinion leaders
– Depression care manager interaction/contacts
– Provider interactions and ongoing education needs
Fed back to implementation/spread teams and
developing “diffusion”
diffusion” tools
Different Measurement Approaches
Knowledge/
Evidence Base
HIGH
• know domains/items?
Key Informant • who has knowledge?
• can you get to them?
Surveys
• will they cooperate?
LOW
Qualitative
Interviews
Informs
survey design
Provider
Surveys
• if variation important
• if QI intervention
requires their change
• AND all of above
• telephone or in-person
• different levels of interview structure
• different levels of stakeholders/informants
Measuring Organizational Characteristics
Using Key Informant Surveys
Measuring Organizational Characteristics
Using Key Informant Surveys
Step #1:
#1: Translate ideas into survey domains
Step #2:
#2: Select measures allowing benchmarking
against other health care organizations
Example: Translate HIV QI strategic plan into domains:
–
–
–
–
HIV screening policies and protocols
Practice arrangements for management of HIV disease
Provider ratings of effectiveness of diff QI interventions
Potential barriers to adoption of HIV guidelines
Example: Evaluate PC organizational predictors of quality
– Institute of Medicine primary care domains (access, continuity…
continuity…)
– Primary care strategic plans
– PC practice managers (observation and interview)
Example:
Example: Depression collaborative care implementation
– Disaggregate care model components—
components—explicitly open “black box”
box”
Measuring Organizational Characteristics
Using Key Informant Surveys
Step #4:
#4: Test and adapt survey to target health
care settings
Cognitive interviews with sample respondents
Vary types of organizations included (big/small)
Develop multiple modules if needed
– By setting (hospitals, freestanding outpatient clinics)
– By respondent type (hosp director, PC chief, lab tech)
Step #5:
#5: Identify key informants
Desired knowledge base, incentives to participate
Social desirability and need for validation, politics
Example:
Example: VA QUERI HIV & HCSUS
Example:
Example: VA, NCQA PSAS, & Kaiser
Example:
Example: VA & DHHS Office of Women’
Women’s Health COEs
Step #3:
#3: Develop new measures that match
structurestructure-process or -outcome model or QI goals
Literature review, expert panel methods—
methods—build on evidence
Talk to “real people”
people” who live in world you are studying
Begin with qualitative interviews or focus groups
Measuring Organizational Characteristics
Using Key Informant Surveys
Step #6:
#6: Sampling organizations
What’
What’s in it for the organization?
Sampling to represent what?
Where do
– Types of organizations, units w/in
you get
organizations, different size and complexity this kind of
information?
– Regions, urban/rural locations, other area
characteristics
Obtaining their cooperation…
cooperation…
– Leadership support, uses of data, IRB, HIPAA
– Funding to compensate for administrative time
4
Measuring Organizational Characteristics
Using Key Informant Surveys
Measuring Organizational Characteristics
Using Key Informant Surveys
Step #7:
#7: Field preparation
Step #8:
#8: Administer surveys
Identify and market to venues common to respondents
Determine regular communication options
– Management meetings, conference calls, broadcast
fax, advance mass mailing
Involve senior leaders/opinion leaders – spokespersons
Market value of participation
– Demonstrate previous uses of data (“
(“good works”
works”)
– Offer incentive (eg
(eg,, summary of survey results,
publications order form, financial)
Contact organizations and talk to support staff
Hardcopy expressexpress-mail with prepaid returns
WebWeb-based with varying email introductions and
reminders
Quality review of survey content with active followfollow-up of
missing data and nonnon-respondents
– 2-week second wave mailouts
– 4-week telephone followfollow-up
Continual data entry (if hardcopy), quality checking
FollowFollow-up postcards and repeat announcements in
original venues
Example Key Informant Surveys
Primary care practice organization (93, 96, 99)
– Care arrangements, teams, staffing, authority,
resources, QI, decision support, care coordination,
profiling, incentives, management structure
Clinical practice organizational survey (05)
93%100%
HIV practice structure/delivery models (00)
– Screening, PC vs. specialty management, HIV clinics,
staffing, provider experience, HIV case management,
HIV guideline use, barriers, provider preferences
83%
– 3 modules:
Network directors (n=21) (~$1
(~$1 billion each)
Chiefs of staff (aka
(aka medical directors) (n=160)
Primary care clinic directors (n=259)
– Mapped to prior VA surveys Æ time trends
– Mapped to NCQA PSAS© and Kaiser IT surveys Æ
benchmarking
Women’
Women’s primary care organizational survey (06)
Women’
Women’s health care delivery organization (01)
– Clinic structures, service availability, referral
arrangements, decision support, QI, leadership,
policies, staffing, authority, provider training
Example Key Informant Surveys
82%100%
– Senior WH clinician or medical director
– Classify every VA by organizational taxonomy
– Evaluate quality of care within different types
Thank you
Elizabeth.yano@med.va.gov
(818) 895-9449
5