Measuring Organizational
Readiness for Quality Improvement
Elizabeth Yano, PhD
VA Greater Los Angeles HSR&D Center for the
Study of Healthcare Provider Behavior
UCLA School of Public Health
Perspectives
Measuring organizational readiness for change
– Traditional approaches (attitudes, beliefs, culture)
– Expanding to clinical structure and care processes
Examples of studies of organizational influences
on quality
– National VA reorganization (emphasizing PC)
– VA quality improvement interventions (QUERI)
Pre-implementation organizational assessment
Post-implementation predictors of sustainability/spread
Review key informant surveys as one method
Organizational Readiness
“Readiness represents a state of mind about the
need for an innovation and the capacity to
undertake change...”
“Readiness consists of people’s beliefs,
attitudes, and intentions about the desirability of
changes, and perceptions about the ability of
individuals and the organization to successfully
make those changes.”
“Readiness represents the predisposition to
unfreeze established patterns of behavior.”
Sources: Beer 1990, 2004; Beer & Eisenstat 1996, 2004; METRIC 2005
Organizational Readiness
Commonly measured as:
– Culture (innovativeness, flexibility, climate)
– Definition of roles
– Investment of resources (including competition)
– Past contracts or agreements (can limit
changes in behavior)
– Threats to power—control over decisions,
resources
Sources: Beer 1990, 2004; Beer & Eisenstat 1996, 2004; METRIC 2005
Contribution of Organizational
Readiness Measures to VA Quality
Culture (staff surveys)
– QI orientation associated with  tobacco counseling rates
Investments of resources/competition (PC leader survey)
– Sufficiency of clinical support arrangements accounts for
substantial variation in prevention performance
Past contracts/agreements (PC leader survey)
– Stringent PC patient assignment associated with lower breast &
cervical cancer screening
Control over decisions, resources (PC leader survey)
– PC practice autonomy over internal operations associated with
higher CRC screening rates and better diabetic control
Sources: Yano et al., JGIM, 2002; Soban & Yano, JACM, 2005; Yano HCOC, 2005;
Goldzweig et al, AJMC, 2004; Jackson, et al., AJMC, 2005.
Organizational Readiness:
Expanding Beyond Attitudes, Beliefs, Culture
Interventions associated with changes in quality
–
–
–
–
Organizational change (biggest effects)
Reminders
Audit and feedback/profiling
Incentives
Organizational change interventions focus on
structures and processes of care (org “enablers”)
– Not attitudes, beliefs, culture – hard to change
– Clinical processes of care, management strategies,
clinic structure, provider training, decision support
Source: Stone, et al., Ann Intern Med, 2002.
Major Reorganization of VA Health
Care System (1996-present)
Historically, VA was…
– Individually-managed hospitals focused on specialty care
– Funded through prior-year cost reimbursement
– Extremely poor quality of care reputation
Congressionally approved VA reorganization…
–
–
–
–
–
–
–
Veterans’ integrated service networks (VISNs) (n=22)
Network-level budget control and performance agreements
Incentivized audit-and-feedback on quality/satisfaction
Funded by population served (capitated)
Eligibility reform changed VA to health plan and payor
Computerized patient record system (CPRS) w/decision support
Primary care as platform for restructuring care delivery
VA Health Care System as the
Organizational Context
VA health care system largest in U.S.
– 163 hospitals, >750 freestanding community-based
outpatient clinics, >130 nursing homes, >200 vet ctrs
– Served over 5 million patients in past year
Affiliated with over 1,200 academic institutions
– Including 107 medical schools, 55 dental schools
– 110,000 students and trainees in >45 disciplines/year
> ½ of all U.S. MDs received part of their training in VA
VA market penetration growing rapidly
– 20% of those <65 and 29% of those >65
VA health care budget $25.9 billion (2003)
VA Improves 12 of 13 Leading Quality
Indicators (1995-2000)
1995
1996
1998
1999
2000
100
90
80
70
60
50
40
30
20
10
0
HbA1c
Flu shots
CRC Screen
ASA MI
Jha NEJM 348:22 2003
VA Beats Medicare 12-1 in 2000
Medicare
VA
100
90
80
70
60
50
40
30
20
10
0
Mammo
Flu shots
Pneumovac
HbA1c
DM eye
DM lipid
Jha NEJM 348:22 2003
Primary Care Organizational Changes
Percent of VAs w/PC Program
100
90
80
PC budget
PC-based QI
Pt-PCP assignment
PC teams
PC Teams
Pt assignment
70
60
PC-based QI
50
40
30
Separate PC budget
20
10
0
1993
1996
1999
Changes in PC Physician Volume/Mix
Mean FTEEs
10
9
8
GIM MDs
Geriatrics MDs
IM Subspecialists
Psychiatrists
7
6
5
4
3
2
1
0
1993
1996
1999
Staff Alignment to Primary Care
Percent of VAs with PC Staff Reporting Only to Primary Care
90
80
70
60
50
MDs
NPs
PAs
MSWs
RNs
40
30
20
10
0
1993
1996
1999
Changes in PC Resource Sufficiency
Percent of VAs Reporting Always/Mostly Sufficient
90
*
80
70
60
*
*
1993
1996
*
50
*
40
30
20
10
0
Admin
offices
Examining
Rooms
Treatment
rooms
PCs
Patient
education
space
Organizational Readiness:
Implications for QI Research
Basic
Science
VA intramural
research program
Clinical
Research
Health Services
Research
TRIP over the
proverbial
“brick wall”
TRANSLATION
“new discoveries”
EFFICACY to
EFFECTIVENESS
“new treatments”
“new cures”
“improved access”
“better quality of care”
IMPLEMENTATION
“barriers”
“barriers”
“barriers”
Routine
Care
Organizational Readiness:
Implications for QI Research
Health Services
Research
• Measure organizational readiness
• Use to select promising sites
• Locally tailor QI intervention(s)
• Fixed characteristics (eg, urban/rural)
• Mutable characteristics (change/adapt)
Routine
Care
Organizational Readiness for QI:
Preparing to Implement Depression Collaborative Care
Depression collaborative care model in 7 VAs
–
–
–
–
Depression care manager
PC-MH collaboration
Informatics/registry (screening, reminders)
Leadership support
Pre-intervention semi-structured telephone
interviews of all PC and MH leaders
– Assess current screening and referral processes
– Assess local barriers (eg, turf, staffing gaps, history)
– Fed back into planning calls, adapted protocols
Sources: Parker LE, Yano EM, Rubenstein LV, 2003; Ficket et al, in prep.
Organizational Readiness for QI:
Preparing to Implement HIV QI Intervention
HIV quality improvement intervention trial (16 VAs)
(Asch et al)
– Group-based QI, audit-and-feedback, reminders
Used national HIV provider survey (n=118 VAs)
to examine how organizational factors affected
adoption of HIV QI activities before starting trial
–
–
–
–
Assessed local QI activities, HIV guideline use
Measured attitudes toward proposed QI modalities
Evaluated regional, facility and practice variations
Fed back to team (site selection, adapted protocols)
Sources: Anaya, Am J Med Qual 2004; Korthuis et al, JAIDS, 2003; Yano, et al., Mil Med 2005.
Organizational Readiness for QI:
Implementing/Sustaining Depression Collaborative Care
Onsite in-person stakeholder interviews
– Network, medical center, clinic site
– PC and MH leaders, PC and MH providers, nurses,
care managers, patients, consumer reps (n=106)
Semi-structured interviews exploring
implementation of each care model component
– Leadership support/opinion leaders
– Depression care manager interaction/contacts
– Provider interactions and ongoing education needs
Fed back to implementation/spread teams and
developing “diffusion” tools
Different Measurement Approaches
Knowledge/
Evidence Base
HIGH
Key Informant
Surveys
• know domains/items?
• who has knowledge?
• can you get to them?
• will they cooperate?
LOW
Qualitative
Interviews
Informs
survey design
• telephone or in-person
• different levels of interview structure
• different levels of stakeholders/informants
Provider
Surveys
• if variation important
• if QI intervention
requires their change
• AND all of above
Measuring Organizational Characteristics
Using Key Informant Surveys
Step #1: Translate ideas into survey domains
Example: Translate HIV QI strategic plan into domains:
–
–
–
–
HIV screening policies and protocols
Practice arrangements for management of HIV disease
Provider ratings of effectiveness of diff QI interventions
Potential barriers to adoption of HIV guidelines
Example: Evaluate PC organizational predictors of quality
– Institute of Medicine primary care domains (access, continuity…)
– Primary care strategic plans
– PC practice managers (observation and interview)
Example: Depression collaborative care implementation
– Disaggregate care model components—explicitly open “black box”
Measuring Organizational Characteristics
Using Key Informant Surveys
Step #2: Select measures allowing benchmarking
against other health care organizations
Example: VA QUERI HIV & HCSUS
Example: VA, NCQA PSAS, & Kaiser
Example: VA & DHHS Office of Women’s Health COEs
Step #3: Develop new measures that match
structure-process or -outcome model or QI goals
Literature review, expert panel methods—build on evidence
Talk to “real people” who live in world you are studying
Begin with qualitative interviews or focus groups
Measuring Organizational Characteristics
Using Key Informant Surveys
Step #4: Test and adapt survey to target health
care settings
Cognitive interviews with sample respondents
Vary types of organizations included (big/small)
Develop multiple modules if needed
– By setting (hospitals, freestanding outpatient clinics)
– By respondent type (hosp director, PC chief, lab tech)
Step #5: Identify key informants
Desired knowledge base, incentives to participate
Social desirability and need for validation, politics
Measuring Organizational Characteristics
Using Key Informant Surveys
Step #6: Sampling organizations
What’s in it for the organization?
Sampling to represent what?
Where do
– Types of organizations, units w/in
you get
organizations, different size and complexity this kind of
information?
– Regions, urban/rural locations, other area
characteristics
Obtaining their cooperation…
– Leadership support, uses of data, IRB, HIPAA
– Funding to compensate for administrative time
Measuring Organizational Characteristics
Using Key Informant Surveys
Step #7: Field preparation
Identify and market to venues common to respondents
Determine regular communication options
– Management meetings, conference calls, broadcast
fax, advance mass mailing
Involve senior leaders/opinion leaders – spokespersons
Market value of participation
– Demonstrate previous uses of data (“good works”)
– Offer incentive (eg, summary of survey results,
publications order form, financial)
Contact organizations and talk to support staff
Measuring Organizational Characteristics
Using Key Informant Surveys
Step #8: Administer surveys
Hardcopy express-mail with prepaid returns
Web-based with varying email introductions and
reminders
Quality review of survey content with active follow-up of
missing data and non-respondents
– 2-week second wave mailouts
– 4-week telephone follow-up
Continual data entry (if hardcopy), quality checking
Follow-up postcards and repeat announcements in
original venues
Example Key Informant Surveys
Primary care practice organization
(93, 96, 99)
– Care arrangements, teams, staffing, authority,
resources, QI, decision support, care coordination,
profiling, incentives, management structure
HIV practice structure/delivery models
93%100%
(00)
– Screening, PC vs. specialty management, HIV clinics,
staffing, provider experience, HIV case management,
HIV guideline use, barriers, provider preferences
Women’s health care delivery organization
83%
(01)
– Clinic structures, service availability, referral
arrangements, decision support, QI, leadership,
policies, staffing, authority, provider training
82%100%
Example Key Informant Surveys
Clinical practice organizational survey (05)
– 3 modules:
Network directors (n=21) (~$1 billion each)
Chiefs of staff (aka medical directors) (n=160)
Primary care clinic directors (n=259)
– Mapped to prior VA surveys  time trends
– Mapped to NCQA PSAS© and Kaiser IT surveys 
benchmarking
Women’s primary care organizational survey (06)
– Senior WH clinician or medical director
– Classify every VA by organizational taxonomy
– Evaluate quality of care within different types
Thank you
Elizabeth.yano@med.va.gov
(818) 895-9449