Independent Monitoring for Quality: An
Evidence-Based Strategy for Building and
Sustaining Effective Programs
Guy Caruso and MaryJo Caruso
2011 National Lifespan Respite Conference
Phoenix, AZ
Presenters
Dr. Guy Caruso
Western Coordinator
Institute on Disabilities at Temple University
guyucdd@consolidated.net
724-934-1142
&
MaryJo Caruso
Care Break Coordinator
The Watson Institute
maryjoa@thewatsoninstitute.org
412-749-2863
Goals of the Presentation
1. Learn about the PA Independent Monitoring for Quality
(IM4Q) evidence-based program and it's 10 years of
experience with collecting, summarizing and sharing
results.
2. Identify how IM4Q has relevance to respite providers
looking for a consumer driven tool to collect data to
make evidence informed decision making.
3. Enhance current program functioning, identify quality
standards and develop some possible respite programs
based on the monitoring for quality model.
IM4Q Beginnings
• 1997, the State developmental disabilities Planning
Advisory Committee recommended that the State
develop Independent Monitoring for Quality (IM4Q) as
part of service system modernization
• Designed to be conducted by people with
developmental/other disabilities, families, and other
interested people
• Does not measure compliance with government
regulations or service/provider quality, and instead it
measures people’s quality of life outcomes
Quality of Life Areas Measured
1.
2.
3.
4.
5.
6.
7.
8.
Satisfaction
Dignity, Respect and Rights
Choice/Control
Relationships
Inclusion
Competence, Personal Growth
Supports for the Person
Physical Setting
(These are part of the interview survey – Essential Data Elements)
Respite Questions
What Quality of Life Areas Would
a Respite Program Measure?
Who Would Develop the Respite
Survey Instrument? For Whom?
Describing IM4Q & What it Does
• IM4Q programs are County contracted/State
approved/funded independent/conflict-free 501c3 entities
with local governing boards that are person/familydriven, and community-based
• The County and local IM4Q Program follow State
protocols and practices to ensure monitoring is done
consistently across the state (IM4Q Manual)
Respite Questions
Who Would Oversee an
independent Statewide Respite
evaluation component?
Who Would Develop Guidelines
for the independent Respite
evaluation component?
Describing IM4Q & What it Does
• The program sends a trained 2 person team including at
least one person with a disability/family member out on
an interview with a standardized interviewing instrument
- Essential Data Elements (EDE)
• Interviews generate a team considerations report sent to
the County to forward to the provider and supports
coordinator to discuss/address the consideration (often
added to a person’s Individual Service Plan - ISP)
Respite Questions
Who Would be on a Respite
Interview Team?
Once an Interview is Completed
• Where Would the Results Go?
• What Would be Done with the
Results?
Describing IM4Q & What it Does
• The State has a contract with the Institute on
Disabilities to develop standardized state,
County, and other relevant reports
Respite Questions
Who Would Develop Reports and
Analyze the Data?
What Would Happen With the
Reports?
Describing IM4Q & What it Does
The State:
• Has a statewide IM4Q Steering Committee to
advise it on IM4Q with diverse stakeholder
representation (e.g., self advocates, providers)
• Contracts with the Institute on Disabilities to
have technical advisors for IM4Q
Respite Questions
What State Committee Structure Would
You Have to Advise the Respite
Evaluation Component?
Would There be Technical Advisors
Whose Role is to Continually Improve the
evaluation program and ultimately the
program? If so, from where?
Yearly IM4Q Interviews
6,000 interviews a year across state
with consumers
3000 with
Family/Friend/Guardians
Respite Questions
How Many Interviews Would You Want to
Conduct a Year?
Who Would be Interviewed?
• People Receiving Respite?
• Families of People Receiving Respite?
• Persons Providing Respite?
Outcomes – Various Reports
• A process for change at the individual level has
been developed between each local program
and County (“Closing the Loop”)
• Statewide steering committee makes
recommendations for change at the systems
level to the State (e.g., Communication Bulletin)
Respite Questions
What Process Would be Created to Insure
People/Family Caregivers Receiving
Respite are Satisfied with What They
Receive?
Would the Lifespan Respite Network /
Coalition, Respite State Advisory Group
Make Recommendations for Program or
Statewide Improvement and to Whom?
Almost a Decade of IM4Q Data
The following graphs and
charts depict some of the
results over the years.
Satisfaction: Do you like where you
live?
YEAR
SATISFACTION
2000*
-
2001
89%
2002
90%
2003
90%
2004
90%
2005
89%
2006
89%
2007
89%
2008
90%
* Wording change in question
• Percent who like
where they live has
remained stable over
time
• These answers
obtained in consumer
interview
Respite Question
What Satisfaction Questions
Would You Develop for People
Receiving Respite and Their
Family Caregivers?
Dignity Scale, 2000-2008
YEAR
DIGNITY SCALE
2000
79.9
2001
78.5
2002
81.4
2003
80.3
2004
81.5
2005
81.5
2006
81.4
2007
81.8
2008
82.5
• Scale ranges from 0 to 100,
with a higher score indicating
higher levels of dignity and
respect
• Dignity and respect for the
individual has remained
stable from year to year.
• These answers obtained in
consumer interview
Respite Question
What Dignity Questions Would
You Develop for the People
Receiving Respite and Their
Family Caregivers?
Choice Scale, 2000-2008
• Scores on the Choice
and Control Scale could
range from 0 to 100, with
a higher score indicating
more opportunities to
exert choice and control
• Greatest opportunity for
choice and control
achieved in 2008
YEAR
Choice Scale
2000*
-
2001*
-
2002
40.2
2003
43.0
2004
42.3
2005
46.9
2006
47.2
2007
47.9
2008
48.2
* Scale changed significantly in 2002
Respite Question
What Choice Questions Would
You Develop for People
Receiving Respite and Their
Family Caregivers?
Inclusion Scale, 2000-2008
YEAR
Inclusion
Scale
2000
41.8
2001
40.2
2002
40.8
2003
41.9
2004
40.5
2005
42.4
2006
41.5
2007
41.4
2008
42.1
• Scores on the Inclusion Scale
could range from 0 to 100, with
a higher score indicating
greater inclusion (going more
frequently to places in the
community)
• Inclusion scores have
remained stable over time
Respite Question
What Questions Would You Use
to Measure Community Inclusion
for the People Receiving Respite
and Their Family Caregivers?
Revisiting Today’s Goals
Relevance to Respite
• How does IM4Q have relevance to respite providers who
are looking for a consumer driven tool to collect data to
make evidence informed decision making?
Possible Respite Models
• How could possible respite programs that enhance
current program functioning and identify quality
standards be developed based on the monitoring for
quality model?
Closing
Questions / Comments?
Dr. Guy Caruso
Western Coordinator
Institute on Disabilities at Temple University
guyucdd@consolidated.net
724-934-1142
MaryJo Caruso
Care Break Coordinator
The Watson Institute
maryjoa@thewatsoninstitute.org
412-749-2863