Pennsylvania’s CQI Effort
Jeanne Schott, Project Manager
Stephanie Maldonado, Project Manager
January 2013
Important January 2013 Dates:
 Sustaining Change Workgroup: 1/2/13
 Crawford Notification Meeting: 1/7/13
 CQI Project Manager Technical Assistance: 1/9/13 Allegheny and
Lackawanna Site Lead teams and 1/29/13 for Montgomery Site Lead Team
 Montgomery Notification Meeting: 1/10/13
 Snyder Notification Meeting: 1/14/13
 QSR Refresher TOC dates: 1/18/13 from 1pm-3pm and 1/25/13 from 10am12pm
 QSR Connect Call: 1/22/13 from 10am-12pm
Important February 2013 Dates:
 Sustaining Change Workgroup: 2/6/13 from 9:30am -12:30pm
 Allegheny County QSR Training: 2/6-2/7; QSR 2/19-2/21 and 2/26-2/28
 QSR Refresher TOC dates: 2/13 from 10am-12pm
 QSR Connect Call: 2/14 10am -12pm
 Lackawanna County QSR Training: 2/27-2/28
Updates for CQI Phase III
Phase III Counties:
1. Erie (15 cases; Week 1: July 16-18 and Week 2: July 23-25)
2. Crawford (12 cases; Week 1: May 14-16 and Week 2: May 21-23)
3. Clarion (7 cases in November 2013)
4. Snyder (5 cases; May 7-9)
5. Montgomery (15 cases; Week 1: April 15-17 and Week 2: April 22-24)
6. Luzerne (15 cases; Dates to be determined)
7. Schuylkill (12 cases; July 9-11)
Phase I and II Counties conducting annual State-supported QSRs:
1. Philadelphia (25 cases; December 2012)
2. Allegheny (20 cases; Week 1: February 19-21 and Week 2: February 26-28)
3. Lackawanna (15 cases; March 11-12 and 13-14)
4. Wyoming (5 cases; June 2013)
CQI Project Managers’ Highlights: Confidentiality and the
Maintenance of QSR Roll-Up Sheets and Written Case
Review Summaries
The following guidance is being provided for counties specific to what they should do with the casespecific information that is gathered during the QSR, specifically the QSR Roll-Up Sheet and Written Case
Review Summaries. QSR reviewers complete a QSR Roll-Up Sheet which includes demographic
information and the ratings scores of each of the 23 indicators in addition to the Written Case Review
Summary, which is a narrative summary of their findings as they relate to the ratings for each of the
indicators and case-specific and agency recommendations. Once these materials are reviewed by the
Site Lead team as part of the post-QSR second level quality assurance process, these finalized
documents are to be forwarded to Hornby Zeller Associations, who maintains the statewide data
warehouse of all the QSR finalized documents.
Once the QSR Written Case Review Summaries have been finalized, the Local Site Lead(s) should
distribute a hardcopy of the document to the caseworker and supervisor. (NOTE: The juvenile probation
officer and supervisor of any shared case responsibility cases should also receive the Written Case
Review Summary). We recommend that the caseworker (and supervisor, when appropriate) hold a team
meeting with all family members, professionals and other team members so that the team can discuss a
summarization of the overall case-specific findings that were outlined as part of the QSR. This is
recommended so that the team can discuss the information garnered via the QSR process and can work
together to decide what next steps are most appropriate. We are NOT recommending that a hard copy of
the Written Case Review Summary document be given to the child/youth’s family and/or team members,
but rather that the information and recommendations offered by the QSR reviewers be summarized for the
team by the caseworker (and supervisor, when appropriate) for consideration by the team. Ideally, this
team meeting should occur within two weeks of the caseworker and supervisor’s review of the Written
Case Review Summary.
Once the county caseworker and supervisor have reviewed the Written Case Review Summary, they are
encouraged to return the Written Case Review Summary to the county’s Local Site Lead for filing of the
document in a separate agency CQI file as opposed to the child/youth/family’s case file. This
recommendation is being made because the information gathered through the QSR process includes
case-specific as well as agency/systemic recommendations, which are to be applied for the purpose of
supporting the child welfare agency’s overall continuous quality improvement efforts.
CQI Project Manager Technical Assistance for Site Lead
Teams
We have held three sessions of the Site Lead team technical assistance. Thank you to Allegheny,
Lackawanna and the Montgomery Site Lead teams for participating. We have also scheduled technical
assistance sessions with Snyder and Crawford and we are in the process of selecting dates for Erie’s
technical assistance. Site Lead teams should reach out to their Lead CQI Project Manager to schedule
the technical assistance session for the Site Lead team if it has not yet been scheduled.
QSR Refresher Training on Content (TOC) for all Site
Leads
There is one more scheduled session for the TOC for Site Leads on 521: QSR Refresher Session. It is
scheduled for the following date/time:
 2/13/13 from 10am-12pm
Please contact Rachael Ickes (rmi6@pitt.edu or 1-877-297-7488) to register for a minimum of ONE of
these sessions. If a State Site Lead (or an interested Local Site Lead) is unable to attend ANY of the
above sessions, please contact Stephanie Maldonado smaldonado@pa.gov and Jeanne Schott
jls192@pitt.edu ASAP, as we are willing to schedule additional dates and times as needed.
Reminder: This TOC is mandatory for State Site Leads since you are most often designated to conduct
this training for your QSR review team. This TOC is optional for Local Site Leads.
If you have already participated in the TOC for the QSR Refresher but have not yet submitted your Trainer
Directory Form, please send your form to Jessica Shiffler (jms137@pitt.edu).
Sustaining Change Workgroup
The Sustaining Change workgroup members continue to meet monthly. Focus of their most recent work
includes:
2
CQI Newsletter – January 2013




Overarching review of the phased-in implementation of Pennsylvania’s Continuous Quality
Improvement (CQI) effort
o Building capacity so that all 67 counties are using a CQI framework
o Resources supporting the statewide implementation efforts
 Ongoing support for QSR Manual, QSR Roll-Up Sheet, etc.
Statewide QSR report – sharing the data with key stakeholders; enhancing the utilization of the
data for policy and practice improvements to improve outcomes from a statewide perspective
o Example: Focused efforts on some practice areas for improvement (i.e. father engagement)
Review of County Improvement Plans
o Highlight the areas that are working well within our system and communicate this message
o Discussion about the gaps identified by counties (i.e. what are the trends)
o Remedies selected by counties and connection to TA Collaborative efforts
Review of the Administration on Children, Youth and Families guidance regarding Continuous
Quality Improvement Systems and how Pennsylvania’s CQI effort aligns with this guidance
o Self-assessment and strength and gap analysis
We are also continuing our outreach effort for new members of the workgroup. We are currently recruiting
new members for this workgroup, particularly from CQI Phase I, II, or III counties or those counties
interested in being a Phase IV or V county. Please contact Jeanne Schott (jls192@pitt.edu) or Stephanie
Maldonado (smaldonado@pa.gov) for more information. You may participate in this workgroup in person,
or via web ex. The next meeting is scheduled for February 6th 9:30am – 12:30pm.
Questions and Answers:
Question: How is the length of the onsite QSR determined?
Answer: The County is able to plan for either a 2-day or a 3-day onsite QSR and cases may be split over 1
or 2 weeks if the sample size is too large to accommodate the entire review team during the same 2 or 3
day period. Onsite QSRs may not go beyond 3 days due to the time commitment that reviewers and Site
Leads are already making to participate. There are several factors to consider when choosing between a 2
or 3 day review. The County should consider: the number of cases in the sample; the logistics of the site
(i.e. space to conduct the review); and time commitment of all those involved. NOTE: The County and Site
Lead teams that are conducting a 2-day review need to recognize that QSR reviewers will need to be
afforded additional time following the onsite QSR days so that they can complete and submit their finalized
Written Case Review Summaries which necessitates additional planning and follow-up by the Site Lead
team even prior to conducting the second-level QA.
Question: What happens when a focus child/youth is not seen during QSR?
Answer: If the focus child/youth is not seen at their scheduled interview, the review team should
immediately contact their Site Leads and attempt to make alternate arrangements to see the focus
child/youth at another time during the onsite review day. This is because it is mandatory that the focus
child/youth be seen by the review team. In the event that the reviewers, despite reasonable efforts, do not
see the focus child/youth, the review team should still proceed with scoring the child/youth/family’s case
(to the best of their ability) and submit their Written Case Review Summary. Site Leads need to notify
their HZA lead and their CQI Project Manager that there was a focus child/youth was not seen. This
needs to be done because the case will NOT be included in the County’s final data presented at the Exit
Conference, in the Final Report or as part of the Next Steps meeting. The child/youth/family’s case is
unable to be included because QSR reviewers were unable to make a definitive rating for the safety of the
focus child/youth without seeing him or her and for that reason the data is not included.
Question: Why are there no stratifications for the QSR case samples?
Answer: The previous stratifications that counties were required to have during Phase I and Phase II of
the statewide CQI effort were based on stratification criteria established in collaboration with ACF,
following PA’s most recent CFSR and as part of PA’s PIP baseline/measurements during the PIP
implementation. Since our successful completion of the PIP, we are no longer required to align with
stratification formulas that were required as part of our PIP measurements, yet PA would still like to align
3
CQI Newsletter – January 2013
with how ACF will assess a state’s continuous quality improvement process to improve outcomes. In
August 2011 (following our successful completion of the PIP), ACF released a CQI Information
Memorandum (IM) that PA would like to adhere to. Unfortunately, there have been no specifics outlined
by ACF about sampling or sample stratification in the CQI IM – and we have asked for further clarification,
but have not received any further information. Therefore, the decision was made by the CQI Sponsor
Team to have all counties participating in a state-supported QSR to have sample stratification based only
on type of case - In Home cases (60%) and Out of Home cases (40%). This was done so that we could
assure a randomized sample of cases across all state-supported QSRs. Any stratification(s) that a county
would like to make to delve into county-specific challenges (or areas of strength) are encouraged to be
done via a county’s internal QSRs and not as part of the state-supported QSRs, so that statewide data
can remain consistent. If each county were to stratify their data based on county-specific need for the
state-supported QSRs, our sampling methodology would be skewed.
Question: Is someone able to pre-populate the demographic information or other information in sections
1-5 on the electronic QSR Roll-Up Sheets for the QSR review team prior to the onsite QSR?
Answer: County staff (either the Local Site Lead or the caseworker/supervisor – whomever the Site Lead
team decides) will pre-populate all the information on Page 1 and Page 2 (sections 1-5) of a hardcopy of
the QSR Roll-Up Sheet. That hardcopy is to be provided to the QSR review team at the onsite QSR and
the review team is responsible for QA’ing the hardcopy QSR Roll-Up Sheet and entering the data on the
electronic version of QSR Roll-Up Sheet on the flash drives that they are given by the Site Lead team.
We are requesting that no one else do the data entry for the QSR team, as they need to review the
information to assure it is correct as part of their role as a QSR reviewer. This is also outlined on page 33
of the QSR Manual (under preparations for the onsite QSR).
Guest Spotlight by Roseann Perry- Update on Changes to
the Licensure Process:
OCYF has been shifting from a compliance based toward a quality of service focus over the past couple of
years. Regional office staff has commented that they found out more about a county operations, strengths
and challenges by participating in a 3-day Quality Service Review than from 3 years worth of annual
inspections.
Beginning at the end of 2011, a workgroup was chartered made up of Regional Representatives and
Supervisors who were charged with reviewing the annual licensure process of County Children and Youth
Agencies to streamline the field work of the inspection, ensure consistency in process within and across
the regions, and, most importantly, provide a link of the regulatory compliance to quality indicators. Some
of the tasks addressed by the group were a uniform inspection tool and LIS format, as well as a review of
the sample size.
The new format and process developed will be applied to all counties, regardless of whether or not a
county has had a Quality Service Review conducted within the licensing period. The most significant
change is with the Sample Size for the inspection. The sample size identifies 10% of cases be reviewed
in each service area (Screen Outs, GPS Assessments, CPS Investigations, In-Home Services, Placement
and Adoption) up to a maximum of 10 cases per service area. One of the differences in this process is
that an individual case will be assessed for all applicable regulations during the period under review. For
example, if a case has an Intake GPS Assessment, becomes accepted for in-home services, then there is
a subsequent CPS report and the child is placed – this one case will serve as one case reviewed in each
of the service type categories. County operated foster home samples will also be selected as well as a
review of personnel records. There will also be at least one case in which there is Shared Case
Responsibility and one case in which a Transitioning Aged Youth is being served.
For those counties who have had a QSR occur, the cases that were part of the QSR will be used as part
of the sample numbers and, in most circumstances, this will meet the minimum required In-Home and
Placement sample size needed. Regional Representatives will work with our QSR counties in advance of
the inspection to determine what additional cases may need to be pulled to meet the minimum sample
4
CQI Newsletter – January 2013
size for the other areas of casework services.
It is important to note that any annual inspection citations and licensing decisions are based solely on
regulatory and statutory compliance findings; however, the workgroup through the use of the LIS provides
a link of regulatory cites to quality indicators and also poses guiding questions for context in reviewing the
impact on case activity.
All of the licensing documents, including the checklist to be used for annual inspections are now located
on the OCYF Portal under the State and County Policy and Program Resource section. Click on the State
collection and then in the Final County Licensing Tool subfolder. The checklist is now one combined
document of all regulations rather than programmatically separated out checklists. The checklist will be
updated on an annual basis for incorporation of any statutory or regulatory changes. Please contact your
OCYF Regional Office with any additional questions.
Ongoing Call for QSR Review Volunteers
We still have 10 state-supported QSRs that will be held between February 2013– November 2013 and we
need to identify all potential people who are willing to be reviewers. All interested individuals should send
their name and contact information to Brandie Gilbert at bms27@pitt.edu.
A Special Thank You
We would like to bid a fond farewell to Jen Kerr who has been a true champion for change and a leader in
moving our continuous quality improvement efforts forward in Pennsylvania. You will be missed but we
wish you well and much success in all your future endeavors!
Future Newsletter Topics
The following topics will be covered in future CQI Newsletters:
 Question & Answer - Any tips for Site Leads preparing for the Exit Conference presentation of
preliminary findings?
 CQI Highlight - continued recommendations of confidentiality of QSR-specific information
Do you have any suggestions for things you’d like to see in the Newsletter? If so, please contact
Stephanie Maldonado (smaldonado@pa.gov) or Jeanne Schott (jls192@pitt.edu).
5
CQI Newsletter – January 2013