Massachusetts’ Quality Rating and Improvement System Standards Overview 2 Purpose of QRIS Overview of QRIS Provisional Standards Revision Process & Stakeholder Feedback The proposed QRIS Standards Next Steps Massachusetts General Law “The department shall establish a comprehensive system for measuring the performance and effectiveness of programs providing early education and care and services.” “The department shall monitor and evaluate on an ongoing basis all early education and care programs and services, including program outcomes in meeting the developmental and educational needs of all children.” M.G.L. c. 15D § 12 Measurement of performance and effectiveness of programs providing early education and care and services M.G.L. c. 15D § 2(h) Measurement of performance and effectiveness of programs providing early education and care and services 3 Create and implement a system to improve and support quality statewide Related Indicators of Success 4 Massachusetts has standards for quality in early education and care programs that are research-based, broadly understood, successfully implemented, culturally appropriate, and aligned with a quality-building support system. Massachusetts has a system that collects, analyzes and disseminates program quality and child outcome data to inform policy and program development and implementation. Programs seeking to improve their quality have access to a range of resources and supports. Purposes of Massachusetts’ QRIS Programs and providers use one streamlined set of standards that are connected to supports and fiscal incentives to help them meet and maintain the standards. Programs receive feedback and are involved in continuous quality improvement. Parents have easily accessible information about the quality of early care and education programs. Policymakers understand where and how to invest additional resources. High-quality early education and care and out of school opportunities are available throughout the Commonwealth that demonstrate improved outcomes for children. 5 Standards, Assessment and Accountability Teacher Quality EEC Core Competencies Professional Qualifications Registry (PQ Registry) Family & Community Context Program Quality (QRIS) Child Outcomes (formative and summative assessments) 6 Overview of the Standards Level 5: Best practice and demonstrable child growth Level 4: Full Integration Level 3: Focused Development Level 2: Self-Assessment Level 1: MA Licensing 7 Stakeholder Input in the development of the MA QRIS • Defines the policy for quality standards and measurements for use in the statewide QRIS • Provide input to inform decision that support high quality practices EEC Board EDC Team • Provides a mechanism to reflect on process, practice and offer evidence-based for quality, and desired outcomes for children 8 Community Stakeholders EEC • Share strategies to support effective implementation and meaningful participation • Defines administrative procedures • Provides oversight and, guidance • Provides resources to support QRIS implementation 2008 • EEC received guidance from EEC Board and EEC Advisory Team (Feb. - March 2008) • Anne Mitchell presented overview of the QRIS Systems to EEC Board (March 2008) • Initial QRS Stakeholder team developed the concepts scope, purpose and mission of MA QRS. Evolved into MA QRIS • EEC presents at CAYL QRIS Roundtable (July 2008) QRIS At –A– Glance 2007 – 2009 2009 2007 • EEC began conducting early research to set the stage for the design of the MA QRS. • EEC contracted with consultant to conduct initial research on QRS design, and system models. 9 • QRIS Standards presentation to Board (Jan. 2009) • ASOST Stakeholders Group convened • Draft posted for public input (Mar -June 2009 (377 survey respondents) • EEC conducted presentations to over 900 members of the field about QRIS • EEC reconvened the QRIS External Stakeholder team (June 2009) • Provisional Standards were reviewed and revised [evidencebased and measurable (Nov. 2009 - Jan. 2010) • Feedback gathered from stakeholders at events with The CAYL Institute (12/16/2009) QRIS At – A– Glance Jan – March • Feedback provided on Provisional Standards at Advisory Team Meeting (1/29/2010),Wheelock Mtg. (2/2/2010). • QRIS Provisional Standards Approved by EEC Board (2/17/2010) • EEC held five regional forums to allow providers the opportunity to hear more about QRIS initiatives, including the Pilot and the grant. (March - April 2010) • To assist in piloting the QRIS ~$3.3M in quality improvement funds were made available for grants of up to $10,000 per program. (March 2010) 10 April – July • QRIS grantees (640) had opportunity to use and “test out” the QRIS Provisional Standards. QRIS grantees used fund to engage in quality improvement activities to move to next level in the standards. (April June 2010) • 840 programs and providers completed a QRIS Application and self-assessment, using the Provisional standards during the piloting phase. • EEC purchased and made available Environment Rating Scale Tools (ECERS, ITERS, FCCERS, SACERS) • EEC begins development of QRIS Web-based Application development January – October 2010 Aug - Sept • EEC contracted EDC to evaluate the QRIS Provisional Standards and engage in a series of activities to make standards more streamlined, less duplicative, informed by research. • EEC through EDC conducted crosswalks of 10 observational measurement tools, Head Start Performance Standards and 3 national accreditation standards (NAEYC, NAFFC, COA) (Sept) • Together for Quality grant to manage FY 11 Grant process, provide training and TA, and conduct monitoring for FY 10 grantees Grantee awarded 3.4M • EEC was selected to participate in the QRIS Learning Table State Network QRIS Standards Revision Process and Stakeholder Feedback Principles Guiding Standards Revision Process Standards already required by the Massachusetts licensing regulations were eliminated Standards were eliminated that: Lacked a strong research base b) Do not have an objective basis for providing documentation c) Are not aligned with existing standard measures d) Are not in line with best practice as articulated by stakeholders and in other states’ QRISs a) 12 Standards were collapsed into categories when documentation is the same for multiple standards QRIS Standards Revision Activities 13 Gathered input from EEC stakeholders on QRIS revision process and proposed revisions (October – Nov 2010) Proposed revisions were posted to EEC website and QRIS standards survey was posted. (Nov 2010) Planning and Evaluation Committee reviewed evidence and made recommendations to Board regarding Workforce Professional Development standards (e.g. should standards be individually focused or focused on program level quality (Nov 2010) Additional revisions to Workforce and Professional development standards were made informed by additional research and stakeholder feedback. Revisions posted on EEC website and QRIS standards survey, was updated to gather additional feedback. (Nov 2010) Presented revisions process, Proposed Revised QRIS Standards to key stakeholders, gathered feedback on standards, measures, and documentation at meeting at Wheelock College. (11/30/2010) EEC disseminated emails to ~28K providers listed in the Registry and encouraged programs to review proposed revisions to Provisional Standards via survey. (Dec) Opportunities for Stakeholder Feedback 5 Regional Forums 2 Conference Calls Focus Groups & Interviews QRIS Standards Survey QRIS Dialogue Meeting 14 Over 400 individuals participated in Regional Forums 30 telephone interviews completed Over 775 surveys completed to date 50 individuals in attendance at QRIS Dialogue Wheelock (11/30/2010) Stakeholder Feedback: Key Themes Identified Comments about standards revision process Concerns that some standards may be challenging for programs and providers to achieve Requests for clarification of expectations related to measurement tools and documentation requirements Inquiries about Program Support and Professional Development, Implementation, and Communications 15 QRIS Standards Survey Do the Proposed QRIS Standards Measure Quality? Percent of Respondents that Strongly Agree or Agree that the Proposed Revisions to Center/School-based Standards Measure Quality 100 Strongly Agree 90 Agree 80 70 60 44 66 50 39 50 47 45 42 42 37 40 30 20 10 0 17 39 36 24 1A. Curriculum, 1B. TeacherAssessment, Child and Diversity Relationships and Interactions 27 28 31 37 31 37 2. Global 3A. 3B. Program 4. Family 5A. 5B. Supervision 5C. Community Environment Administrator Staff Involvement Administration Involvement Qualifications Qualifications Management and and and and Leadership Collaboration Professional Professional Development Development Percent of Respondents that Strongly Agree or Agree that the Proposed Revisions to Family Child Care Standards Measure Quality 100 Strongly Agree Agree 90 80 70 60 47 59 60 41 50 32 38 32 31 40 33 40 30 20 36 32 36 10 0 18 1A. Curriculum, 1B. TeacherAssessment, Child and Diversity Relationships and Interactions 2. Global Environment 25 3A. Family Child Care Educator and Professional Development 29 35 4. Family 5A. 5B. Supervision 5C. Community Involvement Administration Involvement Management and and Leadership Collaboration Percent of Respondents that Strongly Agree or Agree that the Proposed Revisions to ASOST Standards Measure Quality 90 Strongly Agree 80 Agree 70 60 27 42 41 50 35 24 19 41 40 32 37 34 30 20 42 37 31 31 38 37 10 0 19 1A. Curriculum, 1B. TeacherAssessment, Child and Diversity Relationships and Interactions 2. Global 3A. Program 3B. Site 3C. Group Environment Administrator Coodinator Leader Qualifications Qualifications Qualifications and and and Professional Professional Professional Development Development Development 25 25 32 33 4. Family 5A. 5B. Supervision 5C. Community Involvement Administration Involvement Management and and Leadership Collaboration Percent of Respondents that Strongly Agree or Agree that the Proposed QRIS Standards Measure Quality All QRIS Standards 100 Center/ School Based Family Child Care After School/ OST 90 80 70 60 50 40 90 91 83 83 69 30 89 79 75 77 72 82 78 77 66 58 65 62 66 56 73 64 59 79 69 64 75 70 20 10 0 20 1A. 1B. Teacher2. Global 3A. 3B. Program 3C. Group 4. Family 5A. 5B. Curriculum, Child Environment Administrator Staff Leader Involvement Administration Supervision Assessment, Relationships Qualifications Qualifications Qualifications Management and Diversity and and and and and Leadership Interactions Professional Professional Professional Development Development Development 5C. Community Involvement and Collaboration Stakeholder Responses to Revised Standards 21 Vast majority of survey respondents agree that new standards reflect quality early education and care. Some suggestions for changing standards were proposed. Analysis reveals that providers are less likely to agree that standards reflect quality when the standards are challenging to achieve. Questions exist about how to reach standards. Like the Provisional QRIS Standards, the proposed QRIS Standards still: 22 Contain 5 categories, which are customized for each QRIS program type with five levels to measure quality within in each category and uses a block system. Have strong language for the use program improvement plan (based upon self-assessment findings) and individualized professional development plans (IPDP). Are measured as being met by a set of specific criteria, such as having a license in good standing, verification of professional development and Educator qualifications in the PQ registry, the use of the observational tools, ECERS-R, ITERS-R, FCCERS-R, SACCERS, and APT), submission of related documentation, and an on-site verification with the ERS tools. Include Head Start Performance Standards and Accreditation as an option to demonstrate how a program meets certain standards Offer an opportunity to request an exemption for one standard. Highlights: Proposed Revisions Made In Response to Feedback Safe, Healthy Indoor and Outdoor Environments standards include revised language emphasizing the importance of healthy, safe, and nurturing environments. Sinks for hand washing are reincorporated to address recommended health practices associated with reducing poor health outcomes. Workforce and Professional Development revised to focus on the program level quality vs. individuals. Family & Community Engagement Standards were updated to included community involvement standards and clarify the roles of Educators in making comprehensive services. New language recognizing community collaboration as a pathway to quality. Programs and educators are encouraged to use existing networks, such as CFCE programs, family child care networks, Head Start partnerships, as a resource for supporting children and families. Leadership, Management, and Administration standards now address program staff retention and paid staff planning time. Fiscal audit criteria is revised to include roles other than CPA’s qualified to conduct a 3rd party fiscal review. 23 Highlights: Proposed Revisions Made In Response to Feedback Supervision Standards have been enhanced to better incorporate the importance of reflective supervision, and the role of peers, mentors, and coaches. After School/ Out of School Time Standards have been revised to reflect the diversity found in programs (programs purpose, and unique workforce qualities). Center/ School Based are designed so that 100% of classrooms will have at least one Educator with a B.A. in related field by level 4 Family Child Care Standards criteria for site visits by B.A. level staff have been adjusted from weekly visits to two visits per month. 24 Summary of Comments and Responses Curriculum, Assessment & Diversity Concerns were raised about the frequency that progress reports should be completed and expectations to engage in formative assessment practices. EEC acknowledges the importance of having a clear balance between time focused on teacher and child interactions to support learning and time spent assessing children’s developmental progress and learning to The use of child assessment systems and screening in early childhood settings is an important method to support developmentally appropriate individualized teaching and is an integral piece of high quality programming. Family and Community Engagement Concerns were articulated that the additional opportunities for sharing progress reports with parents were too frequent; offering family support was not perceived as central to role nor needed by all families, and developing written collaborative agreements seemed burdensome. As a Strengthening Families Affiliate, Massachusetts has been working to build the protective factors known to reduce child abuse and neglect. Strategies that facilitate children’s social and emotional development, increase parent’s understanding of their own child’s development, and help link families to services and opportunities are known to build these protective factors, and considered indicators of quality. 25 Summary of Comments and Responses Health Standards Some stakeholders shared that the health standards seemed lost and needed their own subcategory again. MA has strong health and safety requirements in the licensing regulations. A separate health standard would be redundant as it is also covered in the Environment Rating Scales and is aligned with other measures. Health Care Consultants ® Several stakeholders had strong concerns requiring annual consultation visits (Level 2). Concerns included health consultants role and related expenses. Both the national health and safety standards and NAEYC identify child care health consultation as an important component of a high quality ECE program. NAEYC accreditation criteria (visit at least 2 times a year, and 4 times for I/T). 26 Summary of Comments and Responses Environment Rating Scales (ERS) vs. Accreditation Many stakeholders voiced concerns about the use and function of national accreditation in the standards. ERS systematically examine global quality and allow programs and external stakeholders to view data on each variable related to quality ERS data allow programs and stakeholders to examine comparable elements of quality across programs MA recognizes that accreditation presents a useful measure of quality, but does not allow programs or stakeholders to compare programs 27 Point vs. Block System Additional comments about the selected block rating structure were offered (i.e., building blocks, points, combination of block and points). In 2009, the decision to use the block system was made requiring that programs meet the expectations of all criteria in in all categories of standards. The proposed QRIS Standards will continue to use the block structure, as it offers a consistent system for demonstrating and measuring quality. Proposed Massachusetts Quality Rating and Improvement System Standards (QRIS Standards) Features of Proposed QRIS Standards Revised QRIS Standards Are Above and Beyond Licensing = Quality Many higher levels exceed criteria in other states’ QRIS Each level reflects increasing levels of quality Based on strong research ® Aligned with other existing measures In line with best practice In other states’ Quality Rating and Improvement Systems Articulated by stakeholders 29 Proposed Structure of the Standards Level Revised Standard Required Observation Measure (ERS) Additional Required Observation Measure Required Documentation Head Start Documentation Option Accreditation Program Documentation Option Level: Follows the existing structure of block system. (Each program will still need to meet all requirements of standard of the proceeding level before advancing to the next “level”). Revised Standard: Using the principles guiding the revision process, these are the current standards presently referred to as the Proposed Revisions to the Provisional QRIS Standards or proposed Provisional QRIS Standards (revised). Required Observation Measure: This column includes measurement tools that will be required by all QRIS participants regardless of program type, or accreditation status, to ensure consistent of measurement tools across program type. Additional Required Observation Measure: This column has been added, to supplement the required tool, to effectively measure additional process (teacher-child Interactions) and Structural (leadership & program administration indicators of quality. Required Documentation: Materials that will be reviewed by EEC as “evidence” of meeting the Standard/ Measurement (i.e. evidence in PQ Registry, demonstrated used of the MA Curriculum Guidelines, and other MA specific requirements). Head Start Documentation Option: This column lists the related Head Start Performance Standard, and the required documentation that a Head Start program submits (i.e. MA specific and/or not addressed via Head Start Performance Standards). Accreditation Program Documentation Option: This column list the related accreditation standard and the required documentation that an accredited program will have to submit )i.e. MA specific and/or not addressed via the 30accreditation standards. Example: Standards Structure with Documentation Level Level 3 Revised Standard Meets Requirements of Level 2 PLUS A daily two way communication system is available between the educators and families through a variety of means. Families are encouraged to volunteer in the program, to assist in the classroom, and share cultural and language traditions or other interests such as their jobs, hobbies and other relevant information. ® Program ensures that there are translators available, as needed, at meetings, workshops and conferences to ensure strong communication between the program and families. Program participates in local community group work that is related to early childhood, and the cultural groups served by the program and/or family support. ® Program ensures young children and their families have access to developmental, mental health, health and nutrition services either through private pay arrangements OR are offered such services through other programs. ®+ 31 Required Observation Measure (ERS) ITERS-R AND/OR ECERS-R reliable rater score average of 5 with no single item below 4 Additional Required Observation Measure Program Administration Scale (PAS) score of 5 or higher by a reliable rater. Required Documentation Document signed by program administrator describing the variety of daily communication methods (e.g. scheduled telephone hour, checklists, e-mail). AND Document signed by program administrator describing translators used for all meetings workshops and conferences. AND Document signed by program administrator describing how the program ensures children and their families have access to developmental, mental health, health, and nutrition services either through private pay arrangements OR are offered such services through other programs (such as, CFCE program, mental health providers, health care providers, etc. Head Start Documentation Option Head Start item # 1304.41(a)(4) 1304.51(c)(1) 1304.51(c)(2) Accreditation Program Documentation Option NAEYC item # 7.B.01 7.B.05 7.A.07 8.A.01 8.A.02 Level 3 Family & Community Engagement (Center-based School-based) Examples of Scaffolding: Family Engagement Standard Level 5: TBD Level 4: Parents participate on the Advisory Board for the program and are actively involved in the policy and decision making for the program. Level 3: Families are encouraged to volunteer in the program, to assist in the classroom, and share cultural and language traditions or other interests (such as their jobs, hobbies and other relevant information). ® Level 2: Parents are offered opportunities to meet with classroom staff, at least monthly. Level 1: Meets licensing standards 32 Example of Scaffolding: Program Staff Qualifications and Professional Development Center/School Based Standard Level 5: TBD Level 4: All (100 percent) of the classrooms have Educator(s) with a Bachelor’s degree or higher who work for the full program day.®+ Level 3: 75 percent of classrooms have Educator(s) with a Bachelor’s degree or higher who work for the full program day. Level 2: 50 percent of classrooms have Educator(s) with a Bachelor’s degree or higher who work for the full program day.® All educators have high school diploma or GED and a minimum of 3 college credits in early childhood education, or related field. Level 1: Meets licensing regulations 33 QRIS Measures and Documentation Research and Principles Guiding Measurement Tools Selection and Documentation Expectations Reviewed research and state QRIS and found: • Environment Rating Scales (ECERS-R, ITERS-R, FCCERS-R, SACERS) used by many states and supported by research. • Other observation tools used by other states and supported by research: PAS, BAS, APT, Arnett CIS, and CLASS Reviewed Massachusetts standards and other measures and found: • ERS aligned with many measures • PAS, BAS, APT, CIS, and CLASS aligned with some measures • Accreditation aligned with many standards and used by some states • Head Start program performance standards aligned with many standards 35 Focus on measurable and doable documentation. MA QRIS Observation Tools and Documentation 36 Require ERS self-study for level 2 and outside reliable raters for levels 3 and 4 for all QRIS Program types Require ERS, PAS, BAS, APT. Require as CLASS or Arnett Caregiver Interaction Scale to assess teacher/child interactions. Require Strengthening Families self-assessment tool (Level 2) Requires specific list of documentation, not reflected in Observational tools (i.e. Use of Ma Preschool and Infant Toddler Guidelines in Curriculum, documentation of workforce qualifications and Professional development in the PQ registry). Accredited and Head Start programs and providers are provided information about alignment, that is “standard-specific”. Environment Rating Scales Program Quality Assessment Instrument Rates 39 (ITERS) 43 (ECERS), 38 (FCCERS), 49 (SACERS) areas of analysis under the following 7 subscales: Center / School- Based ITERS-R Family Child Care ECERS-R Out of School /Afterschool Programs FCCERS-R SACERS Space and furnishings Space and furnishings Space and furnishings Space and Furnishings Personal care routines Personal care routines Personal care routines Health and Safety Listening and Talking Language and reasoning Listening and Talking Supplementary Items (for Activities Activities Activities Activities Interactions Interactions Interactions Interactions Program Structure Program Structure Program Structure Program Structure Parent and staff Parent and staff Parent and staff Staff Development 4 6 children with special needs) Ratings range from 1 to 7: 1 37 Inadequate 2 3 Minimal 5 Good 7 Excellent Next Steps Key Administrative Decisions Process/schedule to revisit the Standards for revision to ensure they are informed by current research and best practice Recommendation for Discussion: Every three years Acceptable frequency of use of observation tools/self assessments Recommendation for Discussion: program must have completed the observation measurement tool within a year of QRIS application submission date Process for communicating ratings to the public during FY2011 and beyond Recommendation for Discussion: EEC to post FY2011 ratings on EEC website (participants will be notified that this information will be shared publically) Parties that will be allowed to complete external ratings for programs Recommendation for Discussion: Initially Staff trained as raters, EEC to discuss with Planning and Evaluation Committee and analyze potential resources and options to develop a recommendation. Acceptable Criteria for Equivalent Qualifications and Professional Development to support program and educator participation in 2011. Recommendation for Discussion: EEC to discuss with Planning and Evaluation Committee recommendations for grandfathering criteria for early educators during the first year of implementation for educators employed as of January 2011 39 Key Administrative Decisions How does EEC handle applications for programs that are in non-compliance with licensing? Recommendation for Discussion: EEC will analyze the various levels of non-compliance and make a recommendation How long does a program keep a rating? Recommendation for Discussion: Up to 2 years, then revisit the expectation to advance a level or demonstrate pathways to advancement if a key quality indicator changes for the program (e.g. if accreditation is revoked, expires etc. Can program apply more often (before their rating expires); EEC proposes that 6 months from time of verification, programs may resubmit a QRIS application. The role of Family Child Care Systems in QRIS while maintaining direct family child care educator participation (how they work with their providers in the application process; are there agreements/policies EEC needs to develop to ensure programs are active in the process; do Systems maintain their fiscal responsibility) Recommendation for Discussion: Systems can be involved in the application process, but the programs must be directly involved with their application and understand how their program can make advancements on the QRIS 40 Implementation Implementation 42 QRIS Program Manager a web-based, electronic QRIS Application Process that enables programs to apply and manage their QRIS application, includes interface for verifying professional qualifications as submitted in the PQ Registry ( Anticipated Launch Jan. 2011) Communications Activities include the development of QRIS FAQs and guidance for prospective QRIS participants Programs Supports 6 Regional Educator and Provider Support (EPS) Grantees to provide coaching and mentoring, competency development and intentional professional development 6 Regional EOE Readiness Centers to provide coordination between early education and care, elementary and secondary education and higher education EEC Initiatives to build the system and support the programs and educators include: Core Competencies Professional Qualification Registry Infant Toddler Physical Environments Environmental Rating Scale (ERS) Training Course development • Literacy • Core Competencies • Preschool Guidelines • Formative Assessments 43 Together for Quality (T4Q) 44 The Department of Early Education and Care (EEC) will provide general oversight and planning support, ensure broad, statewide access to funding and services, and monitor the quality and implementation of T4Q. The United Way has started to monitor and report on a sample of FY2010 QRIS Program Improvement grants and is developing a 2011 QRIS Program Improvement grant process that will award between 300-600 grants to programs in EEC’s mixed delivery system statewide. (Grant funds must be expended by Aug 2011.) The CAYL Institute will provide preliminary training in the QRIS system to programs in EEC’s mixed delivery system statewide. There will be two trainings in each region. The Wheelock College Aspire Institute will serve as the lead program agency, coordinating efforts across partners, as well as oversee a Quality Coach program that will recruit, train and place 26 Quality Coaches to work with a sample of programs receiving QRIS Program Improvement grants.