Getting Started: Educator Evaluation in Non-RTTT Districts Please sit with your district or school team members You will be talking with them during today’s workshop Question: What opportunities will this new educator evaluation framework provide for professional growth and student learning in your district or school? 3 Massachusetts Department of Elementary and Secondary Education Agenda Welcome & Introductions Evaluation Framework: What sets Massachusetts apart? 5-Step Cycle of Evaluation Guest Speaker: What does this look like on the ground? BREAK Communications: Opportunity for Coherence Collective Bargaining and Reporting Requirements District-Determined Measures Training 4 Team Time Wrap-Up Massachusetts Department of Elementary and Secondary Education Priorities of the new evaluation framework Place Student Learning at the Center – Student learning is central to the evaluation and development of educators Promote Growth and Development – Provide all educators with feedback and opportunities that support continuous growth and improvement through collaboration Recognize Excellence – Encourage districts to recognize and reward excellence in teaching and leadership Set a High Bar for Tenure – Entrants to the teaching force must demonstrate Proficient performance on all standards within three years to earn Professional Teacher Status Shorten Timelines for Improvement – Educators who are not rated Proficient face accelerated timelines for improvement We want to ensure that each student in the Commonwealth is taught by an effective educator, in schools and districts led by effective leaders. Massachusetts Department of Elementary and Secondary Education 5 The ESE Educator Evaluation Team 6 Massachusetts Department of Elementary and Secondary Education Our Mission To improve professional growth and student learning, ESE is committed to ensuring the success of the statewide Educator Evaluation framework by providing educators with training materials and resources, meaningful guidance, and timely communications, and by engaging educators in the development and ongoing refinement of the framework. 7 Massachusetts Department of Elementary and Secondary Education Three Key Strategies Teach Learn Connect Teaching the components of the Educator Evaluation framework and sharing implementation resources to build capacity within districts and schools. Learning from and with educators about their successes, challenges, and needs to ensure educator voices are reflected in Educator Evaluation policies and practices. Connecting and aligning Educator Evaluation implementation with other state and district initiatives to improve professional growth and student learning; Creating opportunities for educators to connect and share with one another and ESE. Massachusetts Department of Elementary and Secondary Education 8 ESE Educator Evaluation Team Claire Abbott, Evaluation Training Program, Implementation Support, Resource Development Kathryn Gabriele, Staff & Student Feedback, District-Determined Measures, Data Collection and Reporting Kat Johnston, Communications, Peer Assistance & Review, Implementation Support Simone Lynch, Assistant Director, Office of Educator Policy, Preparation and Leadership Ron Noble, Project Co-Lead, Evaluation System Reviews, DistrictDetermined Measures, Staff & Student Feedback Samantha Warburton, Project Co-Lead, Evaluation Training Program & 9 Vendors, Implementation Support, Resource Support Massachusetts Department of Elementary and Secondary Education Ed Eval Framework: Timeline June 2011: MA Board of Education passed new regulations September 2011: Implementation began in 34 Level 4 schools, 11 Early Adopter districts, and 4 special education collaboratives January 2012: ESE published the Massachusetts Model System for Educator Evaluation September 2012: Implementation began in all RTTT districts ----------------------------------------------------------------------------------- September 2013: All districts implement educator evaluation 10 Massachusetts Department of Elementary and Secondary Education ESE Resources ESE Model System Teacher & Administrator Contract Language School & District Implementation Guides 4 Model Performance Rubrics ESE Training Materials Modules & Workshops Additional Resources & Supports Forms, guidance documents, webinars, presentations, newsletter, approved vendors Massachusetts Department of Elementary and Secondary Education 11 ESE Ed Eval Website More information: http://www.doe.mass.edu/edeval Questions: EducatorEvaluation@doe.mass.edu 12 Massachusetts Department of Elementary and Secondary Education Intended Outcomes Participants will: Know more about the new educator evaluation system and what it will mean to introduce it to your district over the course of the next year Know more about available ESE resources, and how to use them back at your district Be familiar with the implementation timeline for Year 1, including training and reporting requirements Have at least one clear, agreed upon “next step” for action back in your district Massachusetts Department of Elementary and Secondary Education 13 Agenda Welcome & Introductions Evaluation Framework: What sets MA apart? 5-Step Cycle of Evaluation Guest Speaker: What does this look like on the ground? BREAK Communications: Opportunity for Coherence Collective Bargaining and Reporting Requirements District-Determined Measures Training 14 Team Time Wrap-Up Massachusetts Department of Elementary and Secondary Education What sets Massachusetts apart? Two separate ratings Three types of evidence Four common Standards Educator Evaluation 15 Massachusetts Department of Elementary and Secondary Education Agenda Welcome & Introductions Evaluation Framework: What sets Massachusetts apart? 5-Step Cycle of Evaluation Guest Speaker: What does this look like on the ground? BREAK Communications: Opportunity for Coherence Collective Bargaining and Reporting Requirements District-Determined Measures Training 16 Team Time Wrap-Up Massachusetts Department of Elementary and Secondary Education Every educator is an active participant in the Step 2: SLG: ELL students evaluation process master content Step 1: Incoming ELL students underperformed in Math & ELA; Indicator II.A: Instruction was an area in need of impr. for ELLs. Step 5: Tom was rated Proficient on Standards II, III & IV, and Exemplary on I, and met or partially met goals. Overall Summative Rating: Proficient. standards across 3 units. PPG (TEAM): Identify & pilot 3 instructional strategies for improving comprehension. Continuous Learning Step 3: Artifacts (lesson plans, team meeting notes, teacher/parent communication); student work, Step 4: Evidence showed pre/post lab reports, rising ELL student writing assessments,; performance + success w/ 2 17 4 observations w/ new instructional feedback; student strategies; discussed additional outreach to Collaboration and Continuous Learning are the focusfeedback survey in mid-spring) parents re: homework. Massachusetts Department of Elementary and Secondary Education Step 1: Self-Assessment Educators self-assess their performance using: Student data, and Performance rubric Educators propose goals related to their professional practice and student learning needs 18 Part II: School Level Guide (p. 14-22) Module 3 and Workshop 2: Self-Assessment Massachusetts Department of Elementary and Secondary Education Step 2: Analysis, Goal Setting and Plan Development Educators set S.M.A.R.T. goals: Student learning goal Professional practice goal (Aligned to the Standards and Indicators of Effective Practice) Educators are required to consider team goals Educator and Evaluator develop the Educator Plan Part II: School Level Guide (p. 23-31, Appendix B: Setting S.M.A.R.T Goals) Massachusetts Department of Elementary and Secondary Education 19 Step 2: Analysis, Goal Setting and Plan Development The educator plan is based on S.M.A.R.T. goals. 20 Step 3: Implementation of the Plan Educator completes the planned action steps of his/her plan Educator and evaluator collect evidence of practice and goal progress, including: 1. Observations and artifacts 2. Multiple measures of student learning 3. Additional evidence related to performance standards Evaluator provides feedback Part II: School Level Guide Pages 32-39 Massachusetts Department of Elementary and Secondary Education 21 Three Types of Evidence 1. Products of Practice Artifacts related to educator practice Samples that demonstrate educator performance and impact Number to collect varies by educator Observations of practice At least one unannounced Frequent & brief Constructive feedback 22 Module 5 and Workshop 4: Gathering Evidence Massachusetts Department of Elementary and Secondary Education Three Types of Evidence 2. Multiple Measures of Student Learning Evidence of Progress toward educator goals Evidence of Performance associated with one or more Standards 23 Module 5 and Workshop 4: Gathering Evidence Massachusetts Department of Elementary and Secondary Education Three Types of Evidence 3. Other Evidence Related to one or more Performance Standards Student feedback informs the Summative Performance Rating of ALL educators Staff feedback informs the Summative Performance Rating of all administrators ESE will publish guidance and model student and staff survey instruments by July 1, 2013. For more information, please contact Kathryn Gabriele at kgabriele@doe.mass.edu. Module 5 and Workshop 4: Gathering Evidence Massachusetts Department of Elementary and Secondary Education QRG: Staff & Student Feedback 24 Step 4: Formative Assessment/ Evaluation Occurs mid-way through the 5-Step Cycle Typically Jan/Feb for educators on a 1-year plan (formative assessment) Typically May/June for educators on a 2-year plan (formative evaluation) Educator and Evaluator review evidence and assess progress on educator’s goals Part II: School Level Guide Pages 40-47 Massachusetts Department of Elementary and Secondary Education 25 Step 5: Summative Evaluation Summative Evaluation results in: Individual ratings on each of the four Standards Assessment of overall goal progress Overall Summative Performance Rating Evaluator determines the Summative Performance Rating based on: Comprehensive picture of practice captured through multiple sources of evidence Professional Judgment Part II: School Level Guide Pages 48-53 Massachusetts Department of Elementary and Secondary Education 26 Multiple sources of evidence inform the summative rating 27 Massachusetts Department of Elementary and Secondary Education Summative Rating Determining Your Educator Plan Exemplary Proficient 1-yr Self-Directed Growth Plan 2-yr Self-Directed Growth Plan Needs Improvement Directed Growth Plan (1 yr or less) Unsatisfactory Improvement Plan (30 days to 1 yr) *Developing Educator Plan: for new teachers & administrators 28 Massachusetts Department of Elementary and Secondary Education 5 Step Evaluation Cycle Continuous Learning Performance Rubrics Self-Assessment Measurable Goals Brief, frequent observations Evidence Collection Regular, timely feedback 29 Massachusetts Department of Elementary and Secondary Education Agenda Welcome & Introductions Evaluation Framework: What sets Massachusetts apart? 5-Step Cycle of Evaluation Guest Speaker: What does this look like on the ground? BREAK Communications: Opportunity for Coherence Collective Bargaining and Reporting Requirements District-Determined Measures Training 30 Team Time Wrap-Up Massachusetts Department of Elementary and Secondary Education 10-minute BREAK 31 Massachusetts Department of Elementary and Secondary Education Agenda Welcome & Introductions Evaluation Framework: How is Massachusetts different? 5-Step Cycle of Evaluation Guest Speaker: What does this look like on the ground? BREAK Communications: Opportunity for Coherence Collective Bargaining and Reporting Requirements District-Determined Measures Training 32 Team Time Wrap-Up Massachusetts Department of Elementary and Secondary Education Early Learnings Comprehensive, transparent communications strategies across all educators are critical to implementation success in Year One (Early Adopters & Level 4 districts) Key stakeholders view new evaluation system positively and believe it is a significant improvement (3rd party evaluator) Establishing coherence with other initiatives plays key role in making this “meaningful” to educators (Early Adopters & Level 4 districts) Massachusetts Department of Elementary and Secondary Education 33 Opportunity for Coherence I. Curriculum, Planning, & Assessment II. Teaching All Students III. Family & Community Engagement A. Curriculum and Planning A. Instruction A. Engagement 1. Subject Matter Knowledge 1. Quality and Effort of Work 1. Parent/Family 2. Child and Adolescent 2. Student Engagement Engagement Development 3. Meeting Diverse Needs 3. Rigorous Standards-Based B. Collaboration B. Learning Environment Unit Design 1. Learning Expectations 1. Safe Learning 4. Well-Structured Lessons 2. Curriculum Support Environment 2. Collaborative Learning B. Assessment C. Communication Environment 1. Variety of Assessment 1. Two-Way Communication 3. Student Motivation Methods 2. Culturally Proficient 2. Adjustments to Practice C. Cultural Proficiency Communication 1. Respects Differences C. Analysis 2. Maintains Respectful 1. Analysis and Conclusions Environment 2. Sharing Conclusions with D. Expectations Colleagues 1. Clear Expectations 3. Sharing Conclusions with 2. High Expectations Students 3. Access to Knowledge IV. Professional Culture A. Reflection 1. Reflective Practice 2. Goal Setting B. Professional Growth 1. Professional Learning and Growth C. Collaboration 1. Professional Collaboration D. Decision-making 1. Decision-Making E. Shared Responsibility 1. Shared Responsibility F. Professional Responsibilities 1. Judgment 2. Reliability and Responsibility Massachusetts Department of Elementary and Secondary Education Massachusetts Department of Elementary and Secondary Education 34 Communications What strategies are you using (or could you use) to communicate opportunities and expectations to teachers, administrators, union leaders, and school committee members for your district’s implementation? What’s the success story you want to tell five years from now? 35 Massachusetts Department of Elementary and Secondary Education Communications Tips & Strategies Establish a collaborative Educator Evaluation working group comprised of diverse stakeholders Commit to regular two-way communications (ex. monthly newsletter, Wiki page, regular and open channels to provide feedback) Develop a strategic communications plan that includes key messages, timeline, and available resources so educators are continually kept up to date and involved in the process Massachusetts Department of Elementary and Secondary Education 36 Agenda Welcome & Introductions Evaluation Framework: What sets Massachusetts apart? 5-Step Cycle of Evaluation Guest Speaker: What does this look like on the ground? BREAK Communications: Opportunity for Coherence Collective Bargaining and Reporting Requirements District-Determined Measures Training 37 Team Time Wrap-Up Massachusetts Department of Elementary and Secondary Education Collective Bargaining & Reporting Requirements The procedures for conducting educator evaluation are a mandatory subject of collective bargaining in Massachusetts. ESE Resources: Model Contract Language for Teachers (Unit A) (plus annotated version) Model Contract Language for School Administrators (Unit B) District Options: Adopt, Adapt, Revise Districts are encouraged to conduct bargaining in a way that permits the parties to return to educator evaluation periodically over the next several years. Part IV: Model Collective Bargaining Contract Language Massachusetts Department of Elementary and Secondary Education (Appendix C and Appendix D) 38 Collective Bargaining & Reporting Requirements Highlights from RTTT District Implementation RTTT Districts approached contract language concerning educator evaluation in several ways, including: Detailed process included in collective bargaining agreements Some process in the contract and some in side letters or other documents (e.g., guidebooks, manuals) MOU/MOA outlining the districts’ decisions to adopt the model contract language with little to no modifications RTTT Districts Adoption Rate Adaption Rate approx. 25% approx. 60% Approximately 85% of RTTT Districts used the model contract language as their starting point. Massachusetts Department of Elementary and Secondary Education 39 Collective Bargaining & Reporting Requirements Highlights from RTTT District Implementation: Benefits to convening joint labor-management working groups early in the process included: Abbreviated negotiations leading to timely implementation Consistent understanding and appreciation of the framework at the district and school levels Districts unable to reach agreement by Fall 2012 found that some elements of the system were compromised or rushed. Massachusetts Department of Elementary and Secondary Education 40 Collective Bargaining & Reporting Requirements Districts must notify ESE of decision to adopt, adapt, or revise Adapted or revised systems must be submitted for ESE review Submissions must include: Teacher Contract Language or comparable document Administrator (Unit B) Contract Language or comparable document School- and District-Administrator Evaluation Protocol Performance Rubrics Target Submission Date: September 1, 2013 Massachusetts Department of Elementary and Secondary Education 41 Collective Bargaining & Reporting Requirements Key regulatory components subject to review: 1. 5-step evaluation cycle 2. Standards and Indicators of Effective Practice 3. Three Categories of Evidence 4. Four Performance Rating Levels 5. Four Types of Educator Plans 6. Educators rated as having a high, moderate, or low impact on student learning based on trends and patterns in student performance measures including MCAS and districtdetermined measures. Questions? Please contact Ron Noble (rnoble@doe.mass.edu) Massachusetts Department of Elementary and Secondary Education 2-Page Overview of Educator Evaluation Regulations 42 Agenda Welcome & Introductions Evaluation Framework: What sets Massachusetts apart? 5-Step Cycle of Evaluation Guest Speaker: What does this look like on the ground? BREAK Communications: Opportunity for Coherence Collective Bargaining and Reporting Requirements District-Determined Measures Training 43 Team Time Wrap-Up Massachusetts Department of Elementary and Secondary Education Student Impact Rating The Student Impact Rating must be based on at least 2 years of data across multiple measures: State assessments that measure growth (ex. MCAS student growth percentiles) District-determined measures 44 Part VII: Rating Educator Impact on Student Learning Massachusetts Department of Elementary and Secondary Education Using District-Determined Measures of Student Learning Summative Rating Two Ratings Exemplary Proficient 1-yr Self-Directed Growth Plan 2-yr Self-Directed Growth Plan Needs Improvement Directed Growth Plan Unsatisfactory Improvement Plan Low Moderate High Rating of Impact on Student Learning Massachusetts Department of Elementary and Secondary Education 45 District-Determined Measures Definition from the regulations: “Measures of student learning, growth, and achievement related to the Massachusetts Curriculum Frameworks, Massachusetts Vocational Technical Education Frameworks, or other relevant frameworks, that are comparable across grade or subject level district-wide. These measures may include, but shall not be limited to: portfolios, approved commercial assessments and district-developed pre and post unit and course assessments, and capstone projects.” 603 CMR 35.02 Massachusetts Department of Elementary and Secondary Education 46 District-Determined Measures DDMs should measure growth, not just achievement. Assessments should be administered across all schools in the district where the same grade or subject is taught. Districts must use measures of growth from state assessments where they are available. Only applicable to fewer than 20% of educators 47 Massachusetts Department of Elementary and Secondary Education Priorities of the new evaluation framework Place Student Learning at the Center – Student learning is central to the evaluation and development of educators Promote Growth and Development – Provide all educators with feedback and opportunities that support continuous growth and improvement through collaboration Recognize Excellence – Encourage districts to recognize and reward excellence in teaching and leadership Set a High Bar for Tenure – Entrants to the teaching force must demonstrate Proficient performance on all standards within three years to earn Professional Teacher Status Shorten Timelines for Improvement – Educators who are not rated Proficient face accelerated timelines for improvement We want to ensure that each student in the Commonwealth is taught by an effective educator, in schools and districts led by effective leaders. Massachusetts Department of Elementary and Secondary Education 48 District-Determined Measures DDMs may inform both an educator’s summative performance rating and impact rating Summative Performance Rating Student Impact Rating Evidence Evidence Products of practice (e.g., observations) Other evidence relevant to one or more of the four Standards of practice (e.g., student surveys) Multiple measures of student learning, growth and achievement, including: o Measures of student progress on classroom assessments o Measures of student progress on learning goals set between the educator and evaluator Trends and patterns in student learning, growth & achievement At least two years of data At least two measures Statewide growth measures, where available (including MCAS SGP) Additional DDMs comparable across schools, grades, and subject matter district-wide Massachusetts Department of Elementary and Secondary Education 49 District-Determined Measures: Implementation Rollout In Sept. 2013, districts will report to ESE: Grades and subjects for implementation of DDMs in 2013-2014; Grades and subjects for piloting DDMs in 2013-2014; Grades and subjects that still lack DDMs, for which districts will research and/or develop measures to pilot in the spring of 2014; ---------------------------------------------------------------------------A plan for determining impact ratings based on DDMs for some educators by the end of the 2014-2015 school year, and all educators by the end of the 20152016 school year Quick Reference Guide: District-Determined Measures Massachusetts Department of Elementary and Secondary Education 50 District-Determined Measures What Support is ESE Providing? Identification of “anchor standards” for a sub-set of grades and subjects Targeted for publication in July 2013 ESE is overseeing the collection and evaluation of quality assessments from MA districts and beyond that will be made available for use as DDMs Exemplar DDMs targeted to be available in July 2013 Quick Reference Guide: District-Determined Measures Massachusetts Department of Elementary and Secondary Education 51 District-Determined Measures What Support is ESE Providing? Assessment literacy webinar series to build district capacity to identify and/or develop DDMs All webinars will be archived and posted Webinar series will focus on resulting in useful products for planning and implementation Supplemental guidance on the selection of DDMs and the process of determining an Impact Rating Technical Guide A (April 2013) will focus on measuring growth and selecting appropriate measures Technical Guide B (Summer 2013) will focus on determining a Student Impact Rating Questions? Please contact Ron Noble (rnoble@doe.mass.edu) Quick Reference Guide: District-Determined Measures Massachusetts Department of Elementary and Secondary Education 52 District-Determined Measures Recommended Next Steps to Creating a District Plan Identify a team of administrators, teachers and specialists to focus and plan the district’s work on district-determined measures. Complete an inventory of existing assessments used in the district’s schools. Identify and coordinate with partners that have capacity to assist in the work of identifying and assessing assessments that may serve as district-determined measures. Quick Reference Guide: District-Determined Measures Massachusetts Department of Elementary and Secondary Education 53 The two ratings work together 54 Massachusetts Department of Elementary and Secondary Education Agenda Welcome & Introductions Evaluation Framework: What sets Massachusetts apart? 5-Step Cycle of Evaluation Guest Speaker: What does this look like on the ground? BREAK Communications: Opportunity for Coherence Collective Bargaining and Reporting Requirements District-Determined Measures Training 55 Team Time Wrap-Up Massachusetts Department of Elementary and Secondary Education Training “An Act Providing for the Implementation of Educator Evaluation Systems in School Districts” http://www.malegislature.gov/Laws/SessionLaws/Acts/2012/Chapter131 Training Requirement for all evaluators and all educators who must be evaluated under framework to receive “evaluation training program developed by the department of elementary and secondary education” Massachusetts Department of Elementary and Secondary Education 56 ESE Training Programs Training Modules for Evaluators Time: 11 hours (minimum) Content: Overview + Modules 2-6 Audience: Evaluators and School Leadership Teams Facilitators: School- or District-level PD providers; vendors www.doe.mass.edu/edeval/training/modules/ Training Workshops for Teachers and SISP Time: 4 hours (minimum) Content: Orientation + Workshops 1-3 Audience: All educators without evaluator responsibilities Facilitators: school-level educators (administrators or teacher leaders) www.doe.mass.edu/edeval/training/teachers/ Massachusetts Department of Elementary and Secondary Education 57 Vendor Support ESE has committed $1 million in RTTT funds to subsidize module training by approved vendors for non-RTTT districts. Subsidies will go directly to approved vendors who then offer services to districts at a proportionately reduced cost. Information on approved vendors and subsidized services will be available at www.doe.mass.edu/edeval/training/vendors. Massachusetts Department of Elementary and Secondary Education 58 Training Reporting Requirements for Non-RTTT Districts: Fiscal year 2014: Districts should designate some, if not all, of their Chapter 70 foundation budget allotment for professional development and Title IIA funds to subsidize the cost of a training program October 1, 2013: Districts must publish an educator evaluation training schedule November 1, 2013: Districts must submit a funding plan for training to ESE via the Title IIA application process Massachusetts Department of Elementary and Secondary Education 59 Training Resources ESE Guide to Educator Evaluation Training Requirements (www.doe.mass.edu/edeval/training/TrainingRequirements.pdf) Quick Reference Guide: Training Requirements (www.doe.mass.edu/edeval/) ESE Training Programs (www.doe.mass.edu/edeval/training/) Approved Vendors (www.doe.mass.edu/edeval/training/vendors/) Questions? Please contact Claire Abbott (cabbott@doe.mass.edu) 7 Massachusetts Department of Elementary and Secondary Education Agenda Welcome & Introductions Evaluation Framework: What sets Massachusetts apart? 5-Step Cycle of Evaluation Guest Speaker: What does this look like on the ground? BREAK Communications: Opportunity for Coherence Collective Bargaining and Reporting Requirements District-Determined Measures Training 61 Team Time Wrap-Up Massachusetts Department of Elementary and Secondary Education Team Time Question Year 1 Strategic Planning & Implementation Worksheet 62 Massachusetts Department of Elementary and Secondary Education Agenda Welcome & Introductions Evaluation Framework: What sets Massachusetts apart? 5-Step Cycle of Evaluation Guest Speaker: What does this look like on the ground? BREAK Communications: Opportunity for Coherence Training Collective Bargaining and Reporting Requirements District-Determined Measures Team Time Wrap-Up Massachusetts Department of Elementary and Secondary Education 63 Intended Outcomes Participants will: Know more about the new educator evaluation system and what it will mean to introduce it to your district over the course of the next year Know more about available ESE resources, and how to use them back at your district Be familiar with the implementation timeline for Year 1, including training and reporting requirements Have at least one clear, agreed upon “next step” for action back in your district 64 Massachusetts Department of Elementary and Secondary Education Plus – Delta What worked? What would make it better? 65 Massachusetts Department of Elementary and Secondary Education Upcoming ESE Resources Guidance on Determining Summative Performance Ratings April 2013 DDM Webinar Series March 14, 2013 – December 5, 2013 DDM Technical Guide A April 2013 DDM Technical Guide B August 2013 Exemplar S.M.A.R.T. Goals Summer 2013 Massachusetts Department of Elementary and Secondary Education 66 Handouts Today’s Powerpoint Presentation 2-page Overview of Key Features of the Regulations Quick Reference Guide: MA Model System for Educator Evaluation 1- and 2-year Evaluation Cycle Quick Reference Guide: Rubrics Quick Reference Guide: Staff & Student Feedback Rubric-At-A-Glance (Teacher & School Administrator) Quick Reference Guide: Training Requirements Quick Reference Guide: DDMs Strategic Planning & Implementation Worksheet Massachusetts Department of Elementary and Secondary Education 67