Ohio Improvement Process (OIP) Draft OIP Facilitator’s Guide December 2008 T A B L E O F C O N T E N T S Methodology Used in Developing the OIP………………………….4 Acknowledgments………………………………………………………5 Background………………………………………………………………7 Purpose of This Guide How to Use This Guide Overview of the Ohio Improvement Process (OIP) Principles for the OIP Documenting the Improvement Process Preparing for the Ohio Improvement Process…………………...14 Introduction Introductory Session With District Superintendent Orientation for District Leadership Team Stage 1: Identify Critical Needs……………………………………..33 Understand Structure and Requirements of the Decision Framework Complete the Decision Framework Identify and Affirm Critical Focus Areas Stage 2: Develop Focused Plan…………………………………….54 Create SMART Goals Develop Research Based Strategies and Indicators Produce Research-Based Actions and Align Resources DRAFT OIP Facilitator’s Guide December 2008 2 of 95 Stage 3: Implement and Monitor the Focused Plan……………..86 Establish and implement collaborative structures, processes and practices that support a culture of inquiry and distributive leadership Implement the plan systemically and systematically Monitor and analyze changes in student performance and adult implementation and make and report course corrections to the plan Stage 4: Evaluate the Improvement Process…………………….87 Evaluate Plan Implementation, Impact, and Changes Report Summative Plan Progress Modify Instructional Practice and Revise Plan Conclusion……………………………………………………………...88 Acronyms……………………………………………………………….89 Glossary…………………………………………………………………90 Bibliography……………………………………………………………94 DRAFT OIP Facilitator’s Guide December 2008 3 of 95 Methodology Used in Developing the OIP The methods, techniques, and resources in this guide were selected through a process that draws from the knowledge and experience of Ohio educators and from information gleaned from a variety of documents from states and organizations. The content of the guide also is based on the collective wisdom of many national, state, regional, and local educators who have led district improvement planning during the past decade. The Decision Framework is the primary tool used in Stage 1 (identifying critical areas for improving student success to determine the highest priority needs). Ohio Department of Education (ODE) staff developed this tool in consultation with a working group composed of ODE staff, higher education staff, Great Lakes East Comprehensive Center, and district central office staff. Initial drafts of the document were presented to focus groups comprised of end users. Their feedback was used to revise the tool. A draft of the Decision Framework and process to use the tool were piloted with 32 districts involved in the State Personnel Development Grant (SPDG) during the 2007–08 and 2008-09 school years. An external work group comprised of Ohio educators and facilitated by Great Lakes East helped guide the development of the conceptual framework for Stage 2. The charge given to this work group from ODE was to develop, pilot, and train providers in the use of a process and associated products for implementing Stage 2 of OIP, the purpose of which is to assist DLTs in the development of a focused plan leading to improved student achievement. The work group used a variety of materials from Ohio, such as the Ohio Department of Education Reference Guide to Continuous Improvement Planning for Ohio School Districts, Second Edition as well as publications from other states and organizations. A draft OIP facilitator’s guide was sent to select Ohio school districts for review and feedback. A structured feedback instrument was used to gain their feedback. The work group reviewed the information, and changes were made to the document. A revised draft of the document then was sent to state department personnel for their review and comment. These changes were incorporated and presented to the single points of contact for the regional service delivery system for their feedback. A State Level Design Team (SLDT) was formed in May 2008 to refine the draft OIP facilitator’s guide based on feedback provided by State Support Team facilitators who implemented the process with the SPDG districts. The SLDT also recommended methods and resources for Stages 3 and 4 and designed and delivered training for OIP facilitators on the OIP process. This document is a work in progress; as additional ideas, resources, methods, techniques, and research become available, the document will be refined and improved. Supplemental resources developed by state support team OIP facilitators to enhance the process are available on the ODE OIP website. DRAFT OIP Facilitator’s Guide December 2008 4 of 95 Acknowledgments Regional providers who served as members of the State-Level Design Team, district staff, ODE personnel, and partners participating in the development of this guide were generous with both their time and attention. Stage 2 Work Group and District Reviewers Clemons, Vikki, Single Point of Contact (SPOC), State Support Team (SST) 13, Cincinnati Csanyi, George, Single Point of Contact (SPOC), State Support Team (SST) 7, Tiffin Humston, Eric, Single Point of Contact (SPOC), State Support Team (SST) 5, Piketon McGlothlin, Russ, Assistant Superintendent, Maysville Local Schools, Zanesville Skelton, Seena, Consultant, State Support Team 13, Cincinnati Washburn Sr., Don, Field Liaison, Ohio Department of Education Zake, Sue, Single Point of Contact (SPOC), State Support Team (SST) 1, Bowling Green Dorr, Constance, Assistant Superintendent, Fostoria City School District Hausterman, Tom, Associate Superintendent, Winton Woods City Schools Lloyd, Jim, Assistant Superintendent, Olmsted Falls City Schools McWilliams, Ellen, Assistant Superintendent, Akron Public Schools State-Level Design Team Region 1: Sue Zake, Brian Davis Region 2: Larry Melia Region 3: Cathy LaForme Region 4: Candice Hazelwood Region 5: Michele DiMuzio, Jane Sadinski Region 6: Marlene Graf Region 7: George Csanyi Region 8: Mary Ellen Murray Region 9: Sue Long Region 10: Betsy Apolito Region 11: Jacqueline Burke Region 12: Karen Merrin Region 13: Vikki Clemons, Seena Skelton Region 14: John Dorger Region 15: Helen Flowers, Don Washburn, Jr. Region 16: Howard Martin Ashland City Schools: Neil Gupta Muskingum College: Linda Morrow: State Personnel Development Grant (SPDG) – Cohorts 1 and 2 Region 1: Washington Local, Van Wert City Region 2: North Ridgeville City, Sandusky City Region 3: Olmsted Falls City, Euclid City Region 4: West Geauga Local, Painesville City Local Region 5: Warren City, Ashtabula Area City Region 6: Lima City, Elida Local Region 7: Marion City, Ashland City Region 8: Barberton City, Cuyahoga Falls City Region 9: Canton City, Wooster City DRAFT OIP Facilitator’s Guide Region 10: Springfield City, Xenia City Region 11: London City, Groveport Madison Local Region 12: Bellaire Local, Zanesville City Region 13: Winton Woods City, Fairfield City Region 14: Miami Trace Local, Wilmington City Region 15: Ironton City, Scioto Valley Local Region 16: Meigs Local, Trimble Local December 2008 5 of 95 External Partners Great Lakes East Comprehensive Center at Learning Point Associates, Naperville, Illinois: Mark Mitchell, Sheryl Poggi, and Claudette Rasmussen Leadership and Learning Center, Phoenix, Arizona: Connie Kamm and Denver, Colorado: Brian McNulty Ohio Department of Education Barr, Stephan, Associate Superintendent, Center for School Improvement Falor, Stephanie, Consultant, Office for Exceptional Children Focht, Earl, Regional Manager, Office of Field Relations Lather, Tom, Associate Director, Office for Exceptional Children Mattei-Smith, Barb, Associate Director, Center for School Options and Finance MacLearie, Margaret, Regional Manager, Office of Field Relations Speers, Keith, Director, Office of Field Relations Telfer, Deborah, Executive Director, Center for School Improvement Troyer, Marilyn, Senior Associate Superintendent for Educational Programs Vasu-Sarver, Sandy, Executive Director, Center for Students, Families and Communities Washburn, Don, Sr. Field Liaison, Ohio Department of Education DRAFT OIP Facilitator’s Guide December 2008 6 of 95 Ohio Improvement Process Background DRAFT OIP Facilitator’s Guide Purpose of This Guide How to Use This Guide Overview of the Ohio Improvement Process (OIP) Principles of the OIP Documenting the Improvement Process December 2008 7 of 95 Purpose of This Guide This guide was developed under the auspices of ODE, Center for School Improvement. It was created to assist OIP facilitators in their primary work of building the capacity of districts to engage in an improvement process that results in improved student performance. The guide provides the necessary information and materials to facilitate the OIP. By using a common set of procedures and templates throughout the OIP, high-quality services will be delivered more efficiently. The objectives in developing the guide are to: Promote visual and functional consistency within and across services to districts. Promote the use of research-based, processes, methods, and resources. Help maintain a reputation for high-quality services. The benefits of using the guide will be: Increased confidence of districts in processes and services. Improved capitalization of OIP providers’ knowledge and skills. Reduced arbitrary decisions and reinvention of processes and resources. Because no written set of materials can anticipate or address every circumstance in the process or anticipate the needs of every district, OIP facilitators should contact a member of the State Level Design Team for assistance and advice. Throughout the document, the use of the phrase “district plan” refers to the CCIP. How to Use This Guide Each section of the guide is divided into a group of activities to support each stage of the OIP which, when completed, will lead seamlessly into the next stage. They include the following: Introduction provides information about the stage of the OIP, purpose, organization, and definitions for terms used. Outcomes for each stage also are listed. Procedures that describe what takes place before, during, and after each activity. More on Facilitation provides additional specific facilitation information. Resources that include templates and examples/illustrations that support the activities. Navigation icons assist in identifying these throughout the document. Tips are included throughout the document in shadow style text boxes. They include but are not limited to customizing options, session management, and logistics. Navigation icons are used throughout the guide as a reminder to provide OIP facilitators with important information at a glance: NAVIGATION KEY ICON PURPOSE Reference Alerts the OIP to access a reference. Resource Shows that an example or template is available. Tool Identifies tools provided by ODE DRAFT OIP Facilitator’s Guide December 2008 8 of 95 Overview of the Ohio Improvement Process (OIP) The Ohio Department of Education (ODE) recognizes the value of planning in the continuous improvement process and for the past decade has provided guidance and support to districts and schools in this process. Several recent developments have caused the ODE to revisit its current guidance. Recent Developments 1) As technology has evolved and new learning has emerged about the effective use of data in decision making and planning, ODE has identified and gathered data and created tools for organizing and analyzing data. 2) Research has provided more concrete knowledge about what works and how leadership in districts and schools should be distributed to improve instructional practice and student performance. 3) Many federal and state initiatives have required a planning process―some due to funding requirements, others due to the design of a project or program. Although each of these initiatives is a worthy project or program, the planning for each has fostered a design that is fragmented, often resulting in redundancy and duplication of effort. 4) After numerous attempts, the 126th General Assembly created a coordinated, efficient regional educational service delivery system to support state and regional improvement initiatives and to promote a simplified approach to regional service delivery. Substitute House Bill 115 clarified the roles and responsibilities of regional service providers with an emphasis on assisting districts in improving student performance. Integrated, Research-Based Approach Based on these developments, the department recognized that a more integrated approach was needed that is consistently implemented throughout the state and reflects development of a unified state system of support around a focused plan. This approach is the Ohio Improvement Process (OIP). OIP brings together all the data, resources, knowledge, and experience in planning and follows a four-stage cycle that will be guided by regional educational service system providers. Stage 1: Identify critical needs of districts and schools. Stage 2: Develop a focused plan. Stage 3: Implement and monitor the focused plan. Stage 4: Evaluate the improvement process. Within each stage, there is a set of well-defined elements that when completed lead to the next stage. The OIP can be used at the district, school, or classroom level. For the purposes of this document, the stages will be described for district-level use. DRAFT OIP Facilitator’s Guide December 2008 9 of 95 The OIP is based on the most recent research about what causes districts to improve. This research states that: To improve teaching and learning on a large scale, the whole district must be involved, with strong lines of communication. The role of district and school administrators should be refocused, with the highest priority on improving teaching and learning where data are used as the vehicle for changing conversations in ways that allow the most critical problems the district faces to be identified and addressed. It is important to give equal focus to the “how” as well as the “what” of improving teaching and learning, continuously using a cycle of monitoring and evaluating progress in order to constantly improve achievement. District Expectations Based on this research, the Ohio Leadership Advisory Council (OLAC) established common expectations for districts in the planning process: Collaborative structures and community engagement Culture and expectation for the use of data in decision making One plan with focused goals for achievement and instruction Board alignment and support of district goals and district alignment and support of school goals Monitoring goals for achievement and instruction Use of resources to support achievement and instruction, intentionally allocated to the district plan These common expectations are woven through the planning process described in this document. Stages of the Ohio Improvement Process Stage 1: Identify Critical Needs of Districts and Schools uses state and district data to identify the most pressing needs faced by the district. This stage primarily uses the Decision Framework to look at data from four levels: student performance, instructional management, expectations and conditions, and resource management. Stage 2: Develop a Focused Plan is development of a focused plan. Stakeholders including the local school board are involved in specific areas that require their engagement and direction. During Stage 2, focus areas are turned into two to three goals in two areas: 1) student performance and 2) conditions and expectations. Strategies that are grounded in research to achieve the goals are created from the causes of the most important and critical problems. Indicators for each strategy provide the yardstick by which success is measured. Actions are developed for each strategy and resources are aligned. Stage 3: Implement and Monitor the Focused Plan requires that all district employees know their responsibilities for ensuring that the goals and strategies are achieved DRAFT OIP Facilitator’s Guide December 2008 10 of 95 regardless of whether they have an accountability responsibility in the district plan or whether they have specific tasks to achieve certain actions. In addition, this stage requires that each school has an action plan that has been approved by the district and is aligned to the district goals and strategies. Taken together, implementation of district and school plans results in goal accomplishment. Stage 3 also requires that the district and buildings have a process for checking the implementation of each strategy and action taken toward reaching the goal. This can be accomplished in many ways, from key staff members submitting implementation checksheets to district leaders conducting walk-through observations to see whether the actions are occurring. Although actions to reach a strategy and goal are reflected in what district employees will do, the effectiveness of those actions results in improved student performance. Effectiveness also is measured using the indicators that have been created for each strategy. Stage 4: Evaluate the Improvement Process seeks to examine the summative impact on student achievement, degree of implementation and indicators identified for each stage of the OIP. This is the point at which the district determines whether goals have been reached by assessing improvement (pre – post) using data that was collected in Stage 1. The district and school should identify the purposes of the evaluation and questions that need to be addressed in the evaluation design early in the OIP so that the most appropriate methods and procedures are used. The OIP stages are recursive and, therefore, continually inform each other. As districts/buildings move through the four stages of continuous improvement, data will have a major role within each of the stages. As an integral part of the planning and implementation process, data have many purposes. District leadership teams and work groups can use data collection at the beginning of the planning process when implementing a part of the plan, such as professional development, for updating a current plan, or for preparing formative and summative reports to boards and communities. Principles of the OIP The vision of ODE is “higher achievement for all students.” Each district and building is working toward that end, as well as toward ensuring equitable access to high-quality instruction for all student groups in keeping with federal and state laws. Continuous improvement planning is the core process for improving instructional practice, leading to higher achievement for all students. To guide the work, 8 planning principles have been established. These are as follows: 1. Align mission and goals. The continuous improvement planning process should be guided by the mission of the district. The questions should be: “Do the strategies, actions, and resource allocations support our mission, and goals?” and “Are our behaviors and decisions congruent with our mission and goals?” 2. Commit to a continuous process and use the plan as a dynamic document. Districts that are fully committed to high performance view continuous improvement as the core work of every level of the organization. 3. Move beyond preference and opinion by relying on quality data and data interpretation. An effective planning process is predicated on the ability of the district, building and classroom to collect, organize, analyze, and utilize data to identify its highest priority needs for improved student achievement. DRAFT OIP Facilitator’s Guide December 2008 11 of 95 4. Use a collaborative, collegial process. Every plan gets its strength from the people who are committed to it. To make sure the plan will yield positive results, engage many parts of the community in understanding the plan, helping to make it stronger and ultimately becoming invested in making it work. Include business and community representatives, students, parents, teachers, administrators, and district staff in the planning process, and make the draft plan available for input from the entire community. Make sure the plan reflects the combined thinking and planning of a collaborative team who support plan implementation. 5. Communicate with those who are affected by the success of the district at each stage. District priority needs and causes may be related to the issues communities and schools are seeing, and their thoughts may help the planning team(s) better understand the situation. Multiple opportunities for communication and feedback should be included throughout the process. 6. Produce one focused plan that directs all district work and resources. Heretofore, districts have had many plans (e.g., technology, professional development, Title I, Title II, special education, career and technical education) for many reasons (e.g., basis of funding applications, federal or state requirements). Multiple plans diminish the district’s ability to respond to the most critical needs. By developing one focused plan that responds to the most critical needs, the district will leverage resources to achieve lasting success. 7. Expect substantive changes in performance and behavior as a result of implementing, monitoring, and evaluating the process and plan. The purpose of having a well-conceived planning process is to produce a plan that, if implemented with fidelity, will change student and adult behaviors that lead to improved instructional practice and student performance. 8. Be responsive to district context. Each district is unique. Although the planning process stages and elements are defined, they may need to be adjusted for distinctive characteristics and individualized needs of each district. Documenting the Improvement Process After each stage, the district needs to document the process. The documentation can be divided into three sections: methodology, summary and considerations, and final statements. The following table provides guidance about what to document in each stage. This information should be maintained by the program assistant assigned to the process so that it is complete and accessible to anyone who may need the information. In addition, the district should maintain a copy of the current mission, planning policy and procedures, DLT members, and meeting agendas and minutes. The amount of documentation should be detailed enough so that someone external to the district can understand the process. DRAFT OIP Facilitator’s Guide December 2008 12 of 95 Methodology Stage 1 Stage 2 Stage 3 Stage 4 Summary and Considerations Final Statements What data (including source) were collected to respond to each question Who including members of the DLT (names, positions, roles) collected, organized and analyzed the data Completed decision framework Summary of data to support decision framework results List of data that need to be collected in the future Prioritized data needs Quotes and comments, if desired How and when stakeholder forums are conducted Who participates (names, positions, roles) and how they are prepared to do their work How and when communication will occur Summary of decisions made Focused plan Analysis of prompts or questions used to solicit stakeholder input Summary of communication activities Conclusion statements Responses and reactions to communication activities Quotes and comments, if desired What evidence was collected to demonstrate implementation of the goals, strategies, and actions Budget reports to demonstrate funds were used in support of the plan Data collected to measure indicators Which explicit actions for monitoring progress toward the attainment of goals and strategies are used Which data were used for monitoring results How and when monitoring tool place to review student performance and adult practices How and what periodic corrections were made and what they were Progress reports about the degree to which the plan was implemented Analysis of budget expenditures Formative data about the degree to which indicators were met Summary of monitoring results including data analysis and interpretation Progress reports Interpretation of budget expenditures in relation to plan accomplishment Report about the degree to which indicators were met Quotes and comments, if desired Which data were used for Comparison of planned Evaluation report evaluating results How and when evaluation took place How and what corrections were made, and what they were results with achievements Summary of plan changes Conclusion DRAFT OIP Facilitator’s Guide December 2008 statements Recommendations for changes to next year’s plan Quotes and comments, if desired 13 of 95 Ohio Improvement Process Preparing for the Ohio Improvement Process DRAFT OIP Facilitator’s Guide Introduction Introductory Session with District Superintendent Orientation for District Leadership Team December 2008 14 of 95 Introduction The foundation for creating a high-quality plan requires that a work structure be created by identifying who will do the work, how the work will be conducted, and when the work will be completed. The school board’s role in preparing for plan development is to ensure that there is a district policy that provides guidance to the district leadership. Without a shared mission, there is no compelling reason why the plan should be developed. A shared vision is vital for the district because it provides the focus and energy to drive the work. The outcomes expected from preplanning are as follows: Define roles and responsibilities for each person and group participating in the process. Identify resources (people, time, and money) for plan development to gain the long-term benefits of a plan that is “owned” by large numbers of stakeholders. Prepare leaders and groups to work collaboratively, including the creation of a master plan schedule. Revisit or create a shared mission among district and community stakeholders. Introductory Session With District Superintendent Before meeting with the superintendent, as much data and information as possible should be gathered about the district so the OIP can be discussed within the district context. This includes the following: State assessment results Current CCIP Demographics (number of buildings, enrollment, attendance, and so on) The OIP Facilitator should contact the superintendent’s office and set up a mutually agreeable time for this first conversation. It is advisable to ask for at least two hours of uninterrupted time. Also, it is a good idea to request that one or two (but no more) other persons who may have a leadership role participate in the meeting. This may be another district or building administrator, member of a currently operating leadership committee, or a trusted district employee. A sample agenda follows. Note that the format also serves as a record of the meeting discussion and decisions. In this way, the agenda and minutes of the meeting can be captured in one place. An agenda and meeting summary template for this and other meetings is provided as Resource 1. The OIP facilitator will need to bring handouts to correspond to the agenda. Minimally, this includes the OIP Overview Visual (Resource 2), “OIP on a Page” (Resource 3), Decision Framework Flowchart (Resource 4), the Chart of Time Needed to Complete the OIP (Resource 5), and a blank template of Schedule for Plan Development (Resource 6). These resources will assist the OIP facilitator in impressing upon the district the logic of the process as well as the resources needed to be successful. This will ensure that no surprises arise and will form the basis for securing commitment to implement the process with fidelity. DRAFT OIP Facilitator’s Guide December 2008 15 of 95 SAMPLE AGENDA Ohio Improvement Process Date: September 5, 2008 Time: 9:00 a.m.–11:00 a.m. Location: Tactful School District Office Attendees: Superintendent T. B. Wise; Curriculum Director F. T. Child; Regional Facilitators Ima Winner and Ura Saint 9:00 a.m.– 9:15 a.m. Introductions and Purpose 9:15 a.m.– 9:45 a.m. Overview of the Four Stages of the Ohio Improvement Process (OIP) 9:45 a.m.– 10:15 a.m. Roles and Responsibilities of DLT including working with BLTs 10:15 a.m.– 10:45 a.m. District Leadership Team (DLT) Orientation 10:45 a.m.– 11:00 a.m. Securing Commitment and Next Steps Summary of Discussion/Decisions: During the meeting, there will be several critical points that the OIP facilitator will need to discuss with the superintendent. Agreements reached during this meeting will help the process flow smoothly. The OIP facilitator will need to be familiar with the Ohio Differentiated Accountability Model and be able to discuss with the Superintendent how his/her district relates to the model. Following are talking points that correspond to the above agenda and the handouts: DRAFT OIP Facilitator’s Guide December 2008 16 of 95 AGENDA TOPIC Introductions and Purpose OIP FACILITATOR TALKING POINTS HANDOUTS When introducing yourself, share your experience and training in the OIP to lend credibility to your role as external facilitator. See Background— Overview for details Explain why the ODE is embarking on this improved continuous improvement process. Using the OIP Visual, review the four stages. Explain how this process is different from other continuous improvement processes and the way CCIPs have traditionally been developed: Overview of the 4 Stages of the Ohio Improvement Process Facilitated Data Driven Focused, Coherent—One Plan Collaborative and Collegial OIP Overview Visual, “OIP on a Page,” Decision Framework flowchart (Resources 2, 3, 4) Walk through the OIP on a Page to illustrate how the elements result in completing each stage and how the stages inform each other. Discuss how the OIP will be replicated in all buildings using Building Leadership Teams (BLTs) to assure alignment and build capacity Describe the Decision Framework flowchart illustrating how using data and responding to analysis questions result in primary causes of the most critical problems the district is facing to improve student achievement. Review the roles and responsibilities of the groups and individuals that are needed to ensure a collaborative, collegial, and successful process (see More on Facilitation below). Chart of Time Needed to Complete the OIP (Resource 5) Highlight the DLT’s role, including discussion of: Roles and Responsibilities (more detail below) Current district teams and relationship to DLT Roles throughout the four stages and beyond OIP Membership, representation, and length of service Communication within and across DLT Work with BLTs to develop school improvement plans based on data, support implementation and monitor progress Decision-making process TIP: The DLT should be no more than 20 and no fewer than 7 persons. Identify the role of the superintendent as steward of the process, policy implementer, and participant on the DLT as a shared leader, either as chair or cochair. Emphasize the superintendent’s responsibility to fully participate. DRAFT OIP Facilitator’s Guide December 2008 17 of 95 AGENDA TOPIC OIP FACILITATOR TALKING POINTS HANDOUTS Agree on a structure (organizational chart) for the OIP process (see example below). DLT Orientation Securing Commitment and Next Steps Set the agenda for the first session with the DLT to include the messages the superintendent will deliver and the role of the OIP facilitator in the process. Schedule for Plan Development (Resource 6) Determine how the Schedule for Plan Development will be completed (before, during, or after the DLT meeting). Seek and document agreement on the following: Membership and meeting schedule for the DLT (including orientation date) Selection of chair/cochair and program assistant Role of OIP facilitators (internal, if appropriate, and external) Stakeholders and how they will be involved Draft agenda for the DLT orientation Message points for superintendent to deliver to DLT Commitment to follow the OIP Meeting Agenda Summary of Discussions and Decisions (Resource 1) After the meeting, the summary of the discussion and decisions should be finalized and sent to the superintendent as a record of the meeting and a prompt to follow up on the agreements. More on Facilitation One of the most important decisions the district will make in the process is selecting the right people for the work. The following provides guidance on helping with these decisions. Superintendent The superintendent, in addition to being on the DLT, has other responsibilities. As a vision/mission keeper, resource allocator, and architect of the plan, the superintendent also must assume the following responsibilities: Oversee the OIP, establishing additional procedures if needed. Select district leadership team. Set direction and expectations for DLT. Engage staff and community in planning process. Create collaborative structure for plan development. Foster culture of continuous improvement. Approve a single, focused, data- and research-based plan. DRAFT OIP Facilitator’s Guide December 2008 18 of 95 Budget the plan. Monitor plan development. Hold participants accountable for results. Communicate plan content, process, and results. The superintendent will need to identify a structure for how individuals and groups will work together, including determining who has decision-making authority, how communication should flow, and how relationships are structured. Identification of other resources to support the process will be a responsibility of the superintendent. Structure: The following diagram illustrates a proposed structure for the district planning process: Facilitator Board of Education District Leadership Team (including Superintendent) Superintendent Goal 1 Work Group Goal 2 Work Group Stakeholder Input Goal 3 Work Group Decision Making: Regardless of the management structure that is used, there will be decisions to be made by either an individual such as the superintendent or by groups such as the DLT or work groups. It will be important that everyone knows which decisions they can and cannot make. The OIP facilitator, in concert with the superintendent, should determine the types of decisions that can be made by each group or individual. The decision-making process depends on a) the number of people affected by the decision and by the impact the decision’s outcome will have on the plan, b) how much energy and support people must contribute to make the decision’s implementation a success, and c) how critical the decision is to the mission and success of the district. The type of decision-making process used rests on how clear and certain a course of action is. Decisions that clearly have only one correct answer require one type of decision-making process, whereas decisions with many viable choices require another type of process. DRAFT OIP Facilitator’s Guide December 2008 19 of 95 Communication: There can never be enough communication. The superintendent will need to consider how, what, and whom to communicate about the OIP. As a part of the standard DLT agenda, it is suggested that a communication record or log be developed. This document will assist in determining who needs to know what, when, and how. It also will serve as a log of what has occurred. Communication needs to be transparent and every layer of the system needs to be involved. A sample format follows: COMMUNICATION RECORD/LOG Date Audience Content/Format Objective Results Feedback Additional Resources: Equipment, including laptop computers, LCD projectors, and sound systems, as well as space for meetings available either through the district or the community will be needed. If the district does not have free access to these resources, they will need to be budgeted. There are not many material or supply costs to the OIP. However, there are some items that need to be budgeted. These include the following: Refreshments for stakeholder forums Newsprint and markers Folders or binders for district leadership team Substitute costs for building-level teacher leaders participating on the DLT or work groups District Leadership Team (DLT) Membership of the DLT should include individuals with key positions at the various levels of the organization, for example: director), school board o superintendent, members, o treasurer, o stakeholders representing o building-level administrators, parents, local businesses, and community organizations, o teacher leaders, o program directors (e.g., director/supervisor of special education, curriculum o groups such as teacher’s union representation will need to considered The DLT shifts the focus from an individual to a team that can function as purposeful communities. Those chosen for the team should be able to communicate effectively and influence others. The size and composition of the DLT is important to the success of the process. The DLT should be small enough to enable efficient communication and coordination but large enough to represent all areas of the district. In a large DRAFT OIP Facilitator’s Guide December 2008 20 of 95 district, the team may include up to 20 people, whereas in a small district, the team may be as small as seven people. It is desirable to have the superintendent serve as an active participant and he/she may be the chair. The chairperson of the team, if not the superintendent, should be an individual who can represent the team to the superintendent and should be in constant communication with the superintendent. The chairperson must be able to articulate the work of the team, build consensus among team members, and serve as the primary liaison with the OIP facilitator to adjust the process. The superintendent may choose to have a cochair. The benefit to having a cochair is that if one of the two is not available, there is a “back-up.” The downsides of having cochairs are the needs for constant communication to ensure a consistent message and for giving both cochairs equal weight in decision making (one cannot overrule the other). A rotation cycle for the DLT that allows for a core membership group with others serving a specified term may be necessary. For example, it is unlikely that all building-level administrators or teacher leaders can serve on the DLT. However, these individuals could rotate every other year to ensure a broader base of representation, as could teacher leaders and external stakeholders. It is suggested that individuals be asked to serve staggered terms of at least two years but no more than four years in order to have a balance of new and experienced members. In some cases, a rotation may not be required, particularly in small districts. The DLT needs to understand that this work continues after the plan is developed, the work of continuous improvement is ongoing and involves the DLT in all four stages of the OIP. It is estimated that during a two- to three-month period for Stage 1, DLT members will need to devote time to completing Stage 1 of the OIP with additional time by the chairperson/cochairs to plan and communicate with the OIP facilitator, superintendent and team members. The same time estimate applies to Stage 2. Consistent attendance is important for continuity. If a member is frequently absent, the superintendent may need to seek a replacement during the data review and planning process. Designees for DLT members should be allowed only in rare circumstances. A summary of the responsibilities of the DLT is identified below, several of which are drawn from the Ohio Leadership Development Framework. Implement planning policy. Promote commitment to continuous improvement. Maintain districtwide focus on high achievement for all students. Facilitate the efforts of work group(s). Develop a single district improvement plan that focuses on a limited number of district goals for instruction and achievement. Convey to schools and the community the district’s mission for guiding the development of the focused plan. Ensure that schools have focused building improvement plans that are aligned to the district’s goals. OIP Process Guide 21 of 95 December 2008 Monitor the progress of the district plan, performance and indicators and make necessary adjustments based on data. Communicate to ensure coherence and continuity. Provide opportunities for meaningful input and feedback from internal and external stakeholders. Make decisions regarding financial and capital management aligned to district goals and strategies. Board of Education There are several ways in which the local board of education can participate in the planning process and support the individuals and groups who are working on the plan. These include: Make planning a district priority, as evidenced through planning resources. Expect superior performance from all. Share the excitement. Be neutral while others are voicing opinions. Hold individuals and groups accountable for meeting their timelines and process. Ask for progress checks on plan development. Recognize the team for its hard work and commitment to the process. The specific responsibilities of the board members are identified below: Create and adopt planning policy. Approve goals prior to full plan development. Adopt focused plan/budget. Monitor plan results (indicators). Communicate goals, progress, and expectations to community. Stakeholders Stakeholder forums occur at least twice during the OIP. Stakeholders are individuals or representatives from organizations that are affected by the education system. It is important to include stakeholders who provide a variety of perspectives. The same stakeholders will not necessarily participate in both forums. At the beginning, stakeholders will be invited to comment on the mission in the context of critical needs identified through the Decision Framework. The amount of time each invited stakeholder will be asked to devote depends on which type of forum the district chooses. The focus of the feedback during Stage 2 is to provide input into the draft district plan. Stakeholders also will have the option of participating in the required public hearing about the entire plan. It is estimated that stakeholders will need to devote a total of about four hours of their time. OIP Facilitator Facilitation of the process cannot be underestimated. The OIP facilitator's role is to help manage the progression of the discussion and process. The district may choose to use a OIP Process Guide 22 of 95 December 2008 facilitator external to the district or a trusted employee who will be accepted by everyone as an internal facilitator. An OIP facilitator must: Serve as a critical friend who is a trusted partner, advocates for the success of the work, asks thought-provoking questions, and provides focused constructive feedback. Manage and direct meeting processes using a range of facilitation strategies. Understand meetings and make decisions affecting meeting dynamics. Keep the group focused on outcomes. Coplan with district leadership team chair(s). Act as a critical friend to ensure all responses are open and honest. It is estimated that the OIP facilitator will need an average of one day per week during Stages 1 and 2 of the process, or a total of about 25 days, to adequately do this job. The most expensive cost associated with the OIP process may be the facilitator; however, often the state system of support can designate a person to perform this valuable service. Program Assistant It will be important to designate a program assistant to undertake a variety of tasks, including arranging meetings, distributing agendas and materials, communicating with team members, taking meeting minutes, and creating and formatting documents. It is suggested that at least a one-third time position be devoted during Stages 1 and 2 of the process and less time during Stages 3 and 4. This person becomes responsible for inputting information and editing the draft and final plan. Goal Work Groups Membership of the work groups will vary greatly by district. Groups may be used to collect, organize, and summarize data during Stage 1. The goals will determine group membership during Stage 2. These groups may include content (e.g., literacy or mathematics) or program specialists with interdisciplinary perspectives and skills, teacher leaders from multiple levels, classified staff, parent and family representatives, public and private early learning representatives, student support services administrators, building-level administrators from multiple levels, special education designees, English language acquisition designees, assessment administrators, pupil personnel services administrators, and building and grounds administrators. Individuals who will be affected by the plan should participate. Goal work groups have two primary functions: To review district goals and the data upon which the goals were developed. To develop the strategies, indicators, and actions for the goals. The work groups will vary in size based on the goals identified by the DLT. Some individuals may need to participate in multiple goal work groups. For example, if the district has three goals and each includes professional development, it may be necessary to have the district’s professional development specialist(s) participate in each group. In addition, the work groups may need to call in others as appropriate to assist with working on a particular strategy or set of actions. It is estimated that during a two- to three-month period, the work groups will need to devote approximately 20 to 30 hours for Stage 2. OIP Process Guide 23 of 95 December 2008 Orientation for District Leadership Team Before orienting the DLT, and within a week or two of the introductory meeting with the superintendent, the OIP facilitator will need to meet with the chair/cochair to build an agenda, prepare packets, and make meeting arrangements. A meeting management checklist that can be used to prepare for this meeting as well as future DLT meetings is provided at the end of this section. The agenda should be sent to the DLT by the superintendent at least two weeks in advance of the meeting. (A sample agenda follows.) Handouts should include the OIP Overview Visual, “OIP on a Page” (flowchart of stages and basic elements), a template of Schedule for OIP, the Chart of Time Needed to Complete the OIP, the Decision Framework flowchart, Orientation PPT and Meeting Management Checklist (Resources 2 through 8). SAMPLE AGENDA District Leadership Team—Ohio Improvement Process Date: September 15, 2008 Time: 9:00 a.m.–11:00 a.m. Location: Tactful Local School District Office Attendees: Superintendent T. B. Wise; Curriculum Director F. T. Child; DLT members; Regional Facilitators Ima Winner and Ura Saint 9:00 a.m.– 9:15 a.m. Introductions and Purpose 9:15 a.m.– 9:30 a.m. Set Ground Rules Identify Group Assignments 9:30 a.m.– 10:15 a.m. Overview of the Four Stages of the Ohio Improvement Process 10:15 a.m.– 10:45 a.m. Roles and Responsibilities 10:45 a.m.– 11:15 a.m. Create/review a Schedule to Complete OIP Chair-Cochair 11:15 a.m.– 11:30 a.m. Commitment and Next Steps Superintendent Superintendent Chair-Cochair Regional Facilitators Superintendent Summary of Discussion/Decisions: OIP Process Guide 24 of 95 December 2008 During the meeting, there will be several critical issues that the OIP facilitator will TIP: Community schools establish Community School Leadership Teams (CSLT) rather than a DLT. need to discuss with the DLT. Whether a team or group is meeting for the first time or they have been working together for many years, they can be more effective when they are properly prepared for their work. Most meetings have an opening as the first agenda item. One suggested opening follows in the section under “Introductions and Purpose”. Introductions and Purpose The orientation should begin with participants introducing themselves, perhaps having individuals: Give his or her name and current affiliation. Complete the following phrases: My experience with continuous improvement planning is…. I think the greatest challenge this group will face is…. I believe this group has a wonderful opportunity to…. After the agenda for the orientation has been reviewed, the superintendent should provide the charge to the district leadership team. The charge should identify the following: Purpose (e.g., develop a data driven, research-based focused plan) Desired result (e.g., improved student achievement and changes in educator practices) Level of authority (decision making) Communication linkages General timeline Resources available Expectations for membership Importance of getting the district needs assessment and plan right—they affect the quality of building plans Once the superintendent has provided the charge, the chair or OIP facilitator should explain the district planning policy. Setting Ground Rules A vital first step is the development of guidelines for participation, often referred to as "ground rules," to provide a frame to ensure open, respectful dialogue and maximum participation. Most educators have participated in developing or using ground rules, so it is easier to list those rules commonly used and then ask for additional ground rules from the participants. When somebody proposes a ground rule, other participants should be asked if they agree to it. If most do, it should be added to the list. When listing ground rules, explain the intent behind each of them. For example: OIP Process Guide 25 of 95 December 2008 Listen actively—that is, respect others when they are talking by paying attention and keeping an open mind. The goal is not to agree; it is about hearing and exploring divergent perspectives. Speak from your own experience instead of generalizing (I instead of they, we, and you). Practice timely attendance—this requires everyone to be on time when the meeting starts, not to leave the room except when absolutely necessary, and to stay until the end of the meeting. Do not be afraid to respectfully challenge one another by asking questions, but refrain from personal attacks. Participate to the fullest of your ability; growth depends on the inclusion of every individual voice. Be conscious of body language and nonverbal responses; they can be as disrespectful as words. Give examples. Focus on ideas, not people. TIP: Some groups will include individuals who inhibit group progress, for example, power seekers, time dominators, clowns, recognition seekers. As a preemptive strike, these types of individuals may need to be discussed as ground rules are established. It also is important to set a ground rule for how participation will be managed. For example, should participants raise their hands to be called on, or should people speak freely? Some people—especially those who tend to be introverted—need more time to process thoughts before speaking, so the latter option may exclude them from the discussion. Still, the formal process of raising hands to be recognized may detract from the collective atmosphere needed to discuss multicultural issues. Once everyone agrees to the ground rules, they should be posted and visible during each meeting. They then can serve as a reference when there is a sense that participants are failing to sufficiently follow one or more of the items. It is everyone’s responsibility to challenge participants on the ground rules early and often. If the ground rules are not adhered to early in the process, it may become impossible to enforce them later. If a particular ground rule is routinely broken, it should be bounced back to participants. Identifying Group Assignments Group ground rules and assignments form the basic structure of the group. A group assignment is a pattern of behavior expected of a group member. Each member of the group should assume a job, although assignments can be changed or rotated from meeting to meeting. The following list of common assignments can serve as a starting point. Roles should be selected appropriate to the meeting. Chair. Keeps group on track—guides discussion, reminds group of objective, and regulates group activities. Recorder. Serves group memory function, takes minutes of the meetings, and keeps the group’s records and history. OIP Process Guide 26 of 95 December 2008 Timekeeper: Monitors the use of time as allocated in the agenda, and reminds group when time exceeds the allocated time. Assurance Checker: Keeps group focused on the process, assuring the intent is followed. Initiator–Contributor. Offers ideas and suggestions and proposes solutions and new directions. Information Seeker. Requests clarification, solicits evidence, and asks for suggestions and ideas from others. Opinion Seeker. Requests viewpoints from others and looks for agreement and disagreement. Information Giver. Acts as a resource person for the group and provides relevant and significant information based on expertise or personal experience. Clarifier–Elaborator. Explains, expands, and extends the ideas of others; provides examples and alternatives. Director. Draws together ideas of others, shows relationships between facts and ideas, and promotes teamwork and cooperation. Devil’s Advocate. Challenges prevailing point of view for the sake of argument in order to test and critically evaluate the strength of ideas, solutions, or decisions. Process Observer. Takes note of how the group is functioning and provides a summary at the end of the meeting or at appropriate times during the meeting. This person collects descriptive information that might include information about who talks to whom, what strategies group members are using, how groups accomplish their work, and what roles group members assume. Overview of the Four Stages of the OIP Several PowerPoint slides are provided (Resource 7) for the facilitator to use in presenting the OIP to the DLT. These slides provide basic information and may be augmented based on the facilitator’s knowledge of the district. Some key concepts to highlight during the presentation are as follows: The process will use a combination of state and district data to identify the most critical needs. The process highlights the importance of the DLT and the role the DLT plays in developing a coherent district plan that rolls down to all schools in the district. The process will result in one focused plan, eliminating the need for multiple plans (technology, professional development, Title I, and so on). The process is collaborative, not developed by one or a few people, and engages internal and external stakeholders. Roles and Responsibilities The DLT needs to understand their role and the time commitment they are making to the OIP process and beyond the OIP. OIP Process Guide 27 of 95 December 2008 One of the first steps is to introduce the Ohio Leadership Development Framework, including the research base and the self-assessment resource. If time and technology allow, each participant should complete the self-assessment online. This will provide baseline leadership information and data that can be used at Level III of the Decision Framework. One option for the DLT to become familiar with the Ohio Leadership Development Framework is to divide into groups of six, with one of the Framework areas assigned to each group. The group should consider the responsibilities of the DLT in relation to the areas and respond to these questions: □ What are we doing now? □ How did it come to be this way? □ What are we going to do to ensure we can meet these responsibilities? Groups may then report to the whole group after recording responses to the last question. Groups may wish to revisit these, as appropriate, at future DLT meetings. Creating a Schedule for Completing the Process The schedule for completing the process initially will be developed during the first DLT session and will be updated and checked throughout the OIP. An example of a partially completed timeline follows. A blank template for scheduling is provided (Resource 6). Partial Sample Schedule for OIP Template Leadership Team Meeting Schedule □ First and third Thursday of each month Preparation for Leadership Team Meetings (Chair/Cochairs and OIP Facilitator) □ Second and fourth Thursday of each month Start Date: August 25, 2008 Actual Completion Date: ____________ Timeframe – Process PREPARE FOR PLAN DEVELOPMENT August 25– Sept. 15, 2008 OIP Process Guide Introductory Session with Superintendent DLT Orientation Projected Completion Date: Feb. 28, 2009 Tasks Responsibility Expected Date of Completion Select facilitator and program assistant Superintendent August 27 Determine decision-making authority Superintendent August 27 Choose DLT members and appoint chair Superintendent Sept. 1 Review structure and organization of the DF Supt/Chair Sept. 5 Create schedule for plan development Supt./Chair Sept. 5 Purchase supplies and materials Program Asst. Sept. 5 28 of 95 Date of Completion December 2008 Timeframe – Process Tasks Responsibility Expected Date of Completion Communicate time commitments Superintendent Sept. 3 Develop materials, agenda for DLT orientation Chair/OIP Facilitator Sept. 8 Orient district leadership team Chair/OIP Facilitator Sept. 15 Revisit vision and mission statements DLT Sept. 15 Make vision/mission visible throughout the district Superintendent After Sept. 15 Date of Completion Commitments and Next Steps In order to maximize and effectively use time, the district leadership team should evaluate each meeting. This can be done quickly and need not take much time on the agenda. Two options for evaluating meetings are as follows: Plus/Delta—Members identify what worked well and what could be improved by using self-adhesive notes or offering ideas that are written on newsprint. Stop/Start/Continue—Each member writes one item that should be stopped (i.e., discontinue doing), one that should be started (i.e., something to improve the meetings), and one that should be continued (i.e., is working well). After the meeting, the summary of the discussion and decisions should be finalized and sent to the DLT as a record of the meeting and a prompt to follow up on the agreements. A checklist (Resource 9) can be used periodically to evaluate meeting effectiveness during a period of time. This allows everyone in the group to provide written feedback. More on Facilitation As an OIP facilitator of a process, not a program or initiative, it is important to employ certain behaviors. These include the following: Maintain neutrality. Maintain a sense of humor. Provide an equitable process. Be empathetic—show understanding of the parties’ situations, needs and feelings. Listen, paraphrase, clarify, and reflect. Intervene appropriately. Be authentic—without defensiveness or hidden agenda—explain your reasons for the decisions you make. Encourage interaction. OIP Process Guide 29 of 95 December 2008 Provide a safe environment. Confront and challenge—but only after empathy and respect have been established. Don’t talk too much. Be an energizer—set a positive tone. Keep the group moving on the problem or discussion. Avoid interpersonal confrontation. Act as the process guide—don’t get involved in the content. All teams, including DLTs, go through stages of team development. Each stage has certain characteristics that influence how the OIP facilitator responds to the group. These stages and the team characteristics are described below. It is the OIP facilitator’s responsibility to move the team to Stage Four as quickly as possible. It is likely that even within one meeting, a DLT may move in/out of several stages. The OIP facilitator needs to adapt to where the DLT is and use appropriate responses as described in the third column. Facilitation becomes that of a critical friend, shifting the role of the OIP facilitator from passive to actively challenging the DLT to focus on improving instructional practice and student performance and making data-based choices. STAGE OF TEAM DEVELOPMENT TEAM CHARACTERISTICS OIP FACILITATOR RESPONSE Stage One: Forming Testing, polite, impersonal watchful, guarded Be more directive by clearly articulating the purpose of the meeting/group and discussing ground rules for group functioning Stage Two: Storming Infighting, controlling, conflicts, confronting people, opting out, difficulties, feeling stuck Help members deal with conflict, clarify differing viewpoints, make sure hidden agendas or viewpoints are revealed. Stage Three: Norming Getting organized, developing skills, establishing procedures, giving feedback, confronting issues Use problem-solving skills to mediate differing positions, clarify any role ambiguity and when appropriate refocus the purpose or rules for group functioning Stage Four: Performing mature, close, resourceful, flexible, open, effective, supportive Serve as a collaborator and keep the group moving in a nondirective manner Vision/Mission Most districts have a vision or mission—and often principles or ideas—that articulate their perspective about education in their community. Sometimes these are voiced in a motto or statement that communicates what the district considers to be its purpose. The important part of having a mission is not the statement itself as much as it is the collective understanding of how the statement encourages specific behaviors and attitudes. A shared mission can be achieved by learning and identifying the reason a OIP Process Guide 30 of 95 December 2008 district exists (mission). The role of data in shaping the future and identifying mission (e.g., improved student achievement) cannot be underestimated because it will ensure that the district’s most crucial needs guide the work of district employees. If the DLT feels it is necessary to revisit the district mission, this process should occur as part of Stage 1 so the identification of critical needs is not delayed. The following questions may help OIP facilitator’s in working with DLTs that choose to revisit the district mission: Is there an educational, legislative, or political benefit to revising the mission? Have changes in data resulted in new understandings about students, educators, or the community? Have community demographics shifted since the mission originally was developed? Does the district desire to engage the community at this time? Who was involved in developing the current mission? Are they still the primary stakeholders? How long ago was the current mission developed? Is it still timely? Once these questions have been answered, the district can choose one of two options for the mission review. If the answer to most of the questions is “no,” then the district can undertake a quick review. If the answer to most of the questions is “yes,” then the district could use focus groups to gather stakeholder input into the revision of the mission. During the revision process, the district will need to determine who will be involved in the process. Generally, the DLT conducts a quick review. A more complete revision of the mission using focus groups engages the DLT and internal and external stakeholders. Whichever process is used to develop or revise the mission statement, it should be checked against these criteria: Is the mission short and sharply focused? Is the mission clear and easily understood? Can it stand on its own? Is it in language that students, parents, educators, and the community can understand and not misinterpret? Does the mission define why the district does what it does? Does the mission define why the organization exists? Does the mission provide direction for doing the right things? Does the mission inspire commitment? The following anatomy of a mission statement shows how the criteria can be applied: Why the district exists Direction for doing right things The Perfect School District educates all students in our diverse population so each meets or exceeds Ohio’s academic standards in a safe and affirming learning environment and becomes a lifelong learner and productive citizen. Short and focused: 34 words Why the district does what it does OIP Process Guide 31 of 95 December 2008 If the district recently has engaged stakeholders in describing its shared mission, it may choose to do a quick review of the statement and revise as necessary. Use of the data from the Decision Framework will be especially helpful. This process should not take more than two hours to complete and can be conducted by the DLT. If the team finds dissonance between the mission and what they are doing, then it may need to consider using focus groups to revitalize the mission. A review of the mission statement should include the following steps: 1. Write the district mission statement at the top of a page that is divided into three columns. At the top of the first column, write “what we say”; in the second column, write “evidence to measure it”; in the third column, write “evidence we need.” 2. Write each individual part of the mission statement in the “what we say” column. 3. Write the evidence that can measure what is said in the mission statement. For example, “data show that third graders are making steady gains in reading,” or “average daily attendance is 97 percent.” 4. If necessary, gather more evidence to show whether what is said actually is happening within the district. 5. If the district mission statement doesn’t adequately represent what is happening and what should happen now, then create a new statement. It is essential that the results related to the mission be communicated regularly to stakeholders. OIP Process Guide 32 of 95 December 2008 Ohio Improvement Process Stage 1: Identify Critical Needs OIP Process Guide Understand Structure and Requirements of the Decision Framework (DF) Complete the Decision Framework Identify and Affirm Critical Focus Areas Content Expectations and Conditions 33 of 95 December 2008 Introduction A plan is strong only if reliable and useful data have been collected, organized, analyzed, interpreted, and prioritized. It is the prioritization of data that yields the few district goals that form the basis of the plan. Stage 1 of the Ohio Improvement Process allows the district to take a variety of state and district- and building-level data (e.g., grade or department short cycle assessment results, universal screening and diagnostic assessment results, behavior/climate information, perception information, and program information) and filter it through the Decision Framework, the major tool used throughout the OIP. This Decision Framework tool details questions that participants will ask in order to identify and analyze the critical components for improving the academic performance of all students including subgroup populations. A flowchart illustrating the levels and organization of the Decision Framework are included in the prior Section. The outcomes expected from Stage 1 are as follows: ▪ Summarize and analyze data sets. ▪ Understand and apply data to the Decision Framework. ▪ Interpret key findings/information from needs assessment. ▪ Prioritize list of data-based critical problems from Decision Framework. Understand Structure and Requirements of the Decision Framework (DF) TIP: Community schools use the community schools decision framework, which is a hybrid of the district and building Decision Framework. Before meeting with the District Leadership Team again (second DLT session), the OIP facilitator will need to meet with the DLT chair/cochairs to build an agenda, prepare packets and make meeting arrangements. The purpose of this session is to become familiar with the organization of the Decision Framework (see Tools) and the questions that guide the analysis of data in order to complete the Decision Framework. The meeting management checklist (Resource 8) can be used as an aide in preparing for the meeting. The agenda should be sent at least a week before the meeting. Request a copy be emailed to you. A sample agenda for this second DLT session follows. Within each agenda item, time should be built in for questions. The DF may be sent to the DLT before the session if the chairs/cochairs believe that the members will have the time to familiarize themselves with the content before the meeting. If members of the DLT have access to laptop computers, they should be asked to bring them. Other options are to conduct the meeting in a computer lab or in a room that has an LCD projector to display the Decision Framework. OIP Process Guide 34 of 95 December 2008 SAMPLE AGENDA District Leadership Team Meeting Date: September 22, 2008 Time: 8:30 a.m.–12:00 p.m. Location: Tactful School District Office Attendees: Superintendent T. B. Wise; Curriculum Director F. T. Child; DLT Cochairs; DLT Members; OIP Facilitators Ima Winner and Ura Saint 8:30 a.m.– 8:45 a.m. Meeting Purpose, Ground Rule Review, and Assignments Superintendent 8:45 a.m.– 9:15 a.m. Overview of the Decision Framework Structure & Questions Explanation of Types & Methods of Data to Respond to Questions OIP Facilitators 9:15 a.m.– 11:30 a.m. inc. break Identifying Data to Complete the Decision Framework DLT Members 11:30 a.m.– 11:45 a.m. Agreement on Data to be Collected Chair/Cochairs 11:45 a.m.– 12:00 p.m. Confirmation of Assignments & Next Steps Superintendent Summary of Discussion/Decisions: During this second session, the DLT will become knowledgeable of the structure and requirements of the Decision Framework (DF) and will identify existing district data that correlate to the Decision Framework questions. The DF should be presented formally (large group presentation) with ample opportunity to respond to questions. Understanding the four levels of inquiry that comprise the Decision Framework and how they relate to each other is important. The Decision Framework flowchart (Resource 4) is a good resource to use to explain these relationships. Point out that Levels I and III produce the district goals and strategies. Actions are also generated from the Decision Framework. Following are talking points that follow the above agenda. OIP Process Guide 35 of 95 December 2008 AGENDA TOPIC Meeting Purpose, Ground Rule Review, and Assignments OIP FACILITATOR TALKING POINTS Purpose of session is to know how the DF works, understand the data needed to respond to the DF, and identify existing and needed data for the district to complete the DF. Review the agreed upon ground rules that were identified at the first DLT meeting. Determine who will fulfill assignments for this session, for example, timekeeper, process observer, devil’s advocate. ▪ Level I: Student Performance Level 1 requires the district to examine specific academic content areas, beginning with reading and math. Each content area focuses on the specific grade levels of concern using point data and trend data. Analysis includes district, building-level, grade-level(s), and subgroup information relevant to the content area. Each content area then stands alone for further analysis in Level 2. Level I focuses on identifying: o The weakest grade-level or grade level band for reading and math. Overview of the Decision Framework Structure & Questions o Subgroups with poor performance and/or significant gaps. o The magnitude/scope of these problem areas across the district. ▪ Level II: Instructional Management Level II is completed for each academic area independently. ▪ Level III: Expectations and Conditions Level III is completed independently of content areas. ▪ Level IV: Resource Management Level II and III results are analyzed in relation to Level IV. Each level has questions that are used to assist the district in analyzing data in order to respond to each DF item. A summary profile is provided at the end of the DF. Explanation of Types of Data & Methods of Data Collection to Respond to Questions OIP Process Guide Describe the four types of data that can be collected to respond to each question: 1) achievement/student performance, 2) perception, 3) program, and 4) demographic. Most data are collected using one of four methods: 1) surveys (which may be collected from all respondents (e.g., all teachers) or through sampling (e.g., a designated number of teachers from each grade level/school), 2) interviews/focus groups (usually a sample of potential respondents that are representative of the district demographics (e.g., parents representing grade levels, culture, ethnicity), 3) observations (classroom, programmatic—e.g., afterschool programs, 4) documents/records (e.g., assessment results, demographics, policies, lesson plans, meeting agendas/minutes). 36 of 95 December 2008 AGENDA TOPIC OIP FACILITATOR TALKING POINTS The DLT may use Resource 10 or 10A to determine additional types of data and methods of data collection that the district may need to complete the DF. This resource provides a list of possible data needed for informed, data-based decisions necessary to complete each level and area of the Decision Framework. The list includes ideas of where a district may find local data. It is not all inclusive but may serve to stimulate ideas for data sources. Bolded data sources in Resource 10 and 10A are pre-populated in the Decision Framework by the Ohio Department of Education. Identifying Data to Complete the Decision Framework TIP: Ideally, each area should have at least one data source. Otherwise, the analysis will rely on opinion, not fact. OIP Process Guide This tool has two facilitator uses: 1) to guide the discussion to determine what data the district has readily available that can answer the questions in the Decision Framework, and 2) to provide an organizer for summarizing data to support Decision Framework ratings/judgments. Few districts will have data for every question in the Decision Framework. If there is no data, the OIP facilitator needs to help the district make a choice to either 1) identify data that needs to be collected for future decisions, 2) collect data in the very near future to respond to questions in the Decision Framework, 3) include data collection in the district plan as an action, or 4) choose to disregard the question at this time. Whichever choice is made, the district/OIP facilitator will want to make note of it. If the DLT is 10 members or fewer, this activity can be done in whole group. If the DLT is larger, then the DLT can be divided in half, with each group taking responsibility for two of the levels. If the latter is the case, have the DLT members self-select which group to be in but ensure the number of members is fairly balanced in size. Each group will need a facilitator, timekeeper and recorder. The two overriding questions are as follows: What data and information do we have to respond to this item? Have the group look at the items for each Level/Area, taking one area at a time.*Ask the group to scan the items within each area for two to three minutes, making individual notes of any data the district currently has to respond to the item. Using an LCD projector, project Resource 10 or 10A. Record the name of the district document that provides information to respond to each item. Only list a data source once even though it may respond to several items. Once all existing data have been identified, ask if the group believes any additional data need to be collected. Make a list of these data. Explain that this may be data collected for this plan or may be collected for next year’s plan. To be realistic in identifying any new data that need to be collected, the whole group will need to look at data needed for all four levels before a decision can be made about what to collect. After completing each area, reach consensus on the data that will 37 of 95 December 2008 AGENDA TOPIC OIP FACILITATOR TALKING POINTS be collected for this plan, and data that will be collected in the future (possible action in the completed plan.) If there is considerable “data needed,” use the following questions to determine what is critical: Will the data be the only source of data for this area, or will it supplement other existing data? Can the data be collected efficiently and in time to complete the Decision Framework? What resources (people, technology, etc.) would be needed to collect the data? If we don’t have this data, will we be able to make an informed determination/rating? In thinking about what data needs to be collected, the DLT will need to discern meaningful data from irrelevant information. Doug Reeves suggests asking the question: “Will this piece of data help change curriculum, assessment and instruction and thus improve student achievement.” Once the list of existing and needed data has been identified, ask the group to scan the list to ensure it is reasonable for the district at this time. Agreement on Data to Be Collected Identify who will be responsible for either gathering existing data or for determining how the data will be collected and by when. This should be written next to each of the data items so there is a visible record of assignments. It is possible that the people who know about the data, have the data, or who can collect the data are not on the DLT. It will be important for DLT members to identify these individuals and seek their cooperation in this task. Confirmation of Assignments & Next Steps Poll members to ensure they understand their assignments. Another option is to have the DLT member summarize what he/she has agreed to do. This is important as it compels individuals to honor their commitment publicly. The chair/cochairs should summarize the next steps to include: Date of next meeting Initial agenda for next meeting (complete the DF) How to get assistance with assignments, if needed *Level I has two areas (overall student performance and subscale performance). Level II A, B, C has a total of nine areas. Level III A, B, C has a total of 15 areas. Level IV has four areas. OIP Process Guide 38 of 95 December 2008 After the meeting, the summary of the discussion and decisions should be finalized and sent to the DLT as a record of the meeting and a prompt to follow-up on the agreements. The OIP facilitator should work with the chair/cochair to identify how follow-up will occur to assure that assignments are completed on time and with quality. Follow-up can occur through written reminders (notes, e-mails, memos) and/or personal contact. Periodically, a checklist (Resource 9) can be used to allow group members to evaluate meeting effectiveness over a period of time. More on Facilitation A variety of data sources can be accessed to complete the DF. These data, some of which are provided by ODE and prepopulated into the Decision Framework (see list below), will be used to respond to the essential questions in the Decision Framework. Level III Data Level I Data Prepopulated Data in the Decision Framework ▪ District performance on OAT and OGT (overall and subgroup data) ▪ Building performance on OAT/OGT for all students and for each subgroup (minority students, students with limited English proficiency, students with disabilities, students with economic disadvantage) ▪ Student performance data related to content areas ▪ Kindergarten Readiness Assessment Literacy (KRA-L) data ▪ Disciplinary actions (occurrences, suspension, expulsion) ▪ Attendance data ▪ Graduation data ▪ Drop-out data Additional Data Available from Statewide and Specialized Tools ▪ Student Achievement Data identified in the Data Tools Catalog (summary of other useful data tools) http://education.ohio.gov/GD/Templates/Pages/ODE/ODEDetail.aspx?page =3&TopicRelationID=3&ContentID=54928&Content=54947 ▪ High-Quality Teacher Related Data on Professional Qualifications (Teacher Distribution Data Analysis-TDDA and Teacher Distribution File—TDF) ▪ Leadership Data (online self-assessment survey based on the Ohio Leadership Development Framework) ▪ Contextual Data identified in the Data Tools Catalog Local Data Local data will need to be collected by the district. The OIP facilitator will likely need to prompt the conversation about available district data, which may include the following: ▪ Surveys of Enacted Curriculum Topographical Maps and Instructional Practices Frequency Charts OIP Process Guide 39 of 95 December 2008 ▪ Curriculum Alignment Gaps Between Standards, Curriculum, Instruction, and Assessments ▪ Teacher, Parent, Student, and Community Perception Surveys ▪ School and Classroom Observations ▪ Program Evaluations (afterschool and/or summer school, PBIS, character education) ▪ Assessment Data (short-cycle assessment, standardized assessments, diagnostic assessments, screening, progress monitoring, classroom assessments, value added) ▪ Professional Development Data (evaluation results, attendance, agendas, schedule) ▪ Demographic Data (teacher attendance, enrollment) ▪ Documents/Records (lesson plans, policies, building schedules, grade-level meetings agendas/notes/schedules, teacher evaluations, planning and budgeting procedures, newsletters, parent communications, budgets, decision-making structures and processes, audit findings and responses) ▪ Partnerships: membership, purpose, role, results Collect, Organize, and Summarize Data Completing Resource 10 should help the district identify gaps in data and determine the most relevant information that needs to be collected. It will be the responsibility of the person(s) assigned to the data to organize and summarize the data for the DLT to analyze it efficiently. There are a variety of ways to organize and summarize data. Tables, charts, graphs, and written narratives are the most common. When using tables, graphs or charts, the source and timeframe of the data should be given. If at all possible, multiple years of data should be provided. In order to determine trends, at least three years of data are needed. To make comparisons, at least two years of data will need to be provided. Data summaries are primarily looking for trends and patterns that provide a focus for district goals and strategies. The DLT should also have on hand the following references available on the Ohio Department of Education website (www.ode.state.oh.us): ▪ ▪ Ohio Professional Development Standards State Academic Content Standards Although it is impossible to prepare OIP facilitators for all the ways in which data can be organized and summarized, the following are common and simple examples of how this can occur in preparation for the next DLT session. These are offered as ideas to help OIP facilitators as they work with individuals to complete assignments. Level I: Student Performance Level I and Level IIA–2 calls on the DLT to analyze student achievement/performance data. The Ohio Department of Education provides the district with state assessment data charted and graphed. The Data Tools Catalog describes several formats for looking at state level achievement data, both for accountability and diagnostic purposes. These tools allow educators to drill down into the data, examine them from many lenses, link to standards, align resources, perform local assessments, and examine other data sources. Two tools that are particularly helpful are D3A2 (www.d3a2.org) and Success (www.success.ode.state.oh.us). Success offers detailed state assessment (OAT and OGT including alternate assessments) OIP Process Guide 40 of 95 December 2008 results by building, class and student. Data can be disaggregated by subgroup, item analysis and manual re-rostering. D3A2 links state assessment results to aligned academic resources from 14 content providers, including lesson plans, assessments, videos, and video distance learning. One use of D3A2 is to check state assessment data to local data to increase accuracy. This is also where district assessment results can be discussed. Reports such as objectives performance reports, subtest reports, skills reports, content cluster reports, and mastery objectives reports will need to be considered in addition to those provided by the Ohio Department of Education. Often district assessment data are provided in summary or aggregate formats. However, if the district also wants to look at local data such as writing rubric results, the team can summarize the data by charting the percentage of students who performed at the desired level or above similar to other proficiency scores. Once all the achievement graphs/charts are available, the DLT can use highlighters to emphasize the numbers in the charts using the “stoplight” method* indicated by the colors below. For some data, such as the KRA-L or DIBELS, the use of the stoplight method will need to be modified. For example, the DIBELS data show the percent of students who are high, medium, and low for the areas tested. The KRA-L data provide composite scores within three bands. In both of these cases, the district may want to highlight with green (high on DIBELS, Band 3 on KRAL), yellow (medium or Band 2), and pink (low or Band 1). HIGHLIGHT COLOR MEANING % OF STUDENTS (SUGGESTED CUT-OFFS) Blue Green Yellow Pink WOW! Beyond Expectations GOOD! Meets Expectations CAUTION! Below Expectations URGENT! In Need of Immediate Improvement 100% 80–99% 70–79% 0–69% % OF STUDENTS (OUR CUT-OFF LEVELS) *Taken from Data Retreat Facilitator’s Guide, North Central Regional Education Laboratory, 2001. The samples below shows two data charts prepopulated in the Decision Framework with numbers highlighted using the stoplight method described above. Grade Level 2004–2005 SY Metrics Students Tested Grade 3 Not Proficient Students Proficient Proficient Percentage Students Tested READING Subscale Vocabulary Reading Process Informational Text Literary Text OIP Process Guide 2005–2006 SY 2006–2007 SY 163 140 180 33 33 42 130 107 138 79.8% 76.4% 76.7% 168 168 139 3rd 4th 5th 6th 7th 8th Grade 3 89.9% 81.6% 91.8% 87.3% Grade 4 79.4% 83.1% 86.0% 77.9% Grade 5 85.2% 84.6% 88.3% 87.7% Grade 6 86.3% 82.0% 82.6% 90.7% Grade 7 89.5% 86.5% 85.4% 86.0% Grade 8 89.0% 87.2% 86.0% 87.8% 41 of 95 9–12th (OGT) OGT 82.4% 79.0% 80.1% 78.4% December 2008 Three ways of analyzing achievement data to show comparisons are: a) yearly cohort group comparisons, b) yearly grade-level comparisons, and c) within-year progress data. The following is how these can be visually represented. State level data can only be applied to (b) below. If the district wishes to do (a) or (c), it must rely on district or school level data. a) Yearly Cohort Group Comparisons Year 1 Grade 3 Year 2 Year 3 Year 4 Student Group A Grade 4 Student Group A Grade 5 Student Group A Grade 6 Student Group A b) Yearly Grade-Level Comparisons Year 1 Year 2 Year 3 Year 4 Student Group A Student Group A Student Group A Student Group A Grade 3 Grade 4 Grade 5 Grade 6 c) Within-Year Comparisons Year 1 Fall Grade 4 Winter Student Group A Student Group A Spring Student Group A As a reminder, Level 1 data will result in the identification of one to two priority content areas, which will become the student performance goals. Level II: Instructional Management The questions in Level II are related to educator quality, curriculum, instruction and professional development in the specific content area. The Data Tools Catalog describes several tools that can provide contextual data to help understand the causes affecting student achievement results. The Catalog only includes tools sponsored by the Ohio Department of Education. Level II A: Curriculum, Assessment and Instruction—One tool that is available statewide is the Surveys of Enacted Curriculum (SEC) (www.seconline.org). This instrument OIP Process Guide 42 of 95 December 2008 collects self-reported data on the alignment of instructional practices, standards, and assessments. The data examine the extent of alignment between the enacted curriculum (what is taught), the intended curriculum (what is required by Ohio Academic Standards), and the assessed curriculum (what is tested by Ohio state assessments). It is available in four subject areas: English Language Arts, Mathematics, Science, and Social Studies. The Instructional Practices data cover areas such as professional development, use of homework, and teacher opinions and beliefs. The data are organized and summarized for easy use. The follow graphs provide an example of how the intended, enacted, and assessed curriculum is displayed. Viewing Enacted, Intended, and Assessed Curriculum Grade 8 Math Intended Enacted Assessed Level II B: Educator Quality, Teacher and Administrator Turnover, and Educator Qualifications The Data Tools Catalog describes the Teaching and Learning Conditions Survey, which provides data on factors that affect teacher hiring, retention, and mobility across the state. The online survey has fewer than 40 questions and can be completed in 30 minutes or less. If the district has at least 40% participation, districts can receive school and district summaries. Level II C: Professional Development As indicated earlier, the SEC also provides data on professional development. The district may also choose to summarize data gleaned from the Standards-Based Individual Professional Development Plans (IPDP). Beginning in 2008–09, the IPDP aggregated rubric results may serve as a valuable source of data for the district. The System to Achieve Results for Students (STARS) provides dates and participant information about professional development training. This data may be compared with OIP Process Guide 43 of 95 December 2008 observation and achievement data at the classroom/building levels to determine if professional development time and resources are being directed to the priority need areas and to determine if there is an impact on student achievement as a result of participating in professional development. Level III: Expectations and Conditions While Level II is focused on a particular content area, Level III has an overall general/global effect on student academic performance across all content areas and is viewed once. District personnel may also identify a specific issue that is unique to their district or community that needs attention to promote, facilitate, or improve student performance. Primary data sources in Level III are a) perception data, b) demographic data such as mobility, attendance and discipline, 3) program data, and 4) processes and procedures data. Following are ideas for organizing and summarizing this data. a) Perception data will take unique formats depending on the nature of the data. Persons assigned to summarize perception data will need to determine guidelines or cut-off levels for survey data. Prior to looking at the data, the DLT members assigned to this data should answer the question: What would positive results look like? For example, if a survey was ranked on a 5-point Likert scale, positive results may be the percentage of rankings of three or higher. The ranking percentage could be defined as the cut-off level. HIGHLIGHT COLOR SURVEY RESULTS (% OF RANKINGS OF 3 OR HIGHER ON A 5-POINT SCALE) MEANING Blue WOW! Beyond Expectations 100% Green GOOD! Meets Expectations 75–99% Yellow CAUTION! Below Expectations 55–74% URGENT! In Need of Improvement 0–54% Pink OUR CRITERIA For other results, it may be useful to use a similar color-coding process as described above. Because the perception items in the DF are similar across students, parents, and teachers, the results from the questions asked may be shown as in the sample table below. Is should be noted that there may be many questions for which the degree of implementation is the considered judgment of the DLT and not based upon a survey or other instrument. % of Rankings of 3 or higher on a 5-point satisfaction scale Focus on positive student outcomes in (content area) Parent Survey Student Survey Teacher Survey EL M HS EL M HS EL M HS 80% 65% 40% 85% 75% 60% 95% 90% 85% Safe environment for learning Partners in educational process Sensitivity to students needs when they experience academic OIP Process Guide 44 of 95 December 2008 % of Rankings of 3 or higher on a 5-point satisfaction scale Parent Survey EL M HS Student Survey Teacher Survey EL EL M HS M HS and/or behavioral needs Challenging (content areas) curriculum The Data Tools Catalog describes a K–12 school climate survey that provides data on the quality and character of school life. The School Climate Survey (http://www.csee.net/climate) classifies data into four areas: safety, teaching and learning, interpersonal relationships, and institutional environment. Student data can be disaggregated by grade level and subgroup and can be administered online and hard copy. These data are organized and summarized to show rating patterns for each group for each dimension surveyed. Charts are organized in two ways to make it easier to make different kinds of comparisons: o By climate dimension, showing the rating patterns for physical safety, socialemotional safety, and so on for students versus school personnel versus parents. o By population group, showing the rating patterns for students, then school personnel, then parents for each dimension. Profile charts of median ratings for different subgroups, school personnel, and parents show perceptions of school climate dimensions in consistently different ways and which dimensions might be most sensitive to different ratings for subgroups. Subgroups include the following: 1. Students by grade, gender, race/ethnicity 2. School personnel by grade and experience 3. Parents by child, grade, race/ethnicity b) Demographic data in the form of attendance and discipline data are provided through the Ohio Department of Education. Mobility data must be obtained using district data. Mobility is defined as students not in the same building 120 days before the state testing window. Level III B.4 (Multiple Risk Factors) asks about the affect of certain demographic characteristics on student performance. Using these characteristics, the DLT members will need to look at the information for those students who have a pattern of low-proficiency levels (pink if using the spotlight method). By each name, notations should be made on a report according to the demographic data. It may also be appropriate to code program data such as afterschool programs, summer-school programs, tutoring, and other interventions provided to students. Coding may look like this: Code Definition PL Proficiency Level—students who have a pattern of low proficiency, below standards, at risk M Mobility—students not in the same building 120 days before the state testing window A Attendance—students absent for any reason for five or more days. OIP Process Guide 45 of 95 December 2008 D Discipline—students referred for any reason two times or more, all students expelled or suspended AS After School Program—students who attend 80 percent or more of the time SS Summer School—students who attend 80 percent or more of the time T Tutoring—students who receive tutoring DLT members should look for patterns in the data. At this point, the DLT should not concentrate on individual student names, but rather focus on patterns across all of the students. It is important for the DLT not to attach causes to patterns. The DLT members assigned will need to identify the patterns observed for all DLT members to view when completing the Decision Framework. c) Processes and procedures cannot be coded or analyzed. They should be reviewed against the questions in relation to Level III A. In addition to knowing if the processes and procedures exist, the district must also know if they are followed and if they are effective. For example, requiring each building to have a data team and establishing procedures for the data team to follow are good. However, the district must also have records, documents, or reports that show how often the team meets and what they do when they meet. In general, the questions for this level are: How frequently is it used? How is it used? and How effective is it? Level IV: Resource Management Level IV analyzes how resources are used in the district. Resources are identified as people, time, and money. Suggestions to analyze these characteristics to respond to the essential questions are to examine district policies and procedures and analyze expenditures by categories to determine if funds are being allocated to priority needs. Complete the Decision Framework 5 The DLT is ready to complete the Decision Framework. While analysis can be conducted with statistical programs and electronic data tools, the process of digging through it, finding patterns and trends, diagramming observations, and collaborating about what is seen is a very powerful process. Completing the DF by the DLT offers new insights and illuminates views that might not have otherwise been seen if the DF were done by a few people or by individuals. In fact, many districts that piloted the OIP process found this to be the most robust part of the process. As team members analyze the data and respond to the DF questions, not only do they see more clearly, they engage in their own professional growth with their own data. Before meeting with the District Leadership Team to complete the DF (may require multiple sessions), the OIP facilitator needs to meet with the DLT chair/cochairs to ensure that data has been collected, organized and summarized, develop an agenda, prepare packets and make meeting arrangements. The purpose of these meetings is to analyze the data and complete the Decision Framework. The meeting management checklist (Resource 8) can be used as an aide in preparing for the meeting. The agenda should be sent at least two weeks before the meeting. Request a copy be emailed to you. A sample agenda for this DLT session(s) follows. It may be appropriate to divide this agenda into three meetings as there may be a large quantity of data to analyze, and there are approximately 175 areas to rate in the Decision Framework. OIP Process Guide 46 of 95 December 2008 All data summaries will need to be submitted to the chair/cochairs one week prior to the session. Submitting the information in advance of the meeting is necessary for the program assistant to create the first section of the DF Wall (see description below). Let DLT members know that they will be asked to give a three- to five-minute presentation on their data summaries by Level/Area. The DLT may be sent a data packet with all summaries before the meeting to allow them time to familiarize themselves with the content before the meeting. SAMPLE AGENDA District Leadership Team Meeting Date: December 4–5, 2008 Time: 8:30 a.m.–3:30 p.m. Location: Tactful School District Office Attendees: Superintendent T. B. Wise; Curriculum Director F. T. Child; DLT Cochairs; DLT Members; OIP Facilitators Ima Winner and Ura Saint December 4, 2008 8:30 a.m.–8:45 a.m. Meeting Purpose, Ground Rule Review, and Assignments Superintendent 8:45 a.m.–11:30 a.m. DLT Member Presentations—Data Summaries by Levels/Areas, Analysis and Completion of DF DLT Members 11:30 a.m.–11:45 a.m. Presentation on Current CCIP Performance Superintendent 11:45 a.m.–12:30 a.m. Lunch 12:30 a.m.–3:30 p.m. Analyze Data and Complete the DF DLT Members December 5, 2008 8:30 a.m.–12:00 p.m. Analyze Data and Complete the DF DLT Members 12:00 p.m.–1:00 p.m. Lunch 1:00 p.m.–3:00 p.m. Review DF Results and Confirm District Priority Problems 3:00 p.m.–3:30 p.m. Next Steps OIP Facilitator (s) Chair/Cochairs Summary of Discussion/Decisions: 5 During these sessions, the DLT analyzes the available data and completes the Decision Framework (DF). It is advisable that an organizer (e.g., Resource 10A) be used to record the data summary. The role of the recorder will be critical to having a record of the DLT discussion. Remind the DLT that Levels I and III produce the district goals. OIP Process Guide 47 of 95 December 2008 One technique that can be used to present the data summaries is to combine a “Decision Framework Wall” with a gallery walk. This will take approximately three hours to complete. Following is a description: A) Create a “Decision Framework Wall” by building three sections on a large wall. 1. The first section should be displayed on the left side of the wall. It contains the data summaries from the Decision Framework and other local data. Tables, charts, graphs and narratives should be displayed by DF level/area. 2. The middle section will include space for the Decision Framework questions. The questions can be displayed using an LCD so as the DLT discusses the question(s) in relation to the data summaries in the first section. This is the point at which the DLT analyzes all data and information relative to the question. 3. The third section displays the Decision Framework rating options for the question(s). This is where the DLT discusses their criteria for the ratings and makes a decision on the rating per question once all data has been presented and analyzed. This may also be displayed using an LCD projector. B) Beginning with Level I, each DLT member briefly presents the data summary for their assigned Level/Area to the entire DLT. This presentation includes a brief explanation of the process used to collect, organize, and summarize the data for the area assigned and the results. Highlight observations, comparisons, and trends found from the data. Continue the presentations until all Levels/Areas are presented. There are a total of 30 areas. Budget three to five minutes for each presentation. C) Assign one to two members of the DLT to serve as a docent for each of the Levels. The docent role is to answer questions about the process/data but not to defend the data. These individuals will not make formal presentations. The data should speak for itself. The primary function of the wall is to allow group members to view the data, ask questions, and discuss the data and results in their group. In the proposed agenda, seven hours have been allocated, an average of 2.5 minutes per area (175 areas in the Decision Framework). Some items will not require extensive discussion, as the data will be obvious. Other items may be able to be clustered or the same data may be used to respond to several items, thus requiring a limited time for discussion. Other items may require extensive discussion. It is important for the OIP facilitator to keep the conversation moving and to make decisions as quickly as possible. The amount of time needed depends on several variables, for example, the number of DLT members, the quantity and quality of data, and the “emotional” response to the data. D) Once the Decision Framework is complete, the DLT looks at the results by Level and identifies the district priority problems. After going through Level 1, district priority student performance problems should be identified. This list can be created by looking at the level/area from the DF with the lowest scores and highest level of concern. After the session(s), the completed Decision Framework (DF), which includes the profile report, should be sent to all DLT members. The OIP facilitator will need to meet with the chair/cochairs to identify any obvious data gaps found while completing the DF and determine how these gaps will be addressed (possibly as actions in the plan that will be developed in Stage 2). Periodically, a checklist can be used to evaluate meeting OIP Process Guide 48 of 95 December 2008 effectiveness over a period of time (Resource 9). This allows everyone in the group to provide written feedback. More on Facilitation Initial discussions relative to the Decision Framework are often focused on using existing data to the greatest extent possible. That being said, it is also clear that there are important questions that needed to be discussed in most districts for which data do not often exist. For example, few districts can describe with any accuracy the level and alignment of instructional practice. Similarly, many do not know if the “aligned” district curriculum is used. Teacher mobility is not part of most district discussions (most appropriate for medium to large districts). Additionally, many districts/buildings do not conduct student surveys or parent/community surveys. Part of the job of an OIP facilitator is to help the districts and buildings to consider whether a need exists to collect this type of information and, if so, the best method of collection. In the meantime, district leadership teams (DLTs) should be led to honestly discuss where they think they are and what might need to be done immediately. In many cases, the best thing is to plan for collecting the information before making judgments—recall that it is unusual for administrators, teachers, parents, and students to have the same understandings Following are questions organized by level and type of data that OIP facilitators can use as they work with districts to complete the Decision Framework. Level 1: Student Performance KRA-L Data In reviewing data for all students, what percent of the students are coming to school in each level? What is the percentage of children that fall within each KRA-L score band (i.e., assess broadly for intense instruction; access for targeted instruction; assess for enriched instruction) each year for the past 3 years? What is the nature and extent of formal early learning experiences children may have had prior to entering kindergarten and how do these experiences relate to KRA-L results? [compare data according to pre-school experiences to see if the different experiences contribute to variations in student scores] o o o o o o o District pre-school Title I pre-school Head-start Special education Parent education program Private preschools Other public funded pre-schools How do KRA-L data vary across subgroups of children [compare data according to major disaggregated groups to determine if a single group or several groups are coming to school significantly behind their peers]? What mechanisms are in place to monitor ongoing literacy progress in preparation for 3rd grade OAT reading subtest? OIP Process Guide 49 of 95 December 2008 State Test Data: In reviewing disaggregated student data, are there achievement gaps among the groups and what is happening to the gap(s) over time? Are they widening, staying the same, closing? How many (number, percentage) students’ scores count in more than one subgroup? Subscale scores: If you see a cluster of low scores (for instance, in math you might see geometry as well below par or perhaps 5th grade has a sudden drop off), look at the subscale scores for each of the buildings to see if those results tend to stand up. If the results are similar in both low performing and high performing buildings, the probability of cause of the problem increases in areas such as: curriculum alignment, instructional materials alignment, or lack of professional development across the district. Consider that the data portrayed show only percent (%) proficient. For those student groups not meeting the state standard for AYP, look for other signs of significant progress that indicate they are likely/not likely to be proficient by 2013-14. o Is a significant percentage of a student group moving from “below basic” to “basic” over the last three years? o If you average the scale scores for the various groups, is the student group in question progressing at a faster rate over the last three than the others student groups or faster than the “all” group category? If that trend continues, could you determine that most of the students will be proficient by 2013-14? o Do value-added data reflect positive progress even though the percent of proficient students may not meet state standards? Is this progress sufficient to ensure students will graduate with a solid education? o Are other assessment data used to gauge student progress against grade level standards and/or their response to intervention for the purpose of modifying instructional practice (see Assessment Questions)? Level II: Instructional Management; Educator Quality; Professional Development (PD) Curriculum: If a student group is not performing well (e.g., English Language Learners), are there materials that teachers can use as part of the routine instruction to improve success? Are instructional responses to cultural differences a possible partial cause of student underachievement? o Are teachers aware of cultural differences among the student body? Are additional high quality materials, aligned to the curriculum, available to help students succeed (reading sources, universally designed software, on-line tutoring, etc.)? Are curricular materials challenging to the students but within their ability level? Instruction: Is there agreement at the district level on what constitutes high quality teaching and learning? Have teachers been trained in and do they regularly employ differentiated instruction in their classrooms? o Does the district expect more challenging opportunities to be provided to all students who are typically succeeding well in class? o Does the district expect more challenging opportunities to be provided to high-able learners? o Does the district expect instruction to be modified, as needed, to meet the instructional needs of struggling learners? Is the Survey of Enacted Curriculum used to identify the: o Degree of alignment with state standards and assessment? o Degree to which teaching targets higher levels (higher order thinking skills, 21st Century skills, etc.) of learning? Is after-school tutoring available to students who are not achieving at grade level? Are the tutors highly trained in OIP Process Guide 50 of 95 December 2008 the content area? Are students’ language and culture positively integrated into the teaching/learning process? If a student group is not performing well, especially SWD and/or ELL, but other groups are, is the district sure that these students have access to and are participating in regular classroom instruction and learning? Assessment: Does the district review universal screening data for incoming kindergarten students to ensure that instruction is differentiated to meet all learners’ needs? Do data collected from other assessments (e.g., DIBELS, diagnostics, standardized, common formative assessments) validate or challenge the results of the state assessments? o Do any of these assessments allow the district/building to dig deeper into a subset of the content area (i.e., reading comprehension, fluency, vocabulary, etc.)? Is progress monitoring or other strategies used to gauge students’ response to interventions to improve access to grade-level content and to reduce inappropriate referral to special education? Professional Development: Is there a commitment to full implementation of any evidence-based practices learned through professional development? Is there a process in place for principals, teacher teams, and/or building leadership teams to frequently observe/monitor classroom instruction to ensure that the “learnings” from professional development are being consistently implemented at a high level? Are teachers provided with opportunities to learn from each other? Does the district have a policy on PD that is aligned with and/or does the district operate in accordance with the Ohio Standards for Professional Development? Level III: Expectations and Conditions Climate: Is there an account of instructional time lost due to assemblies, field trips, special programs, play day, celebrations, magazine/book sales, walk-/bike-a-thons, announcements, etc.? If not, should there be? What percent of the instructional time in each building is lost? What policies/practices are in place to minimize the loss of instructional time? Is there a proactive and positive behavior policy in place to minimize discipline referrals and increase access to instruction? Is the first language and culture of children valued? In what ways? If there is a drop in discipline occurrences, are data reviewed to ensure that the drop is due to intentional changes at district/building/grade levels, and NOT due to: o o o Increased drop-outs at the high school Decreases in student attendance Decreases in student enrollments If there is an increase in graduation rate, are date reviewed to ensure that the increase is due to more students graduating, rather than a decrease in the number of students available to graduate (i.e., more high school students going to community schools). OIP Process Guide 51 of 95 December 2008 Leadership: Is the use of shared leadership models supported by the district? Are leadership teams used to gain focus and alignment of the work across the district? Are all teachers expected and encouraged to serve on leadership/data/professional learning teams? Does the district have a policy on prevention/intervention that o o o o o Addresses adult actions necessary when children are not meeting grade-level indicators? Ensures the use of a consistent prevention/intervention approach on a district-wide basis? Reduces the inappropriate identification of children as students with a disability (SWD)? Reduces the disproportionate representation of children from racial/ethnic groups as SWD? Improves access to challenging curriculum and instruction for all learners? Does the district have in place practices (e.g., accelerated learning strategies, differentiated instruction, before/after/summer experiences, etc.) to systematically accelerate the learning of students who are below grade level to ensure that they graduate at or above grade level? Has the district eliminated barriers and improved access to accelerated content and placement, including dual credit options, for high-able learners? o Has the district ensured equal access to accelerated content and placement for all students, particularly those from traditionally underrepresented groups? Parent/Family, Community, Student Engagement: Do students generally believe the daily classroom work is challenging and relevant? Do students generally believe that homework is value added and extends their learning? Do parents generally believe the district/school curriculum is challenging and is adequately preparing their child for post-secondary opportunities? Identify and Affirm Critical Focus Areas An understanding of the factors that contribute to each of the priority critical problems is necessary to set goals and identify strategies to address those needs. Only those problems designated as high priority will be addressed in the improvement plan. The district will use the information from Stage I to consider the causes of these problems. The Decision Framework Profile identifies upper level causes. The Decision Framework results identified one to two performance priority areas that emerged from the data and data analysis in level I, e.g., reading and mathematics. These should be set aside while the relationship among the remainder of the causes is examined. The expectations and conditions focus area will be identified from the review of these relationships. Relation diagrams are sometimes used to study the relationships between causes. The interrelationship diagram is used when a complex issue is studied with multiple causes. Resource 11 describes a process to create a relation diagram. At the conclusion of examining the relationships, the DLT should have a clear idea about the expectations and conditions priority area. If there is more than one priority area that is identified, the DLT will need to prioritize and select the one area that will be addressed in the district plan. Causes OIP Process Guide 52 of 95 December 2008 that do not relate to the area selected will need to be temporarily set aside for Stage 2 consideration (strategies and actions). At this point the results from the Decision Framework are presented, interpreted, and prioritized by the district leadership team. It is very important that the district be focused on a limited number (i.e., two to three) of priority needs. Focusing on more than that will cause staff, students, and families to be overwhelmed and struggle to respond to them effectively. Looking at the list of critical district problems, the DLT identified one to two performance priority areas that emerged from the data and data analysis in Level I. The DLT also identified one priority area from the Level III and IV data and data analysis that applies to the district as a whole and does not directly affect the content priority area(s). For example, the data may show that students who have low proficiency in reading are also the students who have poor attendance. Attendance data emerge from Level III but relate to reading and may be one cause of low reading. Therefore, this critical problem is not universal for all students or adults. However, if the data from Level III show that there are insufficient processes and procedures that are consistently implemented across the district; this may be a critical problem that focuses on adult behaviors and environmental issues. This would then become the expectations and conditions priority area. The content priority area(s) and expectations and conditions priority area become the district goals. Before beginning Stage 2, the DLT should take some time to affirm that the right content and expectations and conditions priority areas have been selected. This will occur through dialogue among the DLT members. Looking back at how the questions for this area were answered in the DF, the DLT should ask the following questions: 1) Were the responses to the questions accurate and reflective of data versus opinion? 2) Are the data to support the selection of these priority areas strong, for example, quantitative and qualitatively balanced, high response rate on surveys? 3) Is the ratio of questions to the rating high? 4) Do these areas have the leverage to improve student learning and change teacher practice? Once the DLT is satisfied with the answers to these questions, it is ready to begin development of the focused plan (Stage 2). The DLT may also want to consider what the district could do to improve the Stage 1 process. Suggestions should be recorded. OIP Process Guide 53 of 95 December 2008 Ohio Improvement Process Stage 2: Develop Focused Plan OIP Process Guide Create SMART Goals Develop Research-Based Strategies and Indicators Produce Research-Based Actions and Align Resources 54 of 95 December 2008 Introduction A plan is strong only if reliable and useful data have been collected. Developing a few (i.e., two to three) district goals in two areas (i.e., student performance and conditions and expectations) that staff members must act upon form the cornerstone of the plan. These goals and the strategies to achieve the goals are created from the causes of the most important and critical problems. Selected strategies must be research- or evidence-based and consistently used throughout the district. The indicators for each strategy must be quantifiable and provide the yardstick by which success is measured. Actions are the bricks and mortar of the plan, providing the specific events that will occur to accomplish the strategy. Once the actions are identified and the plan is assembled (mission, goals, strategies, indicators, actions), it is reviewed in draft form by many stakeholders, including a public review of the plan that is called for by the local board. Once the draft plan has been reviewed, it is finalized and adopted by the local board. Following adoption of the plan, responsibilities and resources for plan implementation are developed. The outcomes expected from Stage 2 include the following: ▪ Develop focused SMART (Specific, Measurable, Achievable, Relevant, Timely) goals ▪ Determine prioritized cause-and-effect relationships ▪ Compose strategies for each goal ▪ Establish indicators and baseline and progress measures for each strategy ▪ Create actions that have the greatest likelihood of increasing student performance and changing teacher-leadership practices ▪ Review, revise, and adopt a district plan Create SMART Goals Before meeting with the District Leadership Team to develop goals (sixth DLT session), the OIP facilitator will need to meet with the DLT chair/cochairs to build an agenda, prepare packets, and make meeting arrangements. The purpose of this meeting is to develop SMART goals. The meeting management checklist (Resource 8) can be used as an aide in preparing for the meeting. The agenda should be sent at least a week before the meeting. Request a copy be emailed to you. A sample agenda for this DLT meeting follows. Within each agenda item, time should be built in for questions. A key concept to emphasize with the Superintendent and chair/cochairs is that the district will have two to three focused goals. Therefore, if the district has a current CCIP with more goals than two to three, multiple plans with multiple district goals, or goals that do not match the two types of goals identified above, the OIP facilitator will need to work with the Superintendent/chair/cochairs to determine how these current goals will either be abandoned, revised, or merged with the goals that are developed based on the Decision Framework priority areas. If the district previously has established goals that align with the two types of goals, they can serve as a beginning point or reference. OIP Process Guide 55 of 95 December 2008 SAMPLE AGENDA District Leadership Team Meeting Date: December 20, 2008 Time: 8:30 a.m.–10:30 a.m. Location: Tactful School District Office Attendees: Superintendent T. B. Wise; Curriculum Director F. T. Child; DLT Cochairs; DLT Members; OIP Facilitators Ima Winner and Ura Saint 8:30 a.m.– 8:45 a.m. Meeting Purpose, Ground Rule Review, and Assignments Superintendent 8:45 a.m.– 9:00 a.m. Communicate Research Base and Importance of Focused Goals OIP Facilitators 9:00 a.m.– 9:15 a.m. Goal Criteria and Examples OIP Facilitators 9:15 a.m.– 10:15 a.m. Create District Goals 10:15 a.m.– 10:30 a.m. Next Steps DLT Members Superintendent Summary of Discussion/Decisions: During this meeting, the DLT will create goals based on Level I and Level III of the Decision Framework (DF). The OIP facilitator will need to introduce the concept of a SMART goal and show how these differ from goals that have been developed in the past. The goal-setting process should take no more than a few hours. Following are talking points that follow the above agenda. AGENDA TOPIC OIP FACILITATOR TALKING POINTS HANDOUTS Purpose of meeting is to create SMART goals. Meeting Purpose, Ground Rule Review, and Assignments OIP Process Guide Review the agreed upon ground rules that were identified at the first DLT meeting. Determine who will fulfill assignments for this meeting, for example, timekeeper, process observer, information seeker. 56 of 95 December 2008 AGENDA TOPIC Communicate Research Base and Importance of Focused Goals OIP FACILITATOR TALKING POINTS HANDOUTS Relationship to Leadership (DLT) Role Grain Size Visual— Resource 12 o Setting focused, realistic, and measurable goals is central to an effective planning process. o An analysis of 27 studies conducted since 1970 by the Mid-continent Research for Education and Learning (McREL) found that when district leaders establish goals and keep those goals in the forefront, there is a strong correlation to improved student achievement (Waters & Marzano, 2006). o According to the Ohio Leadership Advisory Council (OLAC), there should be a limited number of district goals that allow for concentrated focus on the core work that needs to be done to improve student performance (Ohio Leadership Development Framework, 2007). Importance of Goals o Help define the district’s mission. o Provide direction and focus to the district’s work and they help avoid chaos. o Help motivate staff by clarifying and communicating what the district is striving to achieve. o Help staff and leaders become aware of problems in a timely fashion, which in turn leads to healthy solutions. o Help the district plan ahead and be prepared. o Serve as a basis of recognizing and measuring accomplishments and successes. Types of Focused Goals o Student Performance Goals focus specifically on closing a gap between current student performance and preferred performance levels. These goals generally begin with “All students. . . .” o Expectations and Conditions Goals improve or increase the opportunities or potential for improved learning. These goals often focus on adult and/or student social behaviors and environmental issues. Note: It is possible that a district may need to develop an additional goal in a cross-content or system wide area. OIP Process Guide Goal, Strategy, Action, Indicator Definitions 57 of 95 December 2008 AGENDA TOPIC OIP FACILITATOR TALKING POINTS HANDOUTS Share Resource 12 and explain the differences between goal, strategy and action in terms of the grain size and the difference between performance indicator (goal) and progress indicators (strategies). This should be revisited when strategies and actions are developed. Difference Between District and School Plans It will be helpful to demonstrate the difference between a district and a school plan. The key differences to emphasize are: Goal Criteria and Examples o District goals and strategies will be used by building leadership teams (BLTs) to create school actions. o The district has the responsibility to approve school plans. o The district will need to provide resources and support to schools in the implementation of plans. o Buildings cannot add or modify goals or strategies but based on building data may not address all strategies in the school improvement plan. Goals need to be written using SMART criteria. Review the SMART criteria, providing examples of goals that meet and do not meet the criteria. It may be useful to explain the SMART of each goal by providing an anatomy of a poor and acceptable goal. See Resource 13 for acceptable goal examples broken down by the SMART criteria. Resource 13 also provides an illustration of a district and school plan that may also be used by the OIP facilitator in thinking about the categories of expectations and conditions goals and ways to describe measures for both types of goals. Sample Goals— Resource 13 S = Specific M = Measurable A = Attainable and Achievable R = Realistic and Relevant T = Timely Performance indicator is the gauge by which a goal is determined to be achieved. Performance indicators usually are written quantitatively, such as return on investment (ROI) for financial goals, customer satisfaction rates for parent and community engagement goals, reduction in time for internal business process goals, or increases in student performance for learning goals. OIP Process Guide 58 of 95 December 2008 AGENDA TOPIC Create District Goals OIP FACILITATOR TALKING POINTS A key concept to emphasize is that the district will have two to three focused goals. Therefore, the Superintendent and/or chair/cochairs will need to explain how the district has elected to create two to three focused goals. HANDOUTS SMART Goal Checklist— Resource 14 Creating goals may be done by dividing the number of district leadership team members by the number of goals. Each group will then draft one goal which is then presented to the remainder of the team. The whole DLT then evaluates each goal, making any changes necessary to ensure it meets the criteria for a SMART goal. Resource 14 provides a checklist for this purpose. Once the goals are drafted, the OIP facilitator should ask the DLT to respond to the following questions: o Is the goal clear and easily understood? o Can it stand on its own? o Is it in language that students, parents, educators, and the community can understand and not misinterpret? If the language is unclear, it may be helpful to create a two-column chart (see below) and guide the DLT through a discussion of each goal. What It Isn’t What It Is The Superintendent will need to share how the goals will be approved by the Board of Education. Next Steps The chair/cochairs should summarize the next steps to include: o Date of next meeting o Draft agenda for next meeting After the session, it becomes the superintendent’s responsibility to secure the approval of the local board of education. Endorsement of the goals is of paramount importance at this juncture in the process as the goals are the platform for the remainder of the plan. The approved goals should be sent to the DLT as the basis for developing strategies. Periodically, a checklist can be used to evaluate meeting effectiveness over a period of time. This allows everyone in the group to provide written feedback. An example of a checklist is provided at the end of the previous section. OIP Process Guide 59 of 95 December 2008 More on Facilitation Communicating the criteria for SMART goals can be aided by the OIP facilitator knowing the criteria in detail. The following provides detailed explanation of each criterion. OIP facilitators will find that developing a goal is not as simple as it appears. There are generally three hurdles that DLTs experience as they develop a goal. These are: First Hurdle: Timeline - Goals are generally multiple year and DLTs often struggle with when to set the end point, particularly in light of federal requirements that impose deadlines. The plan, however, is not an accountability plan but a performance improvement plan. Therefore, the DLT should consider a timeline that is realistic, generally two-three years. Second Hurdle: Reasonable and Realistic Measure - All grades in all content areas are not ordinarily at the same starting point (baseline measure) nor are subgroups within those grades and content areas. TIP: Facilitators should be Districts generally want to set a specific percent of prepared to help the DLT improvement to close a performance gap, e.g., 7% increase answer the following questions in proficiency. However, this percent may not be to overcome the three hurdles appropriate for all grades or subgroups. One option is to to goal development: select a percent and then determine how that percent 1) What is the appropriate increase, e.g., 7%, will affect all grade levels. For example, timeline for a goal? if 3rd grade reading is at 85.5% proficiency with the special 2) What is a reasonable and education subgroup at 65%, 4th grade at 78% with the realistic measure for the goal special education subgroup at 56%, 5th grade at 53.8% with that addresses all grade the special educations subgroup at 40%, is 7% gain levels and subgroups? reasonable and does it set the district on the right trajectory 3) What summative for all students to be proficient? Also, is the 7% calculated assessment will be used to from the baseline or added to the baseline (Does 53.8% go measure progress for all to 60.8% or 57.6%?) Averaging percents across grade levels and grades/subgroups is not advisable. Another option is to set subgroups? a separate increase for each grade/subgroup. These are sticky questions that must be tackled if the goal measure is to reasonable and realistic. Third Hurdle: Assessment Instrument for Measurement – Identifying a summative assessment for grades 3-8 and 10 is relatively easy as the OAT/OGT provides this annual data for multiple years. Districts do not always have summative data for grades K-2, 9, 11 and 12 and may not have a clearly articulated assessment system. The OIP facilitator should help the district describe its current assessment system. Gaps in the system may inform strategies, progress indicators and actions. Data to measure goals is usually available annually. Some instruments can be used both for summative (annual) assessment and formative assessment. For example, the Dynamic Indicators of Basic Early Literacy Skills DIBELS provides data that measures from beginning to end year (summative) but also provides progress monitoring data on a monthly or more frequent basis (formative). Most reading and math series provides formative assessment data such as skill or theme tests but may not provide summative data. Some assessment instruments only provide summative data, such as the Ohio Achievement Test. If the district has only state assessment data available, it may choose to only include those grades tested in the goal until such time that data for the other grades is available. OIP Process Guide 60 of 95 December 2008 SMART Goal Criteria Specific Goals should be straightforward and emphasize what the district wants to happen. Specifics help focus efforts and clearly define what the district is going to do. Specific is the what, why, and when of the SMART model: o WHAT are you going to do? Use action words such as direct, organize, coordinate, lead, develop, plan, build, and so on. o WHY is this important to do at this time? What does the district want to ultimately accomplish? o WHEN is the district going to do it? It is best to formulate goals that are very specific and clear. Instead of setting a goal to ensure that all students meet or exceed high academic standards, set a specific goal to ensure all students meet or exceed standards and benchmarks in reading. Measurable In the broadest sense, the goal statement is a measure for the district: If the goal is accomplished, then it is a success. Set goals that can be measured, so the district can see when change occurs. Goals should address what the district will see when it reaches the goal. For instance, phrasing a goal along the lines of “The district wants to improve reading by X percent” shows the specific target to be measured. In contrast, a goal phrased like “The district wants all students to read with comprehension” is not measurable. When the district measures progress, it stays on track, reaches its target dates, and experiences the exhilaration of achievement. Attainable and Achievable When the district identifies goals that are most critical to improving student performance, the district begins to figure out ways it can make them come true. By setting goals that are attainable and achievable, the district can develop the attitudes, abilities, skills, and financial capacities to reach the goals, and then it will begin to see previously overlooked opportunities to move closer to the achievement of its goals. Realistic and Relevant “Realistic” and “relevant” are not synonyms for “easy.” Realistic, in this case, means “doable.” It means that the learning curve is not a vertical slope, that the skills needed to do the work are within reach, and that the goal fits with the mission of the district. A realistic goal may push the skills and knowledge of the people working on it, but it shouldn’t break them. The goal needs to be realistic for the district but should not be so incremental that substantive growth is not expected. For example, a goal for all students to meet or exceed standards in reading may not be realistic for some districts, especially if a large percent of students are below proficiency. It may be more realistic to set a goal for a designated percent of students to meet or exceed standards in reading. The district can then choose to work toward reducing the percent in a realistic, yet ambitious effort. Sometimes these are referred to as stretch goals. Too difficult and the district is setting the stage for failure, but too low sends the message that the district isn’t very capable. Set the bar high enough for a satisfying achievement. Goals must also be relevant. Relevant goals address the most important and significant aspirations that the district must target in order to improve student learning. OIP Process Guide 61 of 95 December 2008 Timely It is important to set a timeframe that is measurable, attainable, and realistic for each goal. Putting an end point on the goal gives a clear target to work toward. If the district does not set a deadline, the commitment is too vague: Without a time limit, there is no urgency to start taking action, making it difficult, if not impossible, to achieve success. Throughout Stage 2 OIP facilitators will need to emphasize that overall improvement work, focused on how to implement a plan to improve student learning, is of greater importance than completing each plan cell perfectly. Facilitator judgment will determine when the right time is to introduce a plan template for optimal understanding and a readiness to think about implementation. At each point in the plan development, i.e., developing goals, creating strategies, identifying actions, determining measures, wordsmithing will occur. This can bog down the process; therefore, it is advisable that once the key concepts are agreed to, ask pairs of people to work on specific language and then bring back to the whole. During Stage 2, planning for monitoring (Stage 3) and evaluation (Stage 4) occurs. It is likely that monitoring and evaluation actions will be written into the plan. Therefore, it will be important to think about realistic resources (time, people, money) and processes for monitoring and evaluation. For each strategy and associated actions, the OIP facilitator should pose the questions: How will you implement the strategy and actions? How will you monitor the strategy and actions? Develop Research-Based Strategies and Indicators Before meeting with the District Leadership Team to develop strategies and indicators (eighth DLT session or continuation of seventh session), the OIP facilitator will need to meet with the DLT chair/cochairs to build an agenda and make meeting arrangements. The purpose of this meeting is to develop strategies and indicators. If the Decision Framework Results (by level) and Profile identifies a large number of areas with high priority, the DLT may need to use a process that examines the cause and effect among the areas in order to identify a manageable number of cross-cutting strategies. Resource 15 (Affirm Cause and Effect for Developing Strategies) provides a sample agenda and materials for the OIP facilitator to use in this situation. The meeting management checklist (Resource 8) can be used as an aide in preparing for the meeting. The agenda should be sent prior to the meeting. Request a copy be e-mailed to you. A sample agenda for this DLT meeting follows. Within each agenda item, time should be built in for questions. A key concept to emphasize with the Superintendent and chair/cochairs is that the district will have a few (no more than four) focused strategies for each of the two to three goals. A district strategy must influence education throughout the district or have significant impact on overall district performance. It must be flexible enough to enable different buildings, grade levels, disciplines, and/or administrative functions to craft actions OIP Process Guide 62 of 95 December 2008 appropriate to their setting, the groups they serve, and their data needs. Because of this, it is important that the membership of each goal work group be customized based on the goal. Each group must include building-level representation. Building-level representation is important because each building will use the goals and strategies as the foundation for creating its own school improvement plan. The individuals suggested on page 23 of this Guide are the likely candidates for goal work groups. Generally, goal work groups should be no more than 10 persons. If the group needs additional input on a specific strategy, select individuals should be brought into consult with the group, but they should not become a standing member of the group. SAMPLE AGENDA District Leadership Team Meeting Date: January 18, 2009 Time: 8:30 a.m.–11:00 a.m. and 1:00–3:30 p.m. Location: Tactful School District Office Attendees: Superintendent T. B. Wise; Curriculum Director F. T. Child; DLT Cochairs; DLT Members; OIP Facilitators Ima Winner and Ura Saint 8:30 a.m.–8:45 a.m. Meeting Purpose, Ground Rule Review, and Assignments Superintendent 8:45 a.m.–9:15 a.m. Strategy Criteria and Examples OIP Facilitators 9:15 a.m.–10:30 a.m. Create Research Based Strategies 10:30 a.m.–11:00 a.m. Evaluate and Refine Draft Strategies 11:00 a.m.–1:00 p.m. Lunch/Break 1:00 a.m.–1:30 p.m. Indicator Criteria and Examples 1:30 a.m.–2:30 p.m. Create Progress Indicators 2:30 p.m.–3:00 p.m. Evaluate and Refine Indicators DLT Members 3:15 a.m.–3:30 a.m. Next Steps Chair/Cochairs Goal Work Groups DLT Members OIP Facilitators Goal Work Groups Summary of Discussion/Decisions During this meeting, the DLT will create strategies based on the Decision Framework Profile results. The OIP facilitator will need to introduce strategy criteria, discuss what it means to be “research- or evidence-based,” and show how the strategies that will be developed differ from strategies that have been developed in the past. Showing OIP Process Guide 63 of 95 December 2008 examples of strategies from the CCIP and examples of well-developed strategies may be helpful. The strategy and indicator development process should take no more than a few hours. Following are talking points that follow the above agenda. AGENDA TOPIC Meeting Purpose, Ground Rule Review, and Assignments OIP FACILITATOR TALKING POINTS HANDOUTS Purpose of meeting is to create research-based strategies. Review Strategy and Indicator Definition in Resource 12. Defining strategy is important in the plan development process because educators typically reach for short-term remedies that are popular, such as authentic assessment, professional learning communities, and curriculum mapping, without having an overall strategy that justifies the choice or allows the district to evaluate whether the remedy is working to achieve goals. Grain Size Visual— Resource 12 Review the agreed upon ground rules that were identified at the first DLT meeting. Determine who will fulfill assignments for this meeting, for example, timekeeper, process observer, information seeker. Strategies are action oriented and they describe the key approaches the district will implement. They are written as specific, measurable statements about what is going to be accomplished to meet a need and get closer to reaching a goal within a given timeframe. Strategy Criteria and Examples Strategy criteria are as follows: Limited to a reasonable number per goal (2-4) Focused Feasible Practical Based on decision framework profile results Consistent with current research Written using clear jargon-free language that is able to stand on its own without additional explanation Often multiyear To help the goal work groups prepare to write strategies, sharing a sample strategy that meets the above criteria and working as a group to build a sample strategy (learn by doing) will help them know what is expected and allow for questions to be raised that may inform the process. It may also be helpful to show strategy examples from the CCIP and compare them to the sample. A sample goal and strategy might look like this. OIP Process Guide 64 of 95 December 2008 AGENDA TOPIC OIP FACILITATOR TALKING POINTS HANDOUTS Goal: By 2011, all students in Grades K–12 will improve performance on the OAT/OGT (grades 3-8 and 10) and local summative assessments (grades K-2, 9 and 11) by ____ % each year in reading. Strategy: Write the standards-based curriculum so as to focus every administrator, teacher, staff member, and student on understanding and application. Create ResearchBased Strategies TIP: Emphasize the importance of accelerating subgroup performance to meet all school progress. The purpose of this activity is to identify cross-cutting ideas that will result in a manageable number of strategies. Using the Decision Framework Profile, share the recurring ideas and seek agreement (manageable number, e.g., two to four) on the strategy categories. This may require prioritizing and merging of ideas. Divide each goal work group by the number of strategy categories and have each subgroup write a first draft strategy statement that will address the needs listed. Once the strategies are drafted, they need to be checked against the most current research available on the topic. This task serves two functions: 1) to help provide focus to the strategy and 2) to increase the likelihood of improving student performance, assuming that the strategy is successfully implemented. The Ohio Department of Education defines researchbased practices as the process of reviewing, assessing, and applying proven strategies to address data-determined needs. Research-based solutions should be evaluated on two dimensions: quality and relevance. The questions that need to be answered are as follows: o o o o o OIP Process Guide Decision Framework Results To what degree are our strategies grounded in research or evidence? Are there systematic, empirical methods that draw on observation or experiment? Is there rigorous data analysis that is adequate to test and justify the general conclusions drawn? Has the strategy been evaluated using experimental or quasi-experimental designs with appropriate controls to evaluate the effects? Is there sufficient detail and clarity to allow for replication? 65 of 95 December 2008 AGENDA TOPIC OIP FACILITATOR TALKING POINTS o HANDOUTS Has the strategy been accepted by a peer-reviewed journal or approved by a panel of independent experts? A source or explanation should be provided in response to each question. If the group cannot answer a question, then it may need to call on others in the district or search for the answers through the Internet or by interviewing experts in the field. Once the group feels confident that the strategies it has developed are based on prioritized data needs from Stage 1 and are grounded in scientific- or evidence-based research, the group is ready to present to all the goal work groups. Evaluate and Refine Draft Strategies Each of the goal work groups will need to share its strategies with each other. These should be provided in writing and also displayed so that all participants can see them. It is suggested that they either be written on flip chart paper or projected on a screen. Strategy Evaluation Checklist— Resource 16 As each goal work group presents, it will need to describe how the strategy meets the above criteria and respond to questions from the other goal work groups. Each group then should meet to make any necessary revisions and submit the revised statement to the DLT member of their group. The district leadership team will need to meet to review the strategies, examining them for redundancy, overlap, and coherence in order to ensure a reasonably structured set of strategies. Either the DLT or the Goal Work Groups should review all goals and strategies and evaluate the strategies using the Strategy Evaluation Checklist—Resource 16. A strategy indicator is the gauge by which a strategy is determined to be met. There are two types of strategy indicators: adult implementation and student performance. Strategy Indicator Criteria and Examples A baseline measure is established for each type of indicator. Short-term progress measures are set to assess degree of changes in knowledge, skills, attitudes, policies and/or practices and student performance. It is easiest to begin from the baseline and list progress measures that demonstrate a change, e.g., increase in percentages. It is also advisable to identify the data source for each indicator, e.g., as measured by____. Indicators are developed for each strategy, generally one adult implementation and one student performance, although it is possible and desirable, that some of the OIP Process Guide 66 of 95 December 2008 AGENDA TOPIC TIP: Data sources for indicators may include classroom observations, survey data, formative assessment results, analysis of lesson plans or team meeting notes. OIP FACILITATOR TALKING POINTS HANDOUTS same indicators will be used across strategies. That is why indicators cannot be finalized until all strategies have been developed. The questions that the district leadership team needs to ask itself are as follows: o o o What evidence would make us feel we are making progress? How can we collect this evidence? Of all the measures we could have chosen, why did we chose these? For each strategy, two types of measures will need to be developed: o o Baseline measures: These help to identify the starting point for change and provide a reference point in identifying realistic progress measures. Progress measures: These assess movement toward achieving strategies. They are shortterm (e.g., quarterly for districts or monthly for buildings) measures of changes in knowledge, skills, attitudes, policies, and/or practices and student performance. They help to determine whether the district is improving. Strategy indicator criteria are: Data should be available, reliable, accurate, valid and reflect reality Data should be able to be collected on a regular basis, e.g., quarterly for districts, monthly for buildings. Data should be understandable, meaningful and easily communicated Cost of getting data should be acceptable The OIP facilitator shares an indicator that meets the criteria and build a sample indicator (learn by doing) with the group to help the group know what is expected and allow for questions to be raised that may inform the process. This may be done in pairs, triads, or as a whole group depending on size of group and time allotted to task. A sample goal, strategy, and indicator might look like this: Goal: By 2011, all students in Grades K–12 will improve performance on the OAT/OGT (grades 3-8 and 10) and local summative assessments (grades K-2, 9 and 11) by ____ % each year in reading OIP Process Guide 67 of 95 December 2008 AGENDA TOPIC OIP FACILITATOR TALKING POINTS HANDOUTS Strategy: Write the standards-based English Language Arts (ELA) curriculum so as to focus every administrator, teacher, and student on understanding and application. Adult Implementation Indicator: 100 percent of K–12 teachers, students, and administrators in the district will consistently implement the district standards-based ELA curriculum as evidenced by an analysis of lesson plans and classroom observations. Baseline Measure: Less than 20 percent of K– 12 teachers, 5 percent of students in Grades K– 12, and 35 percent of administrators can articulate the district K–5 standards-based curriculum. Level of implementation is unknown. Progress Measures: 10% increase every four months Create Indicators TIP: Identify a goal and/or strategy manager who oversees implementation. This provides opportunity for shared leadership across the DLT. OIP Process Guide Goal work groups will need to review the final draft of the strategies. All data applicable to the goal and strategy should be made available to the group. The groups should work on the same strategies they developed in the morning, and devise baseline and progress measures by responding to the following baseline and progress questions: Baseline Questions o What does the data say about where the district is in relation to this strategy? o If there is insufficient data to create a baseline, what and how will data be immediately collected to form a baseline? Progress Questions o How would it be determined whether the district was successful at the end of three years if this strategy were fully implemented with integrity? o What are the changes in knowledge, skills, attitudes, policy, and practices and student performance that should be seen during the course of three years? o What evidence will be needed to know if the changes occur? o What data need to be collected or are available to document the changes? o What procedures will be followed for collecting the data? 68 of 95 December 2008 AGENDA TOPIC OIP FACILITATOR TALKING POINTS o o How will the data be displayed and communicated? How will it be ensured the indicator(s) will be measured consistently and accurately? Each of the goal work groups will need to share their indicators with each other. These should be provided in writing and also displayed so that all participants can see them. It is suggested that they either be written on flip chart paper or projected on a screen. Evaluate and Refine Indicators HANDOUTS Indicator Evaluation Checklist— Resource 17 As each goal work group presents its indicators, it will need to describe how the indicators meet the above criteria and respond to questions from the other goal work groups. Each group then should meet to make any necessary revisions and submit the revised indicators to the DLT member of their group. The DLT will need to meet to review the indicators, examining them for redundancy, overlap, and coherence in order to ensure a reasonably structured set of indicators. Either the DLT or the Goal Work Groups should review and evaluate the indicators using Resource 17. The Superintendent will need to share how the strategies and indicators will be communicated to the board of education and how stakeholder input will be gathered and used. Next Steps The chair/cochairs should summarize the next steps to include: o o Date of next meeting Procedures for gathering stakeholder input o Draft agenda for next meeting After the meeting, the strategies and indicators should be written and sent out to the DLT in preparation for developing actions. Soliciting stakeholder input into the goals, strategies and indicators will occur at this juncture. There are several options for obtaining feedback from stakeholders about the goals, draft strategies, and indicators. A brief explanation of each follows. Electronic or Paper Survey Creating a survey can be a relatively easy and inexpensive method of collecting feedback about the goals, strategies, and indicators. Some surveys use open-ended prompts. This type of survey often is mailed to stakeholders who then complete the survey and mail it back to the district. Resource 18 provides a sample survey. The district leadership team reviews the responses and makes adjustments as they deem appropriate. OIP Process Guide 69 of 95 December 2008 Surveys also can be electronic. There are websites such as www.surveymonkey.com where surveys can be produced and analyzed for free (depending on the number of respondents and the complexity of the survey) or at minimal cost. This type of survey can use a combination of forced responses and short-answer questions. The benefits of an online survey include their low cost to administer and that the software program analyzes the data. The data also can be sorted by respondent group. The downside of using this technology is that it may not be readily available to all stakeholders from whom the district desires responses. Focus Groups As described earlier, focus groups, or focused group interviews, are facilitated group discussions in which an interviewer asks a group a series of questions. Group members provide responses to the question and a discussion ensues. District Leadership Team Interviews Another method to gather stakeholder input about the goals, strategies, and indicators is for each member of the district leadership team to interview five persons who represent their constituent group and meet specified conditions. Questions for the interviews should include qualitative and quantitative questions. Once the indicators are finalized, they should be included in a spreadsheet or project management tool which can easily be updated and forwarded to district leadership team members. An interview protocol can be found in Resource 20. More on Facilitation The amount of empirical research to support educational improvement is somewhat limited because it requires the publication of findings in refereed journals (scientific publications that employ a process of peer review), duplication of the results by other investigators, and a consensus within a particular research community about whether there is a critical mass of studies that points toward a particular conclusion. Even if educators have access to the scientific evaluation, the research literature sometimes fails to provide clear direction. In these cases, educators must rely on their own reasoning processes as informed by experience. Produce Research-Based Actions and Align Resources Before meeting with the DLT to develop actions (ninth DLT meeting), the OIP facilitator needs to meet with the DLT chair/cochairs to build an agenda and make meeting arrangements. The purpose of this meeting is to develop actions for each strategy. The stakeholder input should be reviewed prior to this meeting, and goals, strategies, and indicators revised by the Superintendent and chair/cochairs if necessary. These should be sent to the DLT prior to this meeting. The meeting management checklist (Resource 8) can be used as an aide in preparing for the meeting. The agenda should be sent prior to the meeting. Request a copy be emailed to you. A sample agenda for this DLT meeting follows. Within each agenda item, time should be built in for questions. The most important thing to remember is that each action for achieving the goal or strategy should either have a direct impact on students or indirectly impact students, such as ongoing professional development and capacity building. OIP Process Guide 70 of 95 December 2008 Each district will approach action development differently depending on how they are organized. For example, if a district has groups or teams that work on specific initiatives, for example, Reading First, Technology, Professional Development, then these are the groups or teams that need to be organized for this task. If a district has departments that have defined work plans (written or unwritten), for example, Curriculum or Teaching and Learning Department/Team, Assessment Department/Team, then these are the groups or teams that need to be organized for this task. Basically, any structure (team, department, work group) within a district that has responsibility for a plan or scope of work needs to engage in this part of the process. Each district group/team/department needs to review its current plans, programs, and practices for three purposes. 1. The first and most important purpose is to decide which actions should or should not continue based on whether they are consistent with the Stage 1 results. It is important to remember that the plan will have a limited number of actions to implement the strategies and achieve the goals. 2. The second purpose is to ensure that the actions meet federal, state, and district requirements, for example, districts are federally required to coordinate and integrate services include appropriate actions to promote effective parent involvement. 3. The third purpose is to ensure that proposed actions are grounded in research. NCLB requires districts to use scientifically based research to guide decisions about which actions to implement. There are many actions a district could identify to implement a strategy. As the Institute of Education Sciences points out, many of these actions claim to be able to improve educational outcomes and, in many cases, to be supported by evidence. This evidence often consists of poorly designed and/or advocacy-driven studies. Districts must sort through these claims to decide which interventions merit consideration. This is one of the most difficult parts of the process to facilitate as it generally requires abandonment or modification of programs and/or initiatives that someone or some group in the district is involved with. It is important to remind the DLT that the intent is to have one focused plan that drives district work, not another plan to add to existing plans. OIP Process Guide 71 of 95 December 2008 SAMPLE AGENDA District Leadership Team Meeting Date: February 15, 2009 Time: 8:30 a.m.–11:00 a.m. and 1:00 p.m.–3:30 p.m. Location: Tactful School District Office Attendees: Superintendent T. B. Wise; Curriculum Director F. T. Child; DLT Cochairs; DLT Members; OIP Facilitators Ima Winner and Ura Saint 8:30 a.m.– 8:45 a.m. Meeting Purpose, Ground Rule Review, and Assignments 8:45 a.m.– 9:15 a.m. Review Goals, Strategies & Indicators and Action Criteria Chair/cochairs 9:15 a.m.– 10:30 a.m. Keep, Drop or Modify Actions DLT/Goal Work Groups 10:30 a.m.– 11:30 a.m. Generate Actions DLT/Goal Work Groups 11:30 a.m.– 1:00 p.m. Break/Lunch 1:00 a.m.– 2:00 p.m. Prioritize Research- or Evidence-Based Actions 2:00 a.m.– 2:45 p.m. Consideration of Universal Actions DLT Members 2:45 a.m.– 3:00 p.m. Affirmation of Actions DLT Members 3:00 a.m.– 3:15 p.m. Monitoring Approach DLT Members 3:15 a.m.– 3:30 a.m. Next Steps Chair/Cochairs Superintendent OIP Facilitators/ Chair/Cochairs Summary of Discussion/Decisions OIP Process Guide 72 of 95 December 2008 During this meeting, the DLT will create actions for each strategy. The action development process may take several hours. Following are talking points that follow the above agenda. AGENDA TOPIC Meeting Purpose, Ground Rule Review, and Assignments Review Goals, Strategies, Indicators and Action Criteria Keep, Drop, Modify Actions TIP: Facilitators will want to continually ask the DLT: OIP FACILITATOR TALKING POINTS Purpose of meeting is to create research-based actions or specific steps to operationalize the strategy and achieve the goal. Review Action Definition in Resource 12. Determine who will fulfill assignments for this meeting, for example, timekeeper, process observer, information seeker. Engage the DLT and goal work groups in reviewing the Board of Education approved goals, strategies and indicators. A thorough understanding of these is the basis for developing the district and building level actions. BoardApproved Goals, Strategies, Indicators Criteria for Actions: Specific, discrete, actionable, and relevant. Often short-term and can be assigned. Each action statement should begin with a verb. Depending on the number of current plans the district has, it may be necessary to use groups for this task. Using current plans (also may need to consider applications, performance reports/agreements, event calendars, and other documents which require the department to take action). Match each action to the approved strategies. Keep actions that do not fit with one or more strategies on a separate list for future discussion with the DLT. It is important that district staff know that the final set of actions will become the work for which they will be held accountable. In addition, every district department/team may not have actions for every goal and/or every strategy. Determine whether each action should be kept (K), dropped (D), or modified (M). If an action requires modification, make the adjustments in some means that will make proposed changes obvious. The following criteria may be used when deciding whether to keep, drop, or modify an action: o OIP Process Guide Grain Size Visual— Resource 12 Review the agreed upon ground rules that were identified at the first DLT meeting. How does the improvement work relate to the on-going work of the district? How do these actions replace and change the work rather than “add on” to the work of the district? HANDOUTS If the action is fully completed, will it contribute to implementation of the strategy? 73 of 95 December 2008 AGENDA TOPIC Generate Actions OIP FACILITATOR TALKING POINTS o Does the action reach the targeted student population and content area(s)? o Does the action reach a critical mass of targeted school staff, students, and/or families? o Given the overall goal of improving student performance, do the benefits outweigh the costs, i.e., time, people, money, materials, supplies, technology? Identify possible new actions by examining the causeand effect diagrams and/or the Decision Framework Profile to check that the priority causes are addressed either by existing actions or by new actions. It is possible that a cause may need to be worded as a possible action. An example of reworking is as follows: HANDOUTS Decision Framework Profile Cause: The bargaining agreement prevents us from offering after professional development. Possible action: Negotiate the bargaining agreement to allow after school professional development on a pilot basis. Determine which actions have strong or possible evidence of effectiveness once a complete list of possible actions by strategies has been developed. This can be accomplished by asking two questions: Prioritize Research- or Evidence-Based Actions o Is the action/intervention backed by strong evidence of effectiveness? o If the action/intervention is not backed by strong evidence, is it backed by possible evidence of effectiveness? If the answer to both questions is negative, then the district should conclude that the proposed action or intervention is not supported by meaningful evidence. This information should be strongly considered when the DLT prioritizes actions. Before the DLT prioritizes, it should identify overlaps, conflicts, relationships, and logical sequence between and among actions listed for each strategy. Combine actions that duplicate or overlap. Review the entire set of actions for each goal and strategy and ask the group to think about these questions: o OIP Process Guide Will this set of actions allow the district to 74 of 95 December 2008 AGENDA TOPIC OIP FACILITATOR TALKING POINTS o o o HANDOUTS accomplish its goals and strategies and enable the district to meet the indicators that have been set? Is this the right set of actions to sufficiently address the goals? Can this set of actions be accomplished within one year)? Are the actions supported by strong or possible evidence-based research? Using “dot voting,” reach consensus on critical actions for each strategy. In order to have systemwide coherence, the district will need to take the set of actions from the above tasks and examine them through universal actions: professional development, communication, technology, resource management, and data. This task can be accomplished by small groups for efficiency. For each strategy and set of actions, the group should ask the following questions: Consideration of Universal Actions o What, if any, professional development and support is needed for this action to occur? o Who needs to know about this action, when do they need to know, and how will they be informed about it? o What technology is needed for this action to be implemented efficiently and effectively? o What resources (time, people, materials, supplies, and funds) are needed to support this action? Will this require reallocation of resources? o What data are needed to support this action? How and by whom will the data be collected? When and for what purpose will it be collected? Are the data for evaluation and monitoring? Responses to these questions may result in additional actions for the district plan. The district also will need to examine each of the proposed actions to ensure that each meets the criteria of a good action statement. Consider whether the action is to be developed, implemented, evaluated, or monitored. Affirmation of Actions OIP Process Guide Once the above task has been completed, ask the whole group to review the plan for coherence and 75 of 95 December 2008 AGENDA TOPIC OIP FACILITATOR TALKING POINTS HANDOUTS alignment. Make any final adjustments as necessary. Complete the Plan Implementation Details Once all actions are complete, ask the DLT to identify the monitoring evidence/data sources that will be used to document that the action is implemented. Responsibilities, timelines and resources will also need to be assigned to each action. The DLT will want to strive for a balance of person/groups responsible for actions steps. Tasks for each action will be generated by the person/groups responsible for each action and reviewed by the strategy managers to ensure equitable distribution of assignments. Tasks are a list of activities that need to be taken for someone to complete an action. At this point the resources needed for each action can be stated in general terms, e.g., software license, printing costs, training materials for a specified number of individuals. At a later time, the treasurer or other person responsible for fiscal funding sources will develop detailed budget breakdowns that correspond to the implementation details. A discussion on the approach the DLT will take to monitor the entire plan needs to occur. Considerations may include: Monitoring Approach o Including actions in the plan that ensure monitoring occurs o Development of a separate document that describes the monitoring approach o Frequent, e.g., quarterly, discussion of monitoring results at DLT meetings Depending on the decisions reached, the DLT may need to schedule a separate session to discuss specific monitoring processes or it may identify a subcommittee to develop alternative proposals for presentation to the whole DLT. It is important that time and attention to this topic be scheduled. The Superintendent will need to share how the plan be reviewed, revised and adopted. Next Steps OIP Process Guide The chair/cochairs should summarize the next steps to include: o Date of next meeting o Draft agenda for next meeting 76 of 95 December 2008 In addition to assigning a position to each action, the district may choose to list other key personnel who also will work on the action. By doing this, the district indicates that there are multiple people who will actively participate in completing the action, yet one individual will still be held responsible and accountable for completion. After the meeting, the plan should be put into the appropriate template (CCIP, district developed or ODE provided—see Resource 21) and checked against the district plan criteria (see Resource 19). The plan should be edited and sent to the DLT and Goal Work Groups. Another important task after the strategies and actions are complete is to allocate and align resources to them. It is the DLTs responsibility to intentionally align resources to achieve the plan goals. In planning, a resource allocation decision is made for using all available resources, for example, human resources, funding in the near term, that is, year, to achieve goals and strategies for the future. Resource alignment and allocations have two rts: First, there is the basic allocation decision and second there are contingency mechanisms. The basic allocation decision is the choice of which items to fund in the plan during which year (if the plan includes multiple year strategies and actions), and what level of funding they should receive. It is suggested that districts have a solid idea of funding resources for each strategy for the next three years. There are two contingency mechanisms. There is a priority ranking of items, showing which items to expand or advance if more resources should become available; and there is a priority ranking of some items in the plan, showing which items could be reduced or postponed if funding is reduced. The DLT will need to consider the following basic tenets when aligning and allocating resources: ▪ Does our resource allocation reflect that student performance is our highest priority? ▪ Will our budget demonstrate accountability to our internal and external stakeholders? Are our budget decisions based upon relevant data? Will our budget process measure the results of our goals, strategies and actions? Does our budget leverage resources from a variety of sources in order to maximize full implementation of the plan? Are our resource allocation decisions based on population-based need in order to achieve results in increased student performance and changes in teacher practice? ▪ ▪ ▪ ▪ More on Facilitation TIMELINES Each action will need a timeline that provides a deadline for the action to be completed. The individuals who will become responsible for the action will create more detailed timelines in the future as they assign tasks to each action. OIP Process Guide 77 of 95 December 2008 The first activity in creating timelines is to draft a sequence of the actions in a three-year plan by indicating the implementation year. (The plan may be two to five years depending on the district’s approach.) Reviewing the strategy indicators will provide a basis for determining what and how much needs to be accomplished over a three-year period. Use a logical sequence and levels of implementation (development, implementation, monitoring, and evaluation) to guide the determination of the year to implement. Once this is completed, the team or group should undertake the actions identified for year one and complete the timelines in the plan format. The actions listed for years two and beyond should be documented and maintained for future plans. Because timelines are affected by a variety of unforeseen factors, it is advisable for the district plan to use a quarter-calendar system for documenting key deadlines. Quarters could correspond, for example, to a fiscal calendar or to the school year calendar. In any case, timelines must be realistic. Considering each action in terms of development, implementation, evaluation, and monitoring will help put actions into a logical sequence. As the timelines are being identified, it is suggested that the district create a plan calendar. This can be easily accomplished by making a chart of each month used in the above timeline and then abbreviating each action next to the month it is listed. This activity will help the district to determine if actions are realistic given the human resources needed to accomplish them and to ensure there is no duplication of effort. For example, if more than one action calls for teams to work on an action and the team members are likely to be the same individuals, the calendar will help identify if team members have schedule conflicts or if the pacing of actions for the team is realistic. Major actions that affect many people in the district may need to be placed on the official district calendar that is distributed to all employees. MONITORING EVIDENCE/DATA SOURCES Evidence of completion of each action will need to be documented and maintained in a portfolio by the program assistant. It is advisable to have no more than two forms of evidence for each action. It also is likely that the same form of evidence may apply to multiple actions. On the plan, evidence can be denoted in key terms or coded using a list of evidence that has been generated. Sources may include but are not limited to observation summaries, schedules, written policies or procedures, professional development training, analysis of logs, analysis of assessment results, meeting summaries and decisions. RESPONSIBILITIES Responsibilities in the plan need to be specifically referenced. They must designate accountability and show a clear relationship between the actions and who has responsibility for them. Some guidelines for assigning responsibility are: □ Do use position titles, e.g., English Language Arts Coordinator. □ Do not use specific names, e.g., Mary Smith. □ Do assign one position, e.g., Assistant Superintendent for Curriculum. □ Do not assign multiple departments, teams, positions, e.g., ELA Coordinator and Assistant Superintendent for Curriculum. OIP Process Guide 78 of 95 December 2008 □ Do provide a position title for someone who is employed by the district. □ Do not assign someone who is not an employee of the district, e.g., PTA President. □ Do assign a position to someone who is actively employed. □ Do not assign a position that is currently vacant. □ Do assign a position to someone who will be and can be held accountable for completing the action. □ Do not assign a position to someone who has no control or responsibility for the action. □ Do assign a position to someone who is close to and has a relationship to the action. □ Do not assign a high-level administrator to each action, e.g., indicating the superintendent is responsible for every action. RESOURCES (BUDGET, MATERIAL, TECHNOLOGY) Once action monitoring evidence/data sources, responsibilities and timelines are assigned, resources including a detailed budget of estimated costs will need to be developed in consultation with the district staff who are responsible for fiscal resources. Once the detailed budget breakdowns are completed, these will need to be rolled up by strategy. These budget figures become the basis for the district Comprehensive Continuous Improvement Plan (CCIP) funding application for the upcoming year. Integrating the CCIP with the resource allocation part of the district plan ensures that federal and state entitlement and competitive funds are used to support the district strategies. This process can be tedious but it is critically important. Frequently, individuals who work on budget line items arbitrarily allocate a lump sum figure to an action, e.g., $10,000, but do not take the time to think through the actual costs. Lump sum estimations usually are incorrect and result in budgets being under- or overestimated, creating spending difficulties for the district. It may be advisable to seek assistance from someone in the treasurer’s office during the development of the detailed budget. An example of a detailed budget is shown below: Example of a Detailed Budget Breakdown Action A.2: Examine and remedy, if necessary, the district curriculum to determine if important concepts and processes are well represented through pictures, figures, charts, video clips, or other graphic formats in addition to narrative descriptions. Object Code Detailed Budget Breakdown TOTAL X Salaries/Stipends 4 Elementary, 1 Middle & 1 HS ELA teacher leader @ 20 hours X $35/hr. $25,200 X Fringe Benefits 6 teachers leaders $700 X .0765 (FICA) $535.50 OIP Process Guide 79 of 95 December 2008 Example of a Detailed Budget Breakdown Action A.2: Examine and remedy, if necessary, the district curriculum to determine if important concepts and processes are well represented through pictures, figures, charts, video clips, or other graphic formats in addition to narrative descriptions. Object Code Detailed Budget Breakdown TOTAL X Materials/Supplies ELA graphic organizer laminated charts X 90 classrooms @50 ELA video clips X 90 classrooms @150 Bar coding labels for above $200 $4500 $13,500 $200 X Purchased Services External facilitator for 8 days @ $750/day including travel expenses $6000 X Capital Outlay SmartBoard @ $15,000 $15,000 X TOTAL $64,936 In creating a detailed budget, it is important to consider the following questions: Will staff be needed to carry out the action? If yes, will the staff be dedicated by: a) a certain percentage of a job, e.g., 0.20 FTE; b) position, e.g. four literacy coaches; c) stipends for beyond the work schedule, e.g., 20 hours per person, or d) substitute costs? Once it is determined who will be needed, the amount of money for the position or the stipend can be determined based on district negotiated amounts. If fringe benefits such as Social Security (FICA), and health and life insurance are needed, these will need to be calculated as well. Generally, if stipends are paid, FICA will need to be calculated at a rate of 0.0765. The detail will be written in the detail budget breakdown column of the plan with a total for that item included in the salary/stipend column. Will any materials or supplies be needed to carry out the action beyond what is provided by the district as part of its ongoing operational budget? If yes, the amount and types of materials and supplies will need to be determined. The treasurer should be able to provide common costs of materials and supplies. Typical materials and supplies include flip chart paper, markers, paper, and copier toner. Unique supplies or materials may include tables and chairs for a teacher planning room. Technology equipment that costs less than a certain amount (see district procedures) may be included in the materials and supplies column. Will other services be needed to carry out the action? If yes, the type of other services and the associated costs will need to be identified. Typical other services (sometimes referred to as purchased services) include travel and accommodation costs, rental costs, membership fees, maintenance costs, refreshment costs (for nonemployees), and contracted services. Again, the contractual agreement or district procedures likely will provide guidance on the amounts that can be budgeted. For example, lodging to attend a meeting may need to meet the state-approved hotel rate or consultant fees may be limited. Most districts do not allow the use of federal and state funds for refreshments for employees of the districts; these funds can be used for meetings involving parents and/or community representatives. OIP Process Guide 80 of 95 December 2008 Will any capital outlay be needed to carry out the action? It is unlikely that capital will be used when developing a budget breakdown because it covers items that are expensive, e.g., technology equipment over a specific amount (see district procedures), or assets that are related to district infrastructure, e.g., building materials or construction costs. Once all actions are estimated, the budget will need to be rolled up to the strategy and goal level. The DLT will need to review the resource allocation decisions by determining the benefits of the actions to implement the strategy and determining if there are any gaps between costs and resources. This may require the group to evaluate the overall strategy in relation to the following questions: o o o o o Which strategies were assigned the highest priority? Which actions are critical to accomplishing the strategy? Are the costs associated with the critical and noncritical actions proportionate? Are there actions that support more than one strategy? Are the actions relevant and culturally responsive, i.e., is there a contextual fit between the setting or culture in which the actions will be implemented? The funding source(s) for each action also will need to be identified. This should become the responsibility of the treasurer’s office in consultation with the DLT. If a gap exists between available resources and the costs of strategies, the DLT should examine whether the district is committing resources to discretionary activities that are not part of the plan. If this is the case, they should recommend to the superintendent and treasurer that resources be shifted from these activities to the strategies and actions in the plan. For example, if the district has always funded an event that has become a tradition, such as an arts showcase with limited student involvement, and this showcase does not directly relate to one of the strategies, this activity may need to be abandoned or another organization such as the local area arts council may need to be approached to take over responsibility. The guiding principle in allocating resources should be that the strategies outlined in the CCIP take precedence over any discretionary activities not in the plan. If a gap still remains, the goal work group can explore the following options: o o o o o Which actions can be eliminated or scaled down without jeopardizing the strategy? Which actions can be redesigned for greater cost-effectiveness? Which actions can be combined or leveraged to share resources and returns? Which actions can be postponed until new or additional funds are available? Which actions can be funded through alternate funding sources? It is important for the DLT to know what fund sources are available in the district so they may be considered during the budgeting of actions. The treasurer’s office should provide a list of funds available by amount and limits to the DLT when they begin the budgeting process. OIP Process Guide 81 of 95 December 2008 RESEARCH- OR EVIDENCE-BASED ACTIONS To answer the question of whether an action is research- or evidence-based, the district must look at the quality of studies (i.e., randomized controlled trials that are welldesigned and implemented) plus the quantity of evidence needed (i.e., trials showing effectiveness in two or more typical school settings, including a setting similar to that of the district, schools, or classrooms). If both quality of the study and quantity of evidence exists, the answer to the first question is positive. Types of studies that can be characterized as having possible evidence are randomized controlled trials whose quality and quantity are good but fall short of strong evidence and/or comparison group studies in which the intervention and comparison groups are very closely matched in academic achievement, demographics, and other characteristics. Other important factors to consider when considering implementing an evidence-based action or intervention are: ▪ Whether an evidence-based intervention will have a positive effect in the district or schools will depend on close adherence to the details of its implementation (fidelity). ▪ When implementing an evidence-based action or intervention, it is important to collect outcome data to check whether its effects in the district or schools differ greatly from what the evidence predicts. Although districts are encouraged to use scientific- or evidence-based research actions or interventions, the reality is that they are available on selected topics. The Ohio Department of Education has identified some of these for schools, such as those contained in the Ohio Professional Development Standards and work undertaken in the area of literacy. Once the list of actions is developed, the work group may wish to validate the actions against what strong evidence has been shown to work. If there are discrepancies, the group should make any needed course corrections. This may be in the form of adding a new action or modifying an existing action to make it more explicit. For example, if the strategy is “Strategy A: Write the standards-based curriculum so as to focus every administrator, teacher, staff member, and student on understanding and application,” actions may include the following: Action A.1: Evaluate the district curriculum to determine if important content is reviewed at least several weeks or several months after the time that it is first encountered by students. Action A.2: Examine and rectify, if necessary, the district curriculum to determine if important concepts and processes are well represented through pictures, figures, charts, video clips, or other graphic formats in addition to narrative descriptions. Action A.3: Determine the degree to which deep (as opposed to superficial) questions are included in the curriculum guides. Action A.3: Evaluate examples and instructional materials to identify which aspects of abstract concepts and their concrete representations are included in the curriculum guides. Facilitation tools such as dot voting, electronic response systems are tools that assist with consensus building. OIP Process Guide 82 of 95 December 2008 DOT VOTING Dot voting is used to prioritize items. 1. Revisit the items you are asking individuals to prioritize. 2. Allocate a designated number of dots to each person. The number is determined by the number of items, the number of participants, and the number of items you wish to select from the total. 3. Ask each person to use their allotted dots to vote on those items they think are most important. Before the voting begins, explain the guidelines. You cannot buy and sell dots. You cannot lobby a colleague to mark a statement. You cannot mark more than one statement. You cannot ridicule someone for their dot placement. 4. Reach consensus by: a) Reviewing the items with the most dots and seeking agreement from the group. This may be done by polling, standing up to show agreement, and so on. b) Reviewing the items with the least dots and seeking agreement from the group that these items will not be considered at this time. c) Discussing the remainder of the items and seeking agreement whether to keep or drop them. OIP Process Guide 83 of 95 December 2008 Review, Revise, and Adopt Plan Engaging a broad base of stakeholders to review the draft plan will help build ownership and commitment to plan implementation. Section 3302.04 of the Ohio Revised Code (S.B.55) requires districts to hold at least one public hearing (with at least two weeks prior notice) about the final draft of the plan prior to board adoption. Major federal and state plan requirements are included in the References should this information be needed. Depending on the degree to which the board desires to engage their constituents in reviewing the plan, there are several formats which could be used. Below is a range of options for the board to consider. STANDARD BOARD MEETING Boards hold regularly scheduled meetings and follow standard procedures under the Open Meetings Act to inform the public about these meetings. One option is for the board to put the plan before the public by placing it on the agenda of a regularly scheduled board meeting. This will necessitate the board either to make the plan available to the public prior to the meeting such as on the district website or to present the contents of the plan at the board meeting. PUBLIC HEARING The main purpose of most public hearings is to obtain public testimony or comment. Some form of public notice is required for all public hearings. Because all public hearings are considered public meetings under the Open Public Meetings Act, the notice requirements of Section 3302.04 of the Ohio Revised Code (S.B.55) must be followed. Board meeting chambers where public meetings usually are held often will be the best place to hold a public hearing. The primary concern is to provide time in the agenda for all attending members of the public to speak if they so desire. TOWN MEETING Town meetings create an opportunity for the general public to give the board direct, substantive feedback on the plan. Town meetings focus on discussion and deliberation among citizens rather than speeches, question-and-answer sessions, or panel presentations. Diverse groups of citizens participate in round-table discussions, deliberating in depth about the plan. DISTRICTWIDE MEETING Although there has been building- and district-level involvement in plan development, not all staff members will have had the opportunity to review the entire plan. Therefore, the board may wish to have an open meeting for anyone interested in reviewing the plan. This can be accomplished in several ways. The meetings could be held at each building in the district, by job-alike groups (e.g., principals’ meeting, teacher leader meeting, grade-level meeting) or for the entire district. Districtwide meetings could be held in similar fashion to the town meeting above. This also presents an opportunity for the district to explain the relationship between the district and school plans. The district leadership team may want to create a synopsis of the plan for distribution to all staff. Following the public hearing, there are a few tasks that need to be accomplished before the plan is submitted to the board for adoption. OIP Process Guide 84 of 95 December 2008 CONSIDER PUBLIC HEARING COMMENTS Incorporating the information from the public review allows the district one more opportunity to create a plan that is clear to all stakeholders and inspires everyone to embrace implementation. Each of the comments from the public hearing needs to be seriously considered by the DLT. A response to each comment should be developed in a format that can be shared with the local board and, if requested, with those who participated in the public review. REVISE PLAN Based on the decisions made in regard to the public review comments, the plan may need to be revised. If that is the case, the revisions need to be made and approved by the district leadership team. Once changes are made, it is suggested that a single individual (rather than a team) edit the plan and finalize any visuals that are used as illustrations in the plan. If the superintendent or board has any guidelines about style or format, these should be made clear at this time. The Ohio Department of Education has created a plan template that a district may wish to use (See Resource 21). Possible guidelines may include, but are not limited to, the following: □ □ □ Use (font), (font style), (XX)” margins for narrative portions and at least (XX) line spacing. Use the template provided by the Ohio Department of Education. Limit the number of pages to no more than XX pages, including cover page and table of contents. Page orientation may be portrait or landscape. Number pages (X of X). SECURE BOARD APPROVAL The final plan should be formally presented at a board meeting. The board may wish to hold a signing ceremony or other event to celebrate adoption of the plan and to recognize those who contributed to its development. The plan should immediately be placed on the district website. A summary of the plan should be made widely available and the local press should be informed. BUILDING LEVEL OIP Once the district plan has been approved, the DLT will need to facilitate the Ohio Improvement Process (OIP) with each building in the district. This will include working with Building Leadership Teams (BLTs) to complete the building-level Decision Framework and develop action plans that respond to identified critical needs and align to district goals and strategies. OIP Process Guide 85 of 95 December 2008 Ohio Improvement Process Stage 3: Implement and Monitor the Focused Plan OIP Process Guide Establish and implement collaborative structures, processes and practices that support a culture of inquiry and distributive leadership Implement the plan systemically and systematically Monitor and analyze changes in student performance and adult implementation and make and report course corrections to the plan 86 of 95 December 2008 Ohio Improvement Process Stage 4: Evaluate the Improvement Process OIP Process Guide Evaluate Plan Implementation, Impact and Changes Report Summative Progress Modify Instructional Practice and Revise Plan 87 of 95 December 2008 Conclusion Over the past 10 years, the Ohio Department of Education has provided guidance to districts that are involved in the continuous improvement process. This guidance has benefited schools and districts. However, the department recognizes that it needs to model continuous improvement and that the guidance and support needs to be constantly improved. The work that has gone into development of the OIP Facilitator’s Guide is the culmination of that improvement. This guide should be considered as a whole. A high-achieving education system accomplishes all the stages outlined in this guide. The result is a coherent plan that drives actions across the system at the district and school levels. The objective is not to simply comply with state and federal requirements; it is to improve education for every student in every school. Districts that have just finished using this guide to work through planning or are now implementing and monitoring the district plan will have accomplished or be on their way to accomplishing the following: □ Completed a comprehensive and systematic analysis of the district’s critical areas for improving student achievement through the decision framework process □ Focused on a few issues that have the greatest impact on student achievement by determining cause-and-effect relationships □ Developed a limited number of SMART (Specific, Measurable, Achievable, Relevant, and Timely) goals that respond to the most critical student performance needs □ Agreed on research-based measurable strategies to reach the goals □ Indicated a limited number of actions with purposeful timelines, and designated responsible person(s) and necessary resources to implement the strategies □ Determined focused, content-specific, high-quality professional development for administrators, faculty, and staff □ Identified specific parent involvement actions to meet the needs of parents and students □ Created a schedule and explicit steps to monitor strategies, actions, student performance, and adult practices □ Established methods/techniques to communicate the plan, plan progress, and results □ Engaged internal and external stakeholders throughout the process The participation of the board of education, superintendent, and district leadership team is essential for the process to be effective. They can contribute to school and student success when they are focused on fulfilling key leadership responsibilities and using the process described in this guide. These responsibilities include: □ Collaborative goal setting for achievement and instruction □ Board alignment and support of district goals □ Monitoring goals for achievement and instruction □ Use of resources to support achievement and instruction As districts improve through effective continuous planning, the planning process itself also will improve. It may be difficult to believe when the first written plan is just being implemented, but districts that are willing to continue focusing their efforts on the effective use of data and planning eventually will notice that the process seems effortless and that it is has become essential to their continued success. OIP Process Guide 88 of 95 December 2008 Acronyms Ohio Leadership Advisory Council AMAOs OSBA Annual measurable achievement objectives Ohio School Boards Association AYP PTA Adequate yearly progress Parent Teacher Association BASA SEA Buckeye Association of School Administrators State educational agency (in Ohio, the Department of Education) BLT SIP Building Leadership Team School improvement plan CCIP SPED or SpEd Comprehensive continuous improvement plan Special education CSLT SBE Community School Leadership Team State Board of Education DLT District Leadership Team ELL English language learners ESL English as a second language ESEA Elementary and Secondary Education Act LEA Local education agency (district) LEP Limited English proficient NCLB The No Child Left Behind Act ODE Ohio Department of Education OLAC OIP Process Guide 89 of 95 December 2008 Glossary Achievement Gap: The disparity in academic performance on tests among identified groups or the difference between how a group performs compared with what is expected of that group. Typically the disparity is defined in terms of white students and students of color, and between students who receive a free or reduced-price lunch and those who do not. Actions: Specific steps to operationalize a strategy and achieve a goal. Building Leadership Teams (BLTs): A team of individuals who promote a culture of common expectations or commitment by maintaining a school-wide focus on improving student achievement and who foster shared leadership and responsibility for the success of every child through the creation of purposeful communities. Capacity Building: Providing opportunities such as job-embedded staff development, coaching and time for reflection on effective instructional practices that enhance the ability of teachers and administrators to positively affect student learning. Community School Leadership Team: Refer to District Leadership Team Common Formative Assessments: Teacher generated periodic or interim assessments that are collaboratively designed by teams for specific units of instruction. Created as short matching pre- and post- assessments to ensure same- assessment to same-assessment comparison of student growth, common formative assessments usually contain a blend of item types, including selected response and constructed response, representing power standards. Consensus: After discussion, a group has reached consensus on a decision if most team members agree with the decision and if those who disagree are willing to accept the decision and try to make it work. Consensus allows those who disagree to gather more data and raise an issue again if indicated. Content Standards: Specific, measurable descriptions of what students should know and be able to do at each grade in each curriculum area. Continuous Improvement Framework: The concept that effective schools are engaged in a long-term process of improvement of teaching and learning that is demonstrated by a pattern of continuous improvement of learning for every child. The continuous improvement cycle includes determination of prioritized needs, planning for focused improvement, implementation of the plan, and monitoring and evaluation of the results. Cultural Competence: Demonstrating a set of values, behaviors, attitudes, and practices that enable people to work effectively across racial, ethnic, and cultural lines. Culturally Relevant Educational Practices: Using the cultural knowledge, prior experiences, frames of reference, and performance styles of ethnically or economically diverse students to make learning encounters more relevant and effective for them. Culturally Responsive Teaching: Understanding and applying knowledge of how culture can affect the ways in which children process and organize information, communicate verbally and nonverbally, and perceive their physical and social environments. Data-Driven Decisions for Academic Achievement (D3A2): An Ohio Department of Education initiative that provides a systematic approach for Ohio educators to access data and aligned resources. Users are able to identify and access resources to meet DRAFT OIP Facilitator’s Guide December 2008 90 of 95 specific needs from different systems that communicate using common standards, for example, Schools Interoperability Framework (SIF) to ensure consistent data standards and the Ohio Standard Identifier Code (OSIC) to show alignment to Ohio’s Academic Content Standards. Data-Driven Decisions: Districts and schools knowledgeably and effectively use a range of data at the classroom, school, and district levels to improve instructional support and practices. Data Teams. Refer to Professional Learning Community Decision Framework: A needs-assessment tool that uses essential questions that can only be answered with student achievement data, perceptual data, and other forms of data at the state and local level. The essential questions are organized around four levels and begin at Level 1 with a focus on student performance in content areas by grade level, building, and subgroup. Levels 2–4 ask essential questions related to the critical student performance problems identified in Level 1 and uncover possible causes of these problems tied to the following: curriculum, instruction, assessment, teacher quality, professional development, leadership, school climate, parents and family, community involvement, and allocation of resources. District: Individuals and organizations that play a role in the education of students who live in a specific local area. District Leadership Team (DLT): A team of individuals who promote a culture of common expectations or commitment by maintaining a district-wide focus on high achievement for all students. Elementary and Secondary Education Act (ESEA): The Elementary and Secondary Education Act. Passed in 1965 as part of President Johnson’s War on Poverty, this Act authorizes the federal government’s single largest investment in elementary and secondary education. The ESEA focuses on children from high-poverty communities and students at risk of educational failure. The Act authorizes several well-known federal education programs including Title I, Safe and Drug Free Schools, Bilingual Education (Title VII), and Impact Aid. English as a Second Language (ESL): Refers to nonnative English-speaking students or programs pertaining to the teaching of nonnative English speaking students. Ensure: Make certain or guarantee. Evidence of Success: Tangible documentation that shows progress toward achieving a strategy. Expectations and Conditions Goal: A broad statement that specifies a desired change to improve or increase the opportunities or potential for improved learning and identifies the end result to be achieved within a given timeframe. Extended Learning Time: Extend the amount of time students have available for school by providing before and after school and summer learning opportunities, modified school calendars, and changes in the structure of the school day. In addition, extended learning time can be provided by reducing/eliminating pullout programs that interrupt regular instructional time, increasing the focus on learning during scheduled class time by reducing extraneous activities, and scheduling longer blocks of time for classes. First- and Second-Order Change: First-order change is change in specific structure or practices with the focus on how those changes are operationalized. This is contrasted to OIP Process Guide 91 of 95 December 2008 second-order change, in which the emphasis is on addressing the underlying beliefs to which the change was in response. Formative Assessment. Assessment carried out during the instructional process for the purpose of improving teaching or learning. To be effective, teachers must be skillful in using various assessment strategies and tools such as observation, student conferences, portfolios, performance tasks, prior knowledge assessments, rubrics, feedback, and student self-assessment. More importantly, they must have a deep understanding of the formative assessment process and understand its close relationship to instructional scaffolding. Grade or Department Level Teams. Refer to Professional Learning Community Hypothesis: A premise, suggestion, or assumption about why a problem exists. Indicator: There are two types of indicators. A performance indicator is the gauge by which a goal is determined to be met. A progress indicator is the gauge by which a strategy is determined to be met. Progress indicators have a baseline measure established and short-term progress measures to assess degree of changes in knowledge, skills, attitudes, policies and/or practices; and documentation is identified to provide evidence that the indicator is met. Mission: The district’s purpose or reason it exists. Fulfilling the mission is how a district realizes its vision. Mobility: Students not in same building 120 days before test window. No Child Left Behind (NCLB) Act: The common name for federal Elementary and Secondary Education Act (ESEA), signed into law by President Bush in 2002, which mandates proficiency of all students by 2014. Nonnegotiable Goal: Goals upon which all staff members must act. Observation: A statement that reflects an opinion, statement, or comment about data. Pattern: Data that show a relationship within the same set of data, for example, multiple grades, discipline, behaviors. Problem Statement: A declaration of a systemic challenge or obstacle faced by an organization that is based on multiple data points. Professional Learning Community: Designed to increase teacher and/or district staff capacity in meeting the challenge to close achievement gaps and raise the bar for all students. Characterized by continuous, school-based/district-based professional development, mutual support and coaching with peers, dedicated time for collaborative work, and permission to take risks as a staff to learn, practice, and hone their skills. Effective school and district leadership is fundamental to creating professional learning communities. Other terms may be used such as data teams, grade level teams, department teams, etc. to describe a professional learning community in a district or building. Randomized Controlled Trial: Studies that randomly assign individuals to an intervention group or to a control group in order to measure the effects of the intervention. Research-Based Practices: The process of reviewing, assessing, and applying proven strategies to address data-determined needs. OIP Process Guide 92 of 95 December 2008 Root Cause: The deepest underlying cause or causes of positive or negative symptoms within any process that if dissolved would result in elimination or substantial reduction of the symptom. Stakeholder: Anyone who affects or is affected by the success of the district. Typical stakeholder groups include students, teachers, paraprofessionals, support staff, school administrators, students’ immediate family members, school board members, community leaders, local business and industry representatives, and citizens who live in the community (ies). Standards: Subject-matter benchmarks to measure students’ academic achievement. Curriculum standards drive what students learn in the classroom. Standards-Based Learning: A performance-based educational system that places significantly greater emphasis on how well students meet specific learning goals and places significantly less emphasis on state-level laws and rules dictating how instruction is provided. Strategy: Key approaches the district will implement that are written as specific, measurable statements about what is going to be accomplished to meet a need and get closer to reaching a goal within a given timeframe. Student Performance Goal: A broad statement that specifies a desired change in student performance to close a gap (what one aims to accomplish) and identifies the end result to be achieved within a given timeframe. Summative Assessment: Assessments, e.g., state assessments, district benchmark assessments, end-of-term or semester exams, given periodically to determine at a particular point in time what students know and do not know relative to content standards to help evaluate the effectiveness of programs, goals, and/or alignment of curriculum. Summative assessments are too far down the learning path to provide information at the classroom level and to make instructional adjustments and interventions during the learning process. Tasks: A listing of steps that need to be taken for someone to complete the action. Trend: A statement based on at least three years of data from the same data source. Vision: A shared understanding of what the district wants to create (picture of the future) by stakeholders who are committed to its realization. OIP Process Guide 93 of 95 December 2008 Bibliography Bolam, R., McMahon, A., Stoll, L., Thomas, S., & Wallace, M. (2005). Creating and sustaining professional learning communities. Research Report Number 637, General Teaching Council for England. London: Department for Education and Skills. Coalition for Evidence-Based Policy. (2003). Identifying and implementing educational practices supported by rigorous evidence: A user friendly guide. Washington, DC: U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance. Frank, S., & Miles, K. H. (2007). District resource allocation modeler (DREAM): A web-based tool supporting the strategic use of education resources (Working Paper 19). Seattle, WA: School Finance Redesign Project. Garmston, R. J., & Wellman, B. M. (1999). The adaptive school: A sourcebook for developing collaborative groups. Norwood, MA: Christopher-Gordon Publishers. Garet, M. S., Porter, A. C., Desimone, L., Birman, B., & Yoon, K. (2001). What makes professional development effective? American Education Research Journal, 38(4), 915–945. Retrieved October 22, 2007, from www.aztla.asu.edu/ProfDev1.pdf Kansas Education Resource Management Study. (2006). Synthesis of highly resourceeffective strategies. Kentucky Department of Education. (2006). Kentucky district improvement planning roles and responsibilities: Guidance for districts. Frankfort, KY: Author. Retrieved February 5, 2008, from http://www.education.ky.gov/NR/rdonlyres/83C23D26-9E3F-4BB7-BC12DE44D68DAA40/0/DistrictPlanningRolesResponsibilities.pdf Krueger, R. A. (1994). Focus groups: A practical guide for applied research (2nd ed.). Thousand Oaks, CA: Sage Publications. Lanford International, Inc. (1999). Langford quality learning: Tool time. Learning Point Associates. (2007). The professional learning communities resource book. Unpublished manuscript. Marzano, R. J., Waters, T., & McNulty, B. A. (2005). School leadership that works: From research to results. Alexandria, VA: Association for Supervision and Curriculum Development. Marzano, R. J., Pickering, D. J., & Pollack, J. E. (2001). Classroom instruction that works: Research based strategies for increasing student achievement. Alexandria, VA: Association for Supervision and Curriculum Development. McBeath, A. (2006). Getting districtwide results. Chicago: Cross City Campaign for Urban School Reform. North Central Regional Education Laboratory. (2001). Data retreat facilitator’s guide. Ohio Department of Education. (2000). Reference guide to continuous improvement planning for Ohio school districts (2nd ed.). Columbus, OH: Author. Ohio Department of Education. (2005). Professional development standards. Columbus, OH: Author. OIP Process Guide 94 of 95 December 2008 Ohio Leadership Advisory Council. (2007). Ohio leadership development framework. Columbus, OH: Ohio Department of Education. Parsad, B., Lewis, L., and Farris, E. (2001). Teacher preparation and professional development: 2000 (NCES Publication No. 2001-088). Washington, DC: National Center for Education Statistics. Retrieved October 22, 2007, from http://nces.ed.gov/pubs2001/2001088.pdf Pashler, H., Bain, P. M., Bottge, B. A., Graesser, A., Koedinger, K., McDaniel, M., & Metcalfe, J. (2007). Organizing instruction and study to improve student learning (NCER 2007– 2004). Washington, DC: National Center for Education Research, Institute of Education Sciences, U.S. Department of Education. Retrieved February 5, 2008, from http://ies.ed.gov/ncee/wwc/pdf/20072004.pdf Porter, A., Garet, M., Desimone, L., Yoon, K., & Birman, B. (2000). Does professional development change teaching practice? Results from a three-year study. Washington, DC: US Department of Education. Retrieved October 22, 2007, from http://www.ed.gov/rschstat/eval/teaching/epdp/report.pdf Porter, B. (1999). Grappling with accountability: Resource tools for organizing and assessing technology for student results. Sedalia, CO: Education Technology Planners. Senge, P. M. (1990). The fifth discipline: The art & practice of the learning organization. New York: Doubleday. Stanovich, P. J., & Stanovich, K. E. (2003). Using research and reason in education: How teachers can use scientifically based research to make curricular and instructional decisions. Jessup, MD: ED Pubs. Standard & Poor’s. Stewart, M., & Kingston, J. (2006). Kansas Education Resource Management study. Phase III: A synthesis of highly resource effective district strategies. New York: McGraw-Hill. Vescio, V., Ross, D., Adams, A. (2006). A review of research on the impact of professional learning communities on teacher practice and student learning. Teacher and Teacher Education. Paper presented at the NSRF Research Forum, Denver, CO. Waters, T. J., & Marzano, R. J. (2006). School district leadership that works: The effect of superintendent leadership on student achievement. Denver, CO: Mid-continent Research for Education and Learning. Weisbord, M. R., & Janoff, S. (1995). Future search: Finding common ground for action in organizations. San Francisco: Berrett-Koehler. Wiggins, G., & McTighe, J. (2007). Schooling by design: Mission, action, and achievement. Alexandria, VA: Association of Supervision and Curriculum Development. Yoon, K. S., Duncan T., Lee. S. W.-Y., Scarloss, B., & Shapley, K. (2007). Reviewing the evidence on how teacher professional development affects student achievement (Issues & Answers Report, REL 2007–No. 033). Washington, DC: U.S. Department of Education, Institute of Education Sciences, National Center for Educational Evaluation and Regional Assistance, Regional Education Laboratory Southwest. Retrieved February 5, 2008, from http://ies.ed.gov/ncee/edlabs/regions/southwest/pdf/REL_2007033.pdf OIP Process Guide 95 of 95 December 2008