Continual Improvement Process
Oregon Department of Education
April, 2012
AGENDA
GOAL: This informational session will review requirements for the
evaluation of the 2011-12 improvement plans and some principles of
Effective Planning for Continual District/School Program Improvement.
1.
2.
3.
4.
5.
Review Rubric for evaluation of implementation of Title III
Improvement Plans.
Lessons Learned
Section A– Planning,
Section B Implementing,
Section C Monitoring.
2
TIMELINE OF ACCOUNTABILITY FOR LEAS
SY 2010-11
SY 2009-10
•Parent Notification Parent Notification
•After four years: After Two years:
Modify Curriculum/ Improvement
Plan; TA
Instruction or
•Funding?
•Replacement of
Personnel
4 yrs.
3 yrs.
SY 2007-08
SY 2008-09
Parent
Notification
1 yr
•Parent
Notification
•After Two
years:
Improvement
Plan; TA
2 yrs.
3
Example of a timeline for
Implementing Plans/Provisions
AMAO
notifications
sent to
districts /
subgrantees
(late summer /
early fall)
AMAO letters
sent to parents
(within 30
days)
Development
of 2 year plans
/ 4 year
provisions
(mid fall)
Implementation of
2 year plans / 4
year provisions
(throughout the
year the subgrantee
did not meet
AMAOs)
4
PLANNING
PROFESSIONAL
DEVELOPMENT
GOALS
Sustainable
District/School
Improvement
Cycle
INQUIRY
MASTER
PLAN
FREQUENCY
EVALUATION
INSTRUTIONAL
STRATEGIES
MASTER
PLAN
5
EVALUATION OF IP PLANS 2011-12
PROCESS:
1. Follow Guidelines in Evaluation Rubric
2. Evaluation will be the first item on the
submission of 2012-13 IP Plans
3. Missing items from the evaluation will result in
returning the 2012-13 IP Plans for review.
4. Why? The 2012-13 plans are a follow-up of the
2011-12 IPs
5. All plans should be sent to Leslie Casebeer by
June 29, 2012.
6
EVALUATION RUBRIC – 3 GUIDELINES
1) Procedures (A series of actions or steps taken to
achieve an end) are provided to monitor to assess the
implementation for all activities set forth in the master
plan;
2) The formative (short-term) evaluation procedures
should seek to determine if the strategy had the
expected effect on student achievement;
3) The summative (long-term) evaluation procedures
should seek to determine if the goals and objectives
(activities) have been attained.
Adapted from the Louisiana Department of Education7
EVALUATION RUBRIC
Guideline I. The evaluation procedures to monitor and
assess the indicators of implementation for all activities
should include at least three of the four of the
following criteria:
1. What data instrument(s) was used to collect
information and what kind of feedback was obtained
as a result of the data analysis (inquiry)?
2. What was measured or assessed, and how was this
?
3. Who conducted the evaluation? Please include the
position title, i.e. Curriculum Director, Principal, etc.
4. How often were the goals/activities monitored
(frequency)?
8
EVALUATION RUBRIC
Guideline II. The formative (short-term) evaluation
procedures should seek to determine if the strategy had
the expected effect on student achievement.
• Did the evaluation procedures provide sufficient
evidence to evaluate the short-term effects for each
strategy on student achievement?
• Short-term effects may have included student portfolios,
comparison of student’s work throughout the year,
teacher-made tests, or other similar type evidence.
9
EVALUATION RUBRIC
Guideline III. The summative (long-term) evaluation
procedures should seek to determine if the goals and
objectives have been attained.
 Did the summative evaluation adequately convey if
the school/district is improving?
 The summative evaluation should include the
applicable testing instruments with descriptions of
how they were used to determine if the goals and
objectives were attained.
 This evaluation should include a comparison and/or
analysis of test data but may also include other types
of assessment and/or qualitative data.
10
QUESTIONS
11
SCHOOL/DISTRICT IMPROVEMENT: THE
LEADERSHIP AND LEARNING CENTER
Comprehensive Needs Assessment: Planning,
Inquiry, SMART Goals
 Implementation: Research-based strategies,
design of master plan, professional learning,
parental engagement
 Monitoring: Master Plan, Frequency, Measuring
Progress, Evaluation
12
COMPREHENSIVE NEEDS
ASSESSMENT PLANNING
Assessment Results
Gather and reflect on school/district data: external and internal
.
13
DATA PORTFOLIO
The following items could make up the Data Portfolio (to be kept on file at the school):
o ELPA data for the last five years.
o AYP report for ELs for the last three-five years.
o Summary of Findings of Survey Data and other source documents. (Teachers, Parents,
Students, Principal) If Parent sample size is inadequate, you might consider Parent Focus
Group(s).
o Summary of Findings of Interview Data and all source documents. (Principal, Counselor,
and Teachers)
o Summary of Findings of Focus Group Data and all source documents. (Teachers, Students,
and Parents)
o Copy of Data Triangulation
o Comprehensive Needs Assessment: Final Report
o Summary of internal assessment data
o Data Analysis (Trend Data history etc.)
o Cognitive Summary Data (PSAT, SAT etc.)
o Citation from monitoring of Federal Programs
Adapted from Improvement Planning Louisiana Department of Education,
14
DATA COMPREHENSIVE NEEDS ASSESSMENT: SUMMARY REPORT
: Rank-order the identified areas of strength (3-5) from the EL student
performance (cognitive data), data, and/or graduation index and indicate
the supporting data sources:
STRENGTHS
DATA SOURCE/INSTRUMENT
1.
2.
3.
4.
5.
Adapted from Improvement Planning, Louisiana Department of Education
15
Part IIa. List the contributing factors from the cognitive,
attitudinal/perceptual, data of the previously identified strengths:
CONTRIBUTING FACTORS TO THE
STRENGTHS
1.
2.
3.
4.
5.
DATA SOURCE
16
DATA COMPREHENSIVE NEEDS ASSESSMENT: SUMMARY REPORT
Rank-order the identified areas of weakness (3-5) from the student
performance (cognitive data), dropout data, and/or graduation index and
indicate the supporting data sources:
CHALLENGES
DATA SOURCE/INSTRUMENT
1.
2.
3.
4.
5.
17
DATA Part IIb. List the contributing factors from the cognitive,
attitudinal/perceptual, behavioral, and archival data of the previously
identified challenges:
CONTRIBUTING FACTORS TO THE
CHALLENGES
DATA SOURCE/INSTRUMENT
1.
2.
3.
4.
5.
The identified challenges will lead to the goals. The contributing factors of
the challenges will lead to the strategies.
18
Comprehensive Needs Assessment
Planning
Teacher Practices
Focuses conversation on research-based best
practices
Educators talk about their practices, share
their knowledge and skills and support one
another in the specific contexts in which they
work.
19
Teaching Practices
Current Curricular Practices
________
________
________
(identify
(identify
(identify
practice)
practice)
practice)
Evidence of Practice (State in definitive/tangible terms)
Is the current practice research-based?
Is it a principle & practice of effective ELD programs?
Has the current practice been effective or ineffective? For how long?
What data source(s) do you have that support your answer? (identify all
applicable sources)
Evidence of effectiveness or ineffectiveness (State in terms of
quantifiable improvement)
Evidence of equitable school support for this practice. Do all teachers
have they need to faithfully implement the practice?
Next Step (changes or continuations)
Adapted from School and District Improvement Planning Template, Tennessee Department of Education
SCHOOL IMPROVEMENT PLANNING FOR
THE SUCCESS OF ENGLISH LANGUAGE
LEARNERS
Leadership Performance: Implementing
Innovations:
Develops, articulates, and communicates a shared
vision of the intended change.
Investing in professional learning.
Monitoring— Checking on progress.
Providing continual assistance.
Creating a context supportive of change.
21
STOP
22
QUESTIONS?
23
STRENGTHS: THE LEADERSHIP TEAM
Reflect and celebrate program strengths;
Determine what is currently working well and why;
Assist in building program capacity in identifying
strengths, areas which require attention to sustain
strengths;
Foster introspection, reflection and analysis;
Lead to better planning with precision and
intentionality;
Act as a catalyst for collaborative and collegial
conversations about sustainability from within.
24
AREAS OF IMPROVEMENT:
THE LEADERSHIP TEAM
Identify areas in need of improvement;
Identify a “vital few” key areas upon which to focus
improvement efforts ;
Acts as a catalyst for collaborative and collegial
conversations about improvement from within;
Provides a forum for consensus building around
program improvement;
Develops a deeper understanding of the unique
improvement needs of schools and facilitates the
communication of the improvement goals;
Identify possible cause and effect scenarios.
25
SMART Goals
Specific and Strategic – Have you articulated precisely
what you want to achieve and have priorities been
strategically selected based on a comprehensive needs
assessment?
Measurable- Are you able to assess/ measure your
progress?
Achievable- Is the goal within your reach and within
your control? Are targets ambitious yet attainable?
Results-Based – Have established base-line data and
targets of where you want to end up?
Time-bound- What is the dead-line for completing
your goal?
26
Specific and Strategic
Ask questions such as:
 In what area are a significant number of students
experiencing difficulty?
 What specific aspect of this area of concern would
make the biggest impact for students if improvement
occurred?
 If students could change and do “X” in this area,
would they benefit greatly – would it positively
impact other areas of their learning?
 If our goal was achieved and the specific area
identified was significantly improved, what would
students be doing? In other words, what would be
the indicators?
27
Measurable
Ask questions such as.....
 What tool(s) will best measure if targets have been
achieved?
 Can the tool(s) be used to establish a baseline?
 What is the achievement target for your students?
28
Attainable
Ask questions such as.....
 Is what we are expecting reasonable?
 Do we have the capacity to make the desired
change?
If “yes”, how do we most effectively use our
capacity to make the changes?
If “no”, how do we prepare ourselves so we have
the capacity to make the changes?
29
Setting Attainable Targets
According to The Leadership and Learning Centre:
If student performance is in the:
1st quartile the percentage increase should be at least 20%
2nd quartile at least 12%
3rd quartile – around 7%
Top quartile – around 4%
(Dr. Douglas Reeves, 2007)
30
Results -Oriented
Ask these questions...
 Why is it important for staff to achieve this goal?
 Have ambitious yet attainable targets been set?
31
Time Bound
Ask questions such as....
 What is the timeframe for achieving this goal?
 What strategies are in place to keep us on track
(monitoring strategies and time)
32
Major Components of the Professional Development Plan:
SMART Goal:
Identify the focused professional development
Planning - The
Professional
Development
System must be
based upon”:
Dates of
delivery
Dates for systematic
consultation with internal
trainers, regional or state
personnel designated to
provide on-going technical
assistance
Dates for
monitoring of
local professional
development
programs.
Analyses of
student
achievement data
Student needs
Personnel needs
(describe your
processes for
collecting data on
personnel needs)
Adapted from North Carolina Improvement Planning
33
Major Components of the Professional Development Plan:
SMART Goal:
Identify the focused professional development
Delivery - “In-service activities
for instructional personnel shall
focus on:
 analysis of student achievement
data,
 on-going formal and informal
assessments of student
achievement, identification and
use of instructional strategies
that emphasize rigor, relevance.
 enhancement of subject content
expertise (ELD and other core
content areas),
 integrated use of classroom
technology that enhances
teaching and learning.
34
PARENT PARTICIPATION
35
1. Convene an annual meeting at a time convenient for parents of Els:
.
 All parents shall be invited and encouraged to attend
 The school will provide information and explain the requirements of
Title III and the rights of parents
Activity/Task
Staff
.
Timeline Steps Accountability/Evaluations
(What will
Responsible (When
be done )
(Who will
it will be it
be doing it) done)
(How (How success will be
measured)
will
be
done)
Adapted from Effective Planning for Continuous Improvement, Ontario, Canada
36
.
3. Involve parents in an organized, ongoing, and timely manner, in the
planning, review, and improvement of the Title III program.
Activity/Task Staff
Timeline Steps Accountability/Evaluations
be done )
it will be it
(What
will
.
Responsible (When
(Who will
be doing it) done)
(How (How success will be
will
measured)
be
done)
37
4. Provide parents of participating children:
 Timely information about the Title III programs
 Description and explanation of the curriculum at the school, the forms of
academic assessment used to measure student progress, and the proficiency
levels students are expected to meet
 Opportunities for regular meetings to formulate suggestions and to
participate, as appropriate, in decisions relating to the education of their
children
Activity/Task Staff
(What will
Timeline Steps Accountability/Evaluations
Responsible (When
be done )
(Who will
it will be it
be doing it) done)
(How (How success will be
measured)
will
be
done)
38
Monitoring and Adjusting Evaluation
(Rubric Indicator Sec C)
Evidence of Monitoring Dates – Narrative response required
What are the calendar dates (Nov/Dec, May/June) when the
School/District Leadership Team will meet to sustain the
Improvement Planning Process? Identify the person(s)
responsible for monitoring and the role they will play in the
monitoring process.
Evidence of a Process for Monitoring Plan – Narrative
response required
What will be the process that the School Leadership Team
will use to review the analysis of the data from the
assessments and determine if adjustments need to be made in
our plan?
Adapted from School Improvement Planning, Tennessee Department of Education
39
Monitoring and Adjusting Evaluation
Evidence of a Process for Adjusting Plan – Narrative
response required
What will be the process that the School Leadership
Team will use for adjusting our plan (person(s)
responsible, timeline, actions steps, resources,
evaluation strategies) when needed?
Evidence of a Plan for Communicating to All
Stakeholders – Narrative response required
How will the School Leadership Team communicate
success/adjustments of the plan to stakeholders and
solicit ongoing input from stakeholders?
40
41
42
Title III Contacts
Kim A. Miller
Education Specialist
Education Improvement and Innovation
kim.a.miller@state.or.us
(503) 947-5712
Leslie Casebeer
Leslie.Casebeer@state.or.us
Phone: (503) 947-5648
Fax: (503) 378-5156
43
Title III Contacts
Carmen West
Program Specialist for
Title III, ESL, Bilingual Programs
Education Improvement and Innovation
carmen.west@state.or.us
503-947-5669
44
44