Data-Driven Instruction
Entry Plan Leadership Workshop
Paul Bambrick-Santoyo
P1
NY State Public School ELA 4th Performance vs. Free-Reduced Rates
100%
90%
Pct. Proficient
80%
70%
60%
50%
40%
30%
20%
10%
10%
20%
30%
40%
50%
60%
70%
Pct. Free-Reduced Lunch
80%
90%
100%
P2
NY State Public School ELA 4th Performance vs. Free-Reduced Rates
100%
90%
Pct. Proficient
80%
70%
60%
50%
40%
30%
20%
10%
10%
20%
30%
40%
50%
60%
70%
Pct. Free-Reduced Lunch
80%
90%
100%
P3
AGENDA
• NETWORKING & ITS ROLE IN SCHOOLS
• REVIEW OF THE KEYS OF DATA-DRIVEN INSTRUCTION
• ENTRY STRATEGIES 101
• CASE STUDIES, PRINCIPAL ENTRY, ROUND 1
• CASE STUDIES, PRINCIPAL ENTRY, ROUND 2
• RESISTANCE SCENARIOS, YEAR 1
• BUILDING PERSONAL ACTION PLANS
• CONCLUSIONS
P4
Informal Networks: “Change” Drivers
• Advice Networks: “experts” (best teachers) are the hubs
• Trust Networks: Culture leaders are the hubs
BOTTOM LINE:
• You need experts AND culture leaders to make change occur
effectively
P5
THE FOUR KEYS:
DATA-DRIVEN INSTRUCTION AT ITS ESSENCE:
ASSESSMENTS
ANALYSIS
ACTION
in a Data-driven CULTURE
P6
ASSESSMENTS:
PRINCIPLES FOR EFFECTIVE ASSESSMENTS:
COMMON INTERIM:
• At least quarterly
• Common across all teachers of the same grade level
DEFINE THE STANDARDS—ALIGNED TO:
• To state test (format, content, & length)
• To instructional sequence (curriculum)
• To college-ready expectations
P7
ASSESSMENTS:
PRINICIPLES FOR EFFECTIVE ASSESSMENTS:
REASSESSES:
• Standards that appear on the first interim assessment
appear again on subsequent interim assessments
WRONG ANSWERS:
• Illuminate misunderstanding
TRANSPARENT:
• Teachers see the assessments in advance
P8
THE FOUR KEYS:
DATA-DRIVEN INSTRUCTION AT ITS ESSENCE:
ASSESSMENTS
(Interim, Aligned, Reassess, Transparent)
ANALYSIS
ACTION
in a Data-driven CULTURE
P9
ANALYSIS:
• IMMEDIATE: Ideal 48 hrs, max 1 wk turnaround
• BOTTOM LINE: Includes analysis at question level, standards
level, and overall—how well did the students do as a whole
• TEST-IN-HAND analysis: Teacher & instructional leader
together
• TEACHER-OWNED analysis
• DEEP: Moves beyond “what” to “why”
P10
THE FOUR KEYS:
ASSESSMENTS
(Aligned, Interim, Reassess, Transparent)
ANALYSIS
(Quick, Bottom line, Teacher-owned, Test-in-hand, Deep)
ACTION
in a Data-driven CULTURE
P11
ACTION:
• PLAN new lessons based on data analysis
• ACTION PLAN: Implement what you plan (dates, times,
standards & specific strategies)
• LESSON PLANS: Observe changes in lesson plans
• ACCOUNTABILITY: Observe changes classroom observations,
in-class assessments
• ENGAGED STUDENTS: Know end goal, how they did, and
what actions they’re taking to improve
P12
THE FOUR KEYS:
ASSESSMENTS
(Aligned, Interim, Reassess, Transparent)
ANALYSIS
(Quick, Bottom line, Teacher-owned, Test-in-hand, Deep)
ACTION
(Action Plan, Accountability, Engaged)
in a Data-driven CULTURE
P13
DATA-DRIVEN CULTURE:
• VISION: Established by leaders and repeated relentless
• “REAL” LEADERSHIP TEAM: Trained and highly active
• CALENDAR: Calendar in advance with built-in time for
assessments, analysis & action
• PROFESSIONAL DEVELOPMENT: Aligned
P14
THE FOUR KEYS:
ASSESSMENTS
(Aligned, Interim, Reassess, Transparent)
ANALYSIS
(Quick, Bottom line, Teacher-owned, Test-in-hand, Deep)
ACTION
(Action Plan, Accountability, Engaged)
in a Data-driven CULTURE
(Vision, Leadership, Calendar, PD)
P15
Quick-Write Reflection
• What are the first things you need to do when you get
your principalship?
P16
Case Study Revisited: Douglass Street School
1. Read case study
2. Whenever a component of the Data-Driven Instruction
Rubric appears in the case, label it according to section
of rubric. (Culture=C, Assessment=As, Analysis=An,
Action=Ac)
Example 1: Assessment calendar is #3 in Culture, so
label “C3.”
Example 2: Test-in-hand analysis is “An4.”
3. Where you see effective use of networks, label
“Networks.”
4. Small Group: Develop chronology of drivers in Brown’s
action plan.
P17
Entry Plan 101:
Strategies for Entering Principalship Successfully
P18
BIG ROCK #1
STUDENT CULTURE:
• Safe
• Kids in school
• Kids in class
• Kids on task
• Eyes on the Prize
P19
BIG ROCK #2
STUDENT ACHIEVEMENT:
• Data-driven instruction & Assessment is quickest, most
effective lever for influencing student achievement
• Mold everything else around that goal:supervision, PD,
calendar, scheduling, etc.
P20
BIG ROCK #3
YOU DECIDE
P21
Support for the Work:
Guides Presented During the Course of This
Year’s Data-Driven Workshops
P22
Phases of Data-Driven Instruction &
Interim Assessments
Adapted, Research by Camden County, GA Public School District
P23
PHASE 1
IGNORANCE, CONFUSION, OVERLOAD:
• “I don’t understand what we’re doing.”
• “This is too much! How am I really supposed to use all
this?”
• “All this analysis! What’s wrong with just grading the oldfashioned way?”
• “Uh? Interim assessments? What are those?”
P24
PHASE 2
FEELING INADEQUATE & DISTRUSTFUL:
• “How can two questions on a test possible establish
mastery? These tests can never measure what I know about
my students’ learning.”
• “This idea of an assessment is terrible! We don’t teach like
that format! We teach it this way.”
P25
PHASE 3
CHALLENGING THE TEST:
• “Question #26 is a poor question. Answer “b” is a trick
answer.”
• “Question #11 is too hard. We need to make it easier.”
• “The kids made silly mistakes because of the pressure of
this pointless test. They know this stuff.”
• Undertone: “I’ve never looked at a test item before, but
I’m going to now if you’re going to hold me accountable.”
P26
PHASE 4
ANALYTICAL but SUPERFICIAL:
• “They just don’t do well on word problems. I just need to
do more word problems.”
• “They just don’t read enough. I’ll get them to read more.”
P27
PHASE 5
LOOKING FOR CAUSES, BUT NO ACTION:
• “These wrong answers tell me that they don’t know the
difference between a summary and a theme.”
• “I always taught grammar in isolation, and this test asked
for it in a more authentic form.”
• The problem with solving algebraic equations for them was
actually the inability to subtract negative integers.”
P28
PHASE 6
CHANGING TEACHING PRACTICES:
• Teachers follow through on analysis
• Lesson plans reflect spiraling, re-teaching, etc.
• Teachers look for best practices outside of their own
classroom
P29
Running Effective Analysis Meetings
Protocols from NLNS Training
P30
ASSESSMENT ANALYSIS I
PART 1—GLOBAL IMPRESSIONS:
Global conclusions you can draw from the data:
• How well did the class do as a whole?
• What are the strengths and weaknesses in the standards:
where do we need to work the most?
• How did the class do on old vs. new standards? Are they
forgetting or improving on old material?
• How were the results in the different question types
(multiple choice vs. open-ended, reading vs. writing)?
• Who are the strong/weak students?
P31
ASSESSMENT ANALYSIS II
PART 2—DIG IN:
• “Squint:” bombed questions—did students all choose same
wrong answer? Why or why not?
• Compare similar standards: Do results in one influence the
other?
• Break down each standard: Did they do similarly on every
question or were some questions harder? Why?
• Sort data by students’ scores: are there questions that
separate proficient / non-proficient students?
• Look horizontally by student: Are there any anomalies
occurring with certain students?
P32
PRECURSORS TO EFFECTIVE ANALYSIS MTGS
• Did teachers see the assessment in advance?
(TRANSPARENCY)
• Did they mark it up: Confident, Not Sure, No Way?
(TEST-IN-HAND, TEACHER-OWNED)
• Did you train teachers in analysis strategies?
(PROF DEVT, DEEP)
• Did they fill out an analysis sheet? Did they answer the
fundamental question: WHY the students did not learn it?
(TEACHER-OWNED, DEEP)
• Did they have to fill out an action plan? Did you model
how to fill out an action plan using these analysis
questions?
(ACTION PLAN, ACCOUNTABILITY)
P33
PRECURSORS TO EFFECTIVE ANALYSIS MTGS
• Did you model a poor and a good conversation so they hear
your expectations?
(PROF DEVT, DEEP)
• Did you analyze their results (above and beyond them
analyzing their own) in preparation for the meeting?
(LEADERSHIP)
• Did you collect their analysis ahead of time and see if it
looked acceptable?
(LEADERSHIP, ACCOUNTABILITY)
• Did you have a plan ready to access content experts if the
problems were beyond your expertise?
(PROF DEVT)
P34
TIPS FOR EFFECTIVE ANALYSIS MEETINGS
• Let the data do the talking
• Let the teacher do the talking (or get them to!)
• Always go back to the test and back to specific questions
• Don’t fight the battles on ideological lines (you’re going to lose)
• There’s a difference between the first assessment and the third
• You’ve got to know the data yourself to have an effective
meeting
• Make sure it’s connected to a concrete plan that you can verify
P35
ANALYSIS MEETING HELPFUL PHRASES:
HELPFUL STARTERS FOR ANALYSIS MEETINGS:
• “So…what’s the data telling you?”
• “Congratulations on the improvement from last time in x
area! You must be really proud of their growth here.”
• “So the _____ [paraphrase their frustration: the test was
hard, the students were difficult, etc.]? I’m sorry to hear
that. So where should we begin with our action plan moving
forward?”
P36
ANALYSIS MEETING HELPFUL PHRASES:
DATA-FOCUSING FOR ANALYSIS MEETINGS:
• “So let’s look at question 18…Why do you think they got it
wrong?”
• “You know, I thought it might be a silly mistake, but what
surprised me is that they did really well on questions x & y.
Why do you think they did so well on these questions and
yet not on your original question?”
• “Let’s look at question 11. What did the students need to be
able to do to answer that question effectively? Is this more
than they are able to do with you in your class?”
• [When new ideas occur or deeper analysis is done at the
meeting than what teacher did previously] “So let’s re-visit
the action plan you created and see how we can incorporate
these additional ideas.”
P37
Running Effective Teacher Meetings:
Results Meeting Protocol
P38
ACTION: RESULTS MEETING
50 MIN TOTAL
• IDENTIFY ROLES: Timer, facilitator, recorder (2 min)
• IDENTIFY OBJECTIVE to focus on (2 min or given)
• WHAT WORKED SO FAR (5 min)
• [Or: What teaching strategies did you try so far]
• CHIEF CHALLENGES (5-10 min)
• BRAINSTORM proposed solutions (10 min)
• [See protocol on next page]
• REFLECTION: Feasibility of each idea (5 min)
• CONSENSUS around best actions (20 min)
• [See protocol on next page]
• PUT IN CALENDAR: When will the tasks happen? When will
the teaching happen? (10 min)
P39
RESULTS MEETING STRUCTURE:
PROTOCOLS FOR BRAINSTORMING/CONSENSUS
PROTOCOL FOR BRAINSTORMING:
• Go in order around the circle: Each person has 30 seconds to
share a proposal.
• If you don’t have an idea, say “Pass.”
• No judgments should be made; if you like the idea, when it’s
your turn simply say, “I would like to add to that idea by…”
• Even if 4-5 people pass in a row, keep going for the full
brainstorming time.
PROTOCOL FOR REFLECTION:
• 1 minute—personal/individual reflection on the list: what is
doable and what isn’t for each person.
• Go in order around the circle once: Depending on size of group
each person has 30-60 seconds to share their reflections.
• If a person doesn’t have a thought to share, say “Pass” and
come back to that person later.
• No judgments should be made.
P40
RESULTS MEETING STRUCTURE:
PROTOCOLS FOR BRAINSTORMING/CONSENSUS
PROTOCOL FOR CONSENSUS/ACTION PLAN:
• ID key actions from brainstorming that everyone will agree to
implement
• Make actions as specific as possible within the limited time
• ID key student/teacher guides or tasks needed to be done to
be ready to teach—ID who will do each task
• Spend remaining time developing concrete elements of lesson
plan:
• Do Now’s
• Teacher guides (e.g., what questions to ask the students or
how to structure the activity)
• Student guides
• HW, etc.
NOTE: At least one person (if not two) should be recording
everything electronically to send to the whole group
P41
KEY TIPS TO MAKING RESULTS MEETING
PRODUCTIVE:
• GET SPECIFIC to the assessment question itself: we can
teach 10 lessons on this standard. What’s the set of lessons
these students need based on the data?
• AVOID PHILOSOPHICAL DEBATES about theories of
Math/Literacy: Focus on the small, specific challenge of the
moment. That’s where the change will begin!
• IF GROUP IS TOO LARGE: After presenter is done, split into
two groups. You’ll generate more ideas and you can share your
conclusions/action plans at the end.
P42
Planning Your Time:
Best Use of 1 hr/wk with Teachers
P43
IF YOU HAVE 1 HR/WK OF FACE TIME:
PRE-WORK
PRE-WORK:
• INTERIM ASSESSMENT & LESSON PLAN REVIEW: Checking
teachers’ lesson plans alongside key re-teach standards
(action plan) or standards on next assessment identifying:
• What are the key moments to observe to check for rigor
or alignment?
• Are the standards well embedded in an engaging lesson?
• MINI-OBSERVATIONS: Mini-observations of class each day
(5-15 minutes per visit) with brief feedback in hallways
P44
IF YOU HAVE 1 HR/WK OF FACE TIME:
INDIVIDUAL MEETINGS
INDIVIDUAL FACE TIME—1st PRIORITY:
• DATA-DRIVEN PROCESS IS FIRST DRIVER OF AGENDA:
• Help teacher analyze interim assessment data & make
action plan (2-6 days after asst; important first time
through)
• Results analysis meeting (once teacher has done
analysis/action plan before meeting with you)
• Review upcoming assessment (6 weeks before)
• Anticipate student performance by marking interim
assessment questions “confident,” “not sure,” “no way”
(1-2 weeks before asst)
DURING REMAINING TIME/WEEKS:
• FEEDBACK ON CLASSES OBSERVED: Dialogue—affirmation
of strengths, suggestions for improvement
• 1-2 goals maximum
• Make concrete follow-up plan if needed
• PLANNING FOR THE FUTURE: Review action plan/lesson
plans, anticipate/plan key moments for upcoming week
P45
IF YOU HAVE 1 HR/WK OF FACE TIME:
TEAM MEETINGS
TEAM MEETINGS:
• RESULTS MEETING PROTOCOL:
• Target just 1 standard to visit for the meeting
• If important, redo results meeting around same standard
the following week to evaluate the effectiveness of their
implementation
• REVIEW STUDENT WRITING:
• Make action plan (2-6 days after asst)
• Review upcoming assessment (6 weeks before)
• Anticipate student performance by marking interim
assessment questions “confident,” “not sure,” “no way”
(1-2 weeks before asst)
• MAKE ACTION PLANS: Mini-observations of class each day
(5-15 minutes per visit) with brief feedback in hallways
P46
Implementing Action
Effective Action Strategies
P47
DATA-DRIVEN CLASSROOM PRACTICES—
TO DO & NOT TO DO:
STRATEGY
WHAT TO DO:
NOT TO DO:
Objectives:
Break down the
standard
Cut & paste standard
from state list
Do Now’s:
Reflect student
learning
Shoot lower than the
assessment rigor
HW:
Meet rigor of asst,
independent work,
easy to grade
Purely computation
worksheets
Walls:
Evidence of standard
No evidence
Questioning Cold calling,
dipsticking
Call only on hands
raised
Student
Responses
Teacher does all the
scaffolding all the
time
Get students to do
the work
P48
DATA-DRIVEN CLASSROOM PRACTICES—
TO DO & NOT TO DO:
STRATEGY
WHAT TO DO:
NOT TO DO:
Analysis:
Check performance in
related standards and
at different reading
levels to see what
objective to target
Ignore related
performance and do
more work on the
standard
Questioning Scripted questions,
Rigor:
moving from heavily
scaffolded to
independent
Ask many similar
questions about the
standard
Student
Student gets
Reading
immediate feedback
Engagemen about results
t
Independent work is
not monitored and
students could or
could not be meeting
the standard
P49
1. TEACHER ACTIONS
• LESSON PLANS: Teachers staple action plan to top of lesson
plans, explicitly showing where they’re re-teaching key stds
• CHECKING FOR UNDERSTANDING: Oral review,
dipsticking, dry erase boards / plastic sheets, every Saphier
strategy imaginable
• DO NOW / QUICK-CHECK: Spiral all standards; collect them
for 4 days and let students correct at end of week
• HOMEWORK: Assess skills that students should be able to do
independently; design HW aligned to the standards
• IN-CLASS ASSESSMENTS: Align in content and format;
spiral a portion of each test
• DIFFERENTIATED INSTRUCTION: Generate strategies to
work with select student groups while other students are
working independently
P50
2. SCHOOLWIDE SUPPORT
• CLASS SCHEDULE: Take 10-15 minutes from math block
every day to re-teach; find places in the schedule where you
could gain learning time for students; utilize lunch/breakfast
time
• SCHOOL SCHEDULE: Build in pre-school/after-school
tutoring, use lunch/breakfast time to target students in need
of improvement
• PULLOUT GROUPS: Other staff members (including you!)
pull out small groups of students to work on toughest
standards (highest or lowest achieving subgroups) so main
teacher can focus on smaller class
P51
2. SCHOOLWIDE SUPPORT (con’t.)
• ADDITIONAL STAFF: Solicit support of non-core academic
staff to teach small groups of students (paraprofessionals,
art/PE teachers, SPED, ELL, tutors, parents, etc.)
• EVERY LEADER ON BOARD: All leaders (deans, VP’s, Dept.
Chairs, coaches) support the process—target teachers who
need most help, follow results meeting protocol, etc.
• PULL: Rotate students into groups with similar skill
deficiencies and teach them in those groups for select time
periods; rotate groups when in new subjects
P52
Five Case Studies
Building a Principal Entry Plan
P53
CASE STUDY PROTOCOL, ROUND 1:
• SELECT CASE STUDY: Pick case that feels closest to
your school’s particular context
• READ CASE STUDY: Take notes and use DDI
implementation rubric to look for presence/absence of
key drivers
• IMPLEMENT RESULTS PROTOCOL: Create a principal
entry plan based on analysis of the case
P54
CASE STUDY PROTOCOL, ROUND 2:
• SELECT SECOND CASE STUDY: Pick case that feels
next closest to your school’s particular context
• READ CASE STUDY: Take notes and use DDI
implementation rubric to look for presence/absence of
key drivers (5 min)
• IDENTIFY CORE CHALLENGES OF THE CASE:
Collectively identify areas of the DDI implementation
rubric that need to be the focus (5 min)
• REVIEW ENTRY PLAN FROM PREVIOUS SMALL
GROUP: Read principal entry plan (2 min)
• BRAINSTORM IMPROVEMENTS/CHANGES TO
ENTRY PLAN: If entry plan is solid, add more detail to
implementation plan (10 min)
P55
Final Exam: Preparing for Challenges
Case Studies in Resistance & Planning
P56
SCENARIOS I
• You don’t have a good feel for how teachers have reacted
to your presentation of the data-driven initiative, but you
just got a nasty anonymous email telling you that
teachers are going to revolt against your plans.
• You’re swamped with discipline issues and you only have
1 hour to do instructional things. What do you do first?
• Teachers complain the test had nothing to do with what
they taught.
• Your leadership team is not convinced by your plan.
P57
SCENARIOS II
• Teachers protest that they don’t have time to analyze data.
(Similar scenario: teachers protest that they don’t have
enough time to grade the assessments or enter data into
templates.)
• Teachers complain that they can’t re-teach because they
have to keep covering the curriculum.
• One teacher challenges the plan (& your authority) in front
of the whole staff at your first staff meeting. What do you
do in that moment?
• A teacher approaches you to tell you that the instructional
leader you have working with her (an AP) doesn’t manage
well her team’s analysis meetings, and they quickly get off
point.
P58
SCENARIOS III
• The teachers say that they have to implement a new math
curriculum this year, and they can't deal with learning a new
curriculum, and a whole new assessment approach.
• Teachers say that the idea of double testing the kids is
ridiculous.
• You realize a few weeks before the assessments that there is
no one else who knows how to run the results and analysis
meetings besides you.
P59
SCENARIOS IV
• Teachers say the state test doesn’t include writing and that
writing is the number one skill students need for college.
• A few teachers say the whole thing is just test prep and
they want to teach more substance. They feel they know
better what students need.
• Some veteran teachers are deeply suspicious that the
interim assessments are just to evaluate their performance
as teachers.
P60
FINAL TAKEAWAYS:
• What do I want to make sure I remember to do?
• What are the areas I want to develop to be ready for next
year? (Who will I turn to for support?)
P61
October Sky:
• What made the difference? Why were the Coalwood boys
ultimately successful in launching the rocket?
P62
Conclusions
Data-Driven Instruction & Assessment
Paul Bambrick-Santoyo
P63
Tale of North Star:
100 High School Freshmen
6P64
North Star 3 Years Later:
Freshmen who made it to senior year
6P65
North Star 4 Years Later:
Plan to attend 4-year colleges
6P66
North Star 5 Years Later:
Actually enroll in 4-year colleges
Mount Holyoke
Boston College
Oberlin
Syracuse
Amherst
Spelman
Rutgers
University Of Chicago
Seton Hall
6P67