National Leadership Forum
Chicago, Illinois
October 2010
presented by Anne Todd, UO
Developed by
Steve Newton, Anne Todd, Rob Horner, UO
Bob Algozzine, & Kate Algozzine, University of NC at Charlotte

Provide an overview of the TIPS system
◦ Research results from two studies

Preview effective meeting practices through
use of the TIPS Meeting Foundations
◦ Preview Foundations Checklist
◦ Roles & Responsibilities
◦ Preview Meeting Minute Form

Be able to identify a problem that includes (not
include) precision elements critical for problem
solving


Every school has teams
Teams are being expected to do problem solving
 Select curricula
 Get training and implement new ideas/programs
 Provide efficient leadership
 “Communities of Practice”



Teams need to report data to staff, families,
administration, district, state
Teams NEED data to do good problem solving.
Most teams are not skilled at running problem
solving meetings and using data for decisionmaking.

Includes:
◦ Tools to define a system for effective meetings
 Roles, responsibilities, materials, accountability and
procedures
◦ Steps of effective problem solving
 A strategy for assessing, monitoring and evaluating
the implementation and results of solutions

Can be used with other data sets

TIPS Training

Team Meeting

Research tool to measure effectiveness of TIPS Training
 One full day team training
 Two coached meetings
 Use of electronic meeting minute system
 Formal roles (facilitator, recorder, data analyst)
 Specific expectations (before meeting, during meeting, after
meeting)
 Access and use of data
 Projected meeting minutes
 DORA (decision, observation, recording and analysis)
 Measures “Meeting Foundations” & “Thoroughness of
Problem Solving”
Building
Capacity
and
Sustainability
Hold effective meetings that use data to
problem solve
and plan
AND that result in
For Social
Competence,
positiveAchievement,
student outcomes
Academic
and Safety
OUTCOMES
Team-based,
SYSTEMS
documentation,
regular
communication
cycles
SWIS
DIBELS
INFORMATION
Aims Web
Meeting
Foundations
PRACTICES
Meeting Minute
Format
Problem solving
routine
Supporting
Staff & Student Behavior and Decision Making

2008-09 Single Subject Study
◦ 4 teams in Oregon
◦ Multiple baseline design
◦ SW PBIS meetings & progress monitoring literacy
meetings

2009-2010 Randomized Control Trial Study
with 34 teams
◦ 22 teams in NC
◦ 12 teams in Oregon

Need to conduct Team Training
◦ Team includes all members and a coach
◦ Define Roles and Responsibilities is critical
 Plan for absences (have back up people)


Coaching is critical
Training critical skills to facilitator, minute
taker, and data analyst
◦ Keep people on track,
◦ document relevant information for progress
monitoring and evaluation
◦ Launch the meeting with a data summary

Documenting decisions, actions, timelines,
evaluation plan is critical for sustainability

Role
◦ To create data summaries that will facilitate the
team in
 determining if there are problems
 jump starting a problem solving discussion, and
 evaluating the impact of solutions and fidelity of
implementation

General Responsibilities
◦ Prepares a brief written summary for distribution at
meetings using each of the data sources needed for
problem solving and decision making
◦ Help to generate reports during the meeting as questions
of the data arise

How?
◦ Establish the role of a data analyst (and backup
person)
◦ Teach data analyst to develop data summary
 Oakes, DIBELS, SWIS…. Etc
◦ Start meeting with defining the problem with
precision
◦ Refine precision of problem statement through
inferences and hypothesis
 Have data accessible for custom report generation
during the meeting
Structure of meetings lays foundation for
efficiency & effectiveness
10/11/2012
13

Define purpose of the team
◦ Decisions to be made, cycle of decision making,
and data source(s) to use


Define roles & responsibilities
Define team agreements about meeting
processes
1. Inform facilitator of absence/tardy before meeting
2. Be prepared for meeting by completing previously
assigned tasks
3. Avoid side talk: Remind each other to stay focused
4. Start and end on time
5. Be an active participant
6. Use electronic meeting minutes
10/11/2012
14

Core roles
◦
◦
◦
◦
◦

Facilitator
Minute taker
Typically NOT the administrator
Data analyst
Active team member
Administrator
Backup for each role
Can one person serve multiple
roles?
Are there other roles needed?
10/11/2012
15
10/11/2012
16
10/11/2012
17

Documentation of





Logistics of meeting (date, time, location, roles)
Agenda items for today’s meeting ( and next meeting)
Discussion items, decisions made, tasks and timelines assigned
Problem statements, solutions/decisions/tasks, people assigned to
implement with timelines assigned, and an evaluation plan to determine
the effect on student behavior
Reviewing Meeting minutes
 An effective strategy for getting a snapshot of what happened at the
previous meeting and what needs to be reviewed during the upcoming
meeting
 What was the issue/problem?, What were we going to do?, Who was going to
do it and by When?, and How are we measuring progress toward the goal?

Visual tracking of focus topics during and after meetings
 Prevents side conversations
 Prevents repetition
 Encourages completion of tasks
10/11/2012
18
Problem
A key to collective problem solving
is to provide a visual context that
Use
allows everyone
to follow and
Data
contribute
Out
of
Time
Solution
10/11/2012
19
Langley Elementary PBIS Team Meeting Minutes and Problem-Solving Action Plan Form
Today’s Meeting:
Date, time, location:
Facilitator:
Minute Taker:
Next Meeting:
Date, time, location:
Facilitator:
Minute Taker:
Data Analyst:
Where in
Datathe
Analyst:Form
would you place:
Team Members (bold are present today)
Today’s Agenda Items
01.
02.
03.
1.Planning for next
PTA meeting?
Next Meeting Agenda Items
1.
2.
Administrative/General Information and Issues
Information for Team, or Issue for Team to
Address
Discussion/Decision/Task (if applicable)
3.Schedule for hallway
monitoring for next
Implementation
monthand Evaluation
Problem-Solving Action Plan
Precise Problem Statement, based on review of
data
(What, When, Where, Who, Why)
Solution Actions (e.g., Prevent, Teach,
Prompt, Reward, Correction, Extinction,
Safety)
Evaluation of Team Meeting (Mark your ratings with an “X”)
2.TooWho?
many students
By When?
in the “intensive
support” for literacy
Who?
By When?
Goal, Timeline,
Decision Rule, & Updates
4.There have been
five fights on
playground in last
month.
Our Rating
Yes
So-So
No
5.Next meeting
report
on lunch-room status.
1. Was today’s meeting a good use of our time?
2. In general, did we do a good job of tracking whether we’re completing the tasks we agreed on at previous meetings?
3. In general, have we done a good job of actually completing the tasks we agreed on at previous meetings?
4. In general, are the completed tasks having the desired effects on student behavior?
10/11/2012
20
Any tasks assigned
get copied to the
meeting minutes
of the next
meeting as a
follow up item
Meeting Agenda
Item: Meeting
Foundations
Tasks: What, by
whom, by when
10/11/2012
22
Meeting Foundations
10/11/2012
23
Team Initiated
Problem Solving
(TIPS) Model
Identify
Problems
Develop
Hypothesis
Evaluate and
Revise
Action Plan
Collect
and Use
Data
Develop and
Implement
Action Plan
Discuss and
Select
Solutions
Problem Solving
Meeting Foundations
24

Defined
◦ SWISTM is a web-based information system for
gathering, entering, summarizing, reporting and
using office discipline referral information

Purpose
◦ A progress monitoring tool for improving the
ability of school personnel to develop safe and
effective learning environments

Universal Screening Tool
◦ Proportion of students with
 0-1 Office Discipline Referrals (ODRs)
 2-5 ODRs
 6+ ODRs


Progress Monitoring Tool
Compare data across time
◦ Prevent previous problem patterns

Define Problems with precision that lead to
solvable problems
N u m b e r o f R e f e r ra ls p e r S t u d e n t
20
10
0
Students
Using office discipline
referrals as a metric for
universal screening of
student social behavior
~5%
6+ office discipline
referrals
~15%
2-5 office discipline
referrals
0-1 office
discipline referral
~80% of Students

Build a picture for the pattern of office
referrals in your school.
Goal
 Compare the picture with a national average
1.
Identify
empirically
 Compare
the problems
picture with previous
years
2. Identify problems early
 Compare the picture with social standards of
3.
Identify problems in a manner
faculty, families, students.
that leads to problem solving
not just whining
Total Office Discipline Referrals
Total Office Discipline Referrals as of January 10
3.49
2.75
2.5
2.7
1.8
1.4
0
.00
Change
Report Options
Grade Range
Number of
Schools
Mean Enrollment Median ODRs
per school
per 100 per
school day
K-6
2565
452
.22
6-9
713
648
.50
9-12
266
897
.68
K-(8-12)
474
423
.42
Elementary School
465 students (465/ 100 = 4.6 X .22= 1.01)
Our rate of
problem behavior
has been above
the national
average for
schools our size
across 9 of 10
months this year.
There has been a
decreasing trend
since Dec.
Elementary School
1000 Students (1000/100 =10 X .22= 2.2)
The rate of
problem behavior
has been at or
below the
national average
for schools our
size across 6 of
10 months. The
past 4 months
have been below
the national
average
Middle School 765 students (765/100 = 7.6 X .50= 3.8)
The rate of
problem behavior
has been at or
below the
national average
for schools our
size across 9 of
10 months. The
past 8 months
have been below
the national
average with a
decreasing trend
Describe the narrative for this school
Describe the narrative for this school
Describe the narrative for this school
Describe the narrative for this school




Our average Major ODRs per school day per month
are higher than national average for a school of our
enrollment size
Our average ODRs per school day per month are
higher this year than for corresponding months of
previous year
Our average ODRs per school day per month are
showing an increasing trend
Faculty, parents, and students say our ODR levels
are too high
43
1. Define problem by identifying What problem
behaviors are involved in ODRs
2. Clarify problem by identifying
a) When ODRs are occurring (time of day)
b) Where ODRs are occurring (location)
c) Who is engaging in problem behaviors that result
in ODRs
d) Why are problem behaviors continuing to occur
44
1a. Too many ODRs
1b. Total of 22 aggression ODRs on playground
last month; twice as many as last year & showing
increasing trend this year; occurring during first
recess; 15 different students involved; aggression
appears to provide peer attention, and resolve
unclear playground rules (who gets equipment),
2a. Verbal threats and gender
harassment in the cafeteria are
increasing; 80% of events are from 4
students during second lunch; We are
unclear what is maintaining these
behaviors.
2b. Behavior in cafeteria is uncivil and unsafe.
3a. Hallway noise is too loud
(disruptive) during 7th grade passing
periods before and after lunch.
3b. Hallway noise is unbearable.
4a. The number of ODRs per day has
increased by 20% each month since
school started.
4b. The number of ODRs per day has increased by
20% each month since school started. Most
incidences are with 4-6 grade, in the afternoon.
Students are engaging in inappropriate language
45
and harassment.
Team Initiated
Problem Solving
(TIPS) Model
Identify
Problems
Develop
Hypothesis
Evaluate and
Revise
Action Plan
Collect
and Use
Data
Develop and
Implement
Action Plan
Discuss and
Select
Solutions
Problem Solving
Meeting Foundations
46


Carly is having reading
difficulties
50% of 2nd graders are
not meeting math
benchmarks


Carly is reading 20 cwpm
(goal is 60), skips or
guesses at words she
doesn’t know, mostly
during language arts
2nd graders, who entered
school after Oct 31, do
not know whole numbers
75-100 and are not
accurately adding two
digit numbers because of
lack of skills
Our school did not
meet AYP last year
The 5th graders are
below the state
proficiency score
as compared to 5th
graders in Oregon
The past two years this
cohort of students (3rd
to 5th grade) has
gradually decreased in
overall proficiency, their
comprehension strand
scores are low, we
shifted to blended
grade classes during
their 4th grade year

Role
◦ To create data summaries that will facilitate the
team in
 determining if there are problems
 jump starting a problem solving discussion, and
 evaluating the impact of solutions and fidelity of
implementation

General Responsibilities
◦ Prepares a brief written summary for distribution at
meetings using each of the data sources needed for
problem solving and decision making
◦ Help to generate reports during the meeting as questions
of the data arise

How?
◦ Establish the role of a data analyst (and backup
person)
◦ Teach data analyst to develop data summary
 Oakes, DIBELS, SWIS…. Etc
◦ Start meeting with defining the problem with
precision
◦ Refine precision of problem statement through
inferences and hypothesis
 Have data accessible for custom report generation
during the meeting


Middle School – Grades 6, 7, & 8
565 students
51
Avg. ODRs Per School Day
20
15
10
5
0
Sep
Oct
Nov
Dec
Jan
School Months
School Avg.
National Avg. = 4.8
52

Identified problem
◦ for last 4 mos., Major ODRs per day higher than
national avg.
◦ increasing trend across all 5 mos.
53
140
120
100
80
60
40
Number of Referrals
Referrals by Location
200
180
20
0
Student No.
54
5:00 PM
4:30 PM
4:00 PM
3:30 PM
3:00 PM
2:30 PM
2:00 PM
1:30 PM
1:00 PM
12:30 PM
12:00 PM
11:30 AM
11:00 AM
Referrals by Problem Behavior
10:30 AM
10:00 AM
9:30 AM
9:00 AM
8:30 AM
40
30
20
8:00 AM
60
50
7:30 AM
90
80
70
Number of Referrals
120
110
100
7:00 AM
Number of Referrals
140
130
1
13
16
18
2
20
24
28
30
33
38
4
9
17
21
37
43
23
31
39
40
41
5
8
11
29
12
22
25
35
42
6
14
34
15
26
36
7
3
19
32
27
10
Café
Hall
Common
Class
Other
Special evt
Bus
Bus Zn
Gym
Bathrm
Library
Music rm
Stadium
Off-Campus
Locker rm
Office
Unknown
Park lot
Minor
Tardy
Bomb
Arson
Weapons
Other
Unknown
Drugs
M-Prpty Misuse
M-Other
M-Dress
M-Tech
Tech
Inapp affection
Out bounds
M-Unknown
Gang display
Skip
Truan
Lying
M-Disruption
Dress
Tobacco
Alcohol
Combust
M-Inapp lan
Forge/Theft
Vandal
M-Contact
M-Disrespt
Prop dam
Agg/Fight
M-Tardy
Skip
Harass
Disrespt
Inapp lan
Disruption
10
0
Plygd
Number of Referrals
Trevor Test Middle School
11/01/2007 through 01/31/2008 (last 3 mos.)
Referrals by Time
130
120
110
100
90
80
70
60
50
40
30
20
10
0
Referrals by Student
80
160
70
60
50
40
30
20
10
0
1.
2.
3.
Most Disruptions occur in Cafeteria
Most Disruptions occur in Cafeteria
between 11:30 AM and 12:00 PM
Most instances Inappropriate Language
occur in Cafeteria between 11:30 AM and
12:00 AM
Now…use a Custom Graph to confirm (or
disconfirm) your inferences, starting with
Disruptions, by grade level
55
Many instances of disruption (what)…
 occurring in cafeteria (where)…
 between 11:30 AM and 12:00 PM (when)…
 with large majority involving 6th graders
(who)…
 particularly Student #10 (who)

56



Write your precise Problem Statement as
one element of your “Problem-Solving
Action Plan”
The P-S Action Plan is simply a record of
◦ team decisions and
◦ actions needed to implement the decisions
Here’s Problem Statement section of P-S
Action Plan for Trevor Test Middle School
57
58
59


Many instances of disruption occurring in
cafeteria between 11:30 AM and 12:00 PM;
large majority involving 6th graders,
particularly Student #10…
because (a) cafeteria overcrowded at that
time, (b) 6th graders have received
insufficient instruction in cafeteria
expectations, and (c) disruption results in
attention from adults and peers
Here’s hypothesis statement incorporated
into P-S Action Plan
60
61
 Is
best explanation for what the data and
your experience tell you
 Provides a possible “why” for other Ws you
discovered
 AND guides you toward possible solutions
62

Prevent – Remove or alter “trigger” for problem behavior

Define & Teach – Define behavioral expectations; provide

Reward/reinforce – The expected/alternative behavior

Withhold reward/reinforcement – For the problem

Use non-rewarding/non-reinforcing corrective
consequences – When problem behavior occurs

Consider Safety issues
demonstration/instruction in expected behavior (alternative
to problem behavior
when it occurs; prompt for it, as necessary
behavior, if possible (“Extinction”)
63
Trevor Test Middle School
Hypothesis - cafeteria overcrowded; 6th graders with insufficient instruction in
cafeteria expectations; attention from adults and peers rewarding disruption
Prevent “Trigger”
Define & Teach
Reward/Reinforce
Withhold Reward
Corrective consequence
Other
Safety
64
Trevor Test Middle School
Hypothesis - cafeteria overcrowded; 6th graders with insufficient instruction in
cafeteria expectations; attention from adults and peers rewarding disruption
Prevent “Trigger”
Change lunch schedule so fewer students are eating between 11:30 AM
& 12:00 PM?
Define & Teach
Focus on 6th graders; define cafeteria expectations; develop and post
expectation signage in cafeteria; demonstrate/teach expectations in class
periods occurring just prior to lunch
Reward/Reinforce
Set up “Friday 5” (extra 5 mins. of lunch time on Friday, if no ODRs
occur in cafeteria during lunch time)
Withhold Reward
Ensure staff don’t argue back and forth with student if instance of
disruption occurs (may be an inadvertent reward); remind students that
paying attention to a disruptive student can mess up Friday 5
Corrective consequence
Ensure active supervision during lunch (add one supervisor between
11:30 AM and 12:00 PM?); ensure quick corrective consequence, per
our handbook
Other
Determine whether Behavior Support Program has been initiated for
Student #10; if it has, make sure it includes focus on disruption in
cafeteria
Safety
65



Choose the least number of solution(s)that
will have the biggest impact on decreasing
the problem.
Implementing the solution requires action
Here are solution actions, added to the P-S
Action Plan…
66

Fidelity
◦ Did we do what we said we would do?
 Make it simple

Student Outcomes
◦ Did our intervention produce the outcomes we were
expecting
 Use the right data to answer the questions you are
asking

Choose the solutions that will create an
environment that makes the problem irrelevant,
inefficient, and ineffective.
 Choose least amount of work that will have the biggest
impact on decreasing the problem.



Implementing the solution requires action and
time lines
Problems
need
goals
so thatthe
we can
measure
Are
we
doing
plan?
progress and know when to move on.
Use weekly
1-52
survey
of …..
playground
monitors
to
1
…..
…..3
4
…..
5
assess implementation of plan
No
Yes
68
Team Initiated
Problem Solving
(TIPS) Model
Identify
Problems
Develop
Hypothesis
Evaluate and
Revise
Action Plan
Collect
and Use
Data
Develop and
Implement
Action Plan
Discuss and
Select
Solutions
Problem Solving
Meeting Foundations
69