Power Standards and SMART Goals;
Expectations for Assessment and
Data-based Decision-Making.
Presented by:
Ben Ditkowsky, Ph.D.
Waukegan Unit School District 60
Power Standards
• A power standard refers to those few
things that are really important to teach
(Reeves).
– BIG IDEAS (Kame’enui & Carnine)
– Enduring understandings (Wiggins & McTigue)
• The key question here seems simple:
• What is it that we want students to know
and be able to do?
The practical reality of
NCLB
If we are going to be able to
“leave no child behind” the state
standards, then we are going to have
to have a way of determining how far
behind they are, from where we expect
them to be, so that we can do
something to accelerate their
academic growth.
…a way of determining how
far behind they are, from
where we expect them to be
That means we need to use
data to make decisions
Increase
Monitor
Identify
Promote
To
ensure
decide
adjust
student
apretest
new
student
teacher
teaching
who
student
filecompliance
goals
has
progress
reflection
anxiety
as
for
failed
“at
are
students
–risk”
to
not
for
and
meet
aempty
goal
goal
who
standards
setting
have
Plan
Grade
Meet
Lock
and
students
district
ahigh
group
into
for
and
apractices
instruction
send
group
home
Make
–folders
stakes
decision
based
exceeded
(grade
retention)
expectations
on one
test
Uses of data? + -
Good Use
Poor Use
Plan and group for instruction
Monitor student progress for a
goal
Identify a student as “at –risk”
Identify new goals for students
who have exceeded
Promote student reflection and
goal setting
To adjust teaching practices
Grade a pretest and send
home
To ensure file folders are not
empty
Meet district compliance
Lock students into a group
Make high – stakes decision
based on one test
To decide who has failed to
meet standards (grade
retention)
Increase teacher anxiety
When we use data we need to FOCUS:
Things to consider
• Entry level skills
– If students come in with less skill than we
expect then we need to get them to learn more
with the same amount of time.
• Efficiency and Effectiveness of Instruction
– Time on task
– Intensity of instruction
• Data-based decision-making
– Frequency of Assessment
– Use of data to drive instructional decisionmaking
Entry Level Skills
More
Skill
Less
Skill
• Typical students have a benefit over low performing
students.
• They come in higher, so they don’t need to gain as
much
• Keep in mind that we need to make sure students
have prerequisite skills
Instruction
More
Skill
Less
Skill
• To make up the difference, instruction should:
– Be focused on the goal
– Maintain student time on-task
– Provide sufficient practice to ensure students have the
opportunity to learn
Assessment
• The more dire the student needs, the
more carefully we should watch their
progress.
• When students are not progressing,
we need to change what they receive
in terms of instruction.
Monitoring progress requires us to pay
attention
• Is the instruction working?
• Is it intense enough?
• Do we need to change what we are
More
doing?
Skill
Less
Skill
Use Assessments from the Curriculum
• To determine when to change instruction
More
Skill
Less
Skill
• Rule: 3 data points below the aim-line
Continuously Evaluate
• Is it working now?
More
Skill
Less
Skill
• Well enough to meet the goal?
Continuously Evaluate
• Is it working now?
More
Skill
Less
Skill
Set Goals, and make them smart!
•• Goals should be
SSpecific and Strategic
•• Goals should be Observable and
M
Measurable
•• Goals should be Acceptable
Action oriented and Attainable
A
•• Goals should be Realistic and
RRelevant
•• Goals should be Trackable and
TTime-bound
How do we know what
goals to set?
Use Power Standards:
What is it that we want students
to be able to DO?
Curriculum-Based Measurement
• Does how quickly a student can
orally read relate to how well
they do on ISAT?
•
Will a 5th grade student who…
– … reads 120 cwpm meet standards?
– What about a student who can read 150
cwpm, will they meet?
Think – Pair -Share
1. Does fluency matter? Yes/No
2. How many words should a student
in your class be able to read in a
minute?
3. Why do you think so?
Does Fluency really
matter?
1. Does it matter? Yes/No
2. How many words should a 5th grade
student read in a minute?
Let’s consider some data…
How many words can
they read in a minute?
• How many correct words did 5th grade
students read in one minute?
• Does the number of words that students can
read in one minute relate to how well they do
Mark read 175 words
on the ISAT?
Dorothy read 225 words
Sarah read 147 words
Kelsey read 120 words
Let’s reorganize things
• Some times we need to change the
way we look at things before we can
FOCUS.
Setting up a chart to
compare scores
Grade 5
240
220
200
180
160
Exceeds
Before ISAT testing what we knew was how
Meets
Exceeds
fast students read a grade level passage.
ISAT
Below
140 Meets
120 Below
Warning
100
80 Warning
Dorothy read 225 words
Mark read 175 words
Sarah read 147 words
60
Kelsey read 120 words
40
20
0
290
280
270
260
250
240
230
220
210
200
190
180
170
160
150
140
130
120
110
100
90
80
70
60
50
40
30
20
10
0
Setting up a chart to compare scores
Grade 5
240
220
200
Exceeds
180
Meets
160
Below
ISAT
140
120
Warning
100
80
60
40
20
0
290
280
270
260
250
240
230
220
210
200
190
180
170
160
150
140
130
120
110
100
90
80
70
60
50
40
30
20
10
0
R-CBM
A note about “Scatter – Plots”
Scatter-Plot can help us to see
– How scores from two different tests
relate to each other.
– Where we should set cut-points for
students meeting expectations
– How we can increase student’s
chances to meet ISAT Standards.
Looking at relations
Grade 5
240
How did she do?
220
Dorothy read 225 Words in a minute
And Dorothy obtained a 169 on ISAT
200
Exceeds
180
Meets
160
Below
ISAT
140
120
Warning
100
Let’s see how Dorothy did on ISAT…
80
60
40
20
0
290
280
270
260
250
240
230
220
210
200
190
180
170
160
150
140
130
120
110
100
90
80
70
60
50
40
30
20
10
0
BDitkowsky@Waukeganschools.org
R-CBM
Let’s consider…
Grade 5
240
220
200
Exceeds
180
Meets
160
Below
ISAT
140
120
Warning
100
80
60
40
20
0
290
280
270
260
250
240
230
220
210
200
190
180
170
160
150
140
130
120
110
100
90
80
70
60
50
40
30
20
10
0
R-CBM
Students who read less than 120
correct words per minute
Grade 5
240
She read 120 words correctly in a minute
She obtained a score of 150 on ISAT
220
200
Exceeds
180
Meets
160
Below
ISAT
140
120
Remember Kelsey?
Warning
100
80
60
100% of (62, 5th grade) students who read less than 120 words
20
per minute were below state standards on ISAT
40
0
290
280
270
260
250
240
230
220
210
200
190
180
170
160
150
140
130
120
110
100
90
80
70
60
50
40
30
20
10
0
R-CBM
Students who read less than 130
correct words per minute
Grade 5
240
220
200
Exceeds
180
Meets
160
Below
ISAT
140
120
Warning
100
80
60
40
100%
of (62, 5th grade) students who read less than 130 words
20
per
minute were below state standards on ISAT
0
290
280
270
260
250
240
230
220
210
200
190
180
170
160
150
140
130
120
110
100
90
80
70
60
50
40
30
20
10
0
R-CBM
Students who read less than 140
correct words per minute
Grade 5
240
220
200
Exceeds
180
Meets
160
Below
ISAT
140
120
Warning
100
80
77%
of (62, 5th grade) students who read less than 140 words
60
per
40 minute were below state standards on ISAT
20
0
290
280
270
260
250
240
230
220
210
200
190
180
170
160
150
140
130
120
110
100
90
80
70
60
50
40
30
20
10
0
R-CBM
What can we infer about students who
are not fluent readers and ISAT?
(<150 cwpm)
Grade 5
240
220
200
Exceeds
180
Meets
160
Below
ISAT
140
120
Warning
100
80
60
th grade) students who read less than 150 words
75%
of
(62,
5
40
per minute were below state standards on ISAT
20
0
290
280
270
260
250
240
230
220
210
200
190
180
170
160
150
140
130
120
110
100
90
80
70
60
50
40
30
20
10
0
R-CBM
Think – Pair - Share
What are two reasons why you think
the speed at which our students read
influences their scores on high –
stakes tests?
1.There is time to complete the test.
2. They don’t have to spend time
decoding, instead they can spend
time comprehending.
Let’s consider just the
students who read more than
150 correct words per minute
And Dorothy?
Grade 5
240
220
Remember Mark?
200
Exceeds
180
Meets
160
Below
ISAT
140
120
Warning
100
80
60
th
90%
40 of (62, 5 grade) students who read more than 150 words
per minute were above state standards on ISAT.
20
0
290
280
270
260
250
240
230
220
210
200
190
180
170
160
150
140
130
120
110
100
90
80
70
60
50
40
30
20
10
0
R-CBM
Place your bets
• If you were going to bet that a student
was going to meet expectations on the
ISAT who would you choose?
– A student who reads 120 cwpm?
• 100% of our sample did not
– A student who reads 130 cwpm?
• 100% of our sample did not
– … 140 cwpm? Or 150 cwpm?
• 77% and 75% of our sample did not
– What about a student who reads more than 150
cwpm?
• 90% of our sample of students reading more than 150
cwpm met State Standards.
– Pick your student.
Reading CBM: An
Instructional Focus by grade level
For 5th grade students
• By April 2, 200x
• When given a randomly selected passage from the
5th grade Reading Series and one minute to read
• 90% of students will orally read
• Specific
• 150 words correctly with
95 to 98% accuracy
• Measurable
Consider the data:
90% of 5th grade
students who read 150 words
correctly met ISAT standards
• Action-oriented
• Relevant
• Time-bound
Think – Pair - Share
What are two factors that WE CAN CONTROL,
other than the speed at which children read,
that might influence their scores on
high – stakes tests?
1. Vocabulary
2. Comprehension strategies
3. Other Basic skills
4. Language skill
5. Test – taking strategies, etc…
What about 3rd Grade?
Grade 3
220
200
Exceeds
180
of students
who read
92% of students
93% of students
who100%
read
who
read
140
CWPM
met
Standards
94 CWPM
115orCWPM
more met
or
more
ISAT
met
Standards
ISATISAT
Standards
160
9/14 : 64% of students who read less than 94 did
not meet ISAT Standards
Below
140
120
Warning
Can you see a pattern?
R-CBM
25
0
24
0
23
0
22
0
21
0
20
0
19
0
18
0
17
0
16
0
15
0
14
0
13
0
12
0
11
0
10
0
90
80
70
100
60
ISAT
Meets
Should Oral Reading Fluency be a
FOCUS for third grade?
Consider this:
– 92% of 3rd students in the (2 school sample) who read
more than 115 words correctly in one minute met ISAT
standards.
– Fewer than 10% of students who failed to read at a
target rate of 85 cwpm met ISAT standards.
• Make goals S, M, A, R, T
By April 2, 200x, when given a randomly selected
passage from the 3rd grade Harcourt Reading
Series and one minute to read, 90% of 3rd grade
students will orally read 94 words correctly with
95 to 98% accuracy
Re-confirmation of data
(Grade 3 N > 500)
Curriculum-Based Measurement and High Stakes Testing
Grade 3 (N =537)
220
27%
66%
90%
210
190
180
170
160
150
140
130
73%
10%
34%
Reading (CWPM)
0
26
0
24
0
22
0
20
0
18
0
16
0
14
0
12
0
10
80
60
40
20
120
0
Reading ISAT Scale Score
200
Re-confirmation with data
(Grade 5 N > 500)
Curriculum-Based Measurement and High Stakes Testing
Grade 5 (N =523)
220
36%
61%
88%
210
190
180
170
160
150
140
130
64%
12%
39%
Reading (CWPM)
0
26
0
24
0
22
0
20
0
18
0
16
0
14
0
12
0
10
80
60
40
20
120
0
Reading ISAT Scale Score
200
What should we expect?
• Some variation is expected in oral reading fluency
depending on passage difficulty, but clear targets
are supported:
• Grade 3
– Fall - 60 is cause for concern  80 is a target
– Winter - 75 is cause for concern  100 is a target
– Spring - 90 is cause for concern  115 is a target
• Grade 5
– Fall 95 is cause for concern  130 is a target
– Winter - 125 is cause for concern  145 is a target
– Spring - 140 is cause for concern  155 is a target
What about Vocabulary?
• A pilot for middle school
Grade level Teams
Identify critical vocabulary
Student friendly definintions
Vocabulary Matching - CBM
1
Directions:
Science
Vocabulary Matching
You will have 5 minutes to work on this activity. Read through the definitions below.
Beside each definition, write the number of the word that best matches that
definition. Remember, not all of the words will be used.
Name:
Date:
Definitions
The planet you live on
(sample)
Anything that has mass and volume
Force of attraction between two objects
In an experiment, the part that is changed on purpose
Force needed to change movement
The maintenance of a stable internal environment
The amount of matter in a given space; mass per unit volume
Correct
All of space
In an experiment, the part that is measured
Force of a moving object
Way to compare things
Study of genetics
The process by which one or more substances undergo change
to produce one or more different substances
A series of steps that scientists use to answer questions and
solve problems
Relating to science
A collection of organs that gather and interpret information about
the body's internal and external environment and respond to
that information.
Living and nonliving organisms interacting
Relationship among organisms
A prediction based on available information.
18
4
16
5
14
20
1
7
10
11
3
17
22
13
8
6
26
12
21
25
Word Bank
1 density
2 respiratory system
3 Momentum
4 matter
5 manipulated variable
6 scientific
7 accurate
8 scientific method
9 immune system
10 Universe
11 responding variable
12 ecosystem
13 chemical reaction
14 Inertia
15 atom
16
17
18
19
Gravity
Measurement
Earth
adaptation
20 homeostasis
21 Interaction
22 Heredity
23 reproduction
24 Volcano
25 hypothesis
26 nervous system
Administration and Scoring
Vocabulary Matching
• VM Curriculum-Based Measurement
can be individually or group
administered.
Teacher
Student
Reads standardized directions
Reads for two to five minutes.
Monitors student performance
on practice items
Writes the number of words
matching definitions
Monitors students while they
take the test.
Counts the total number of
correctly circled items
Does Vocabulary relate to how well students
do on the state test?
• How many students who
obtained a score of fewer
than 5 on a VM (5 minutes)
measure met standards for
grade 7 science?
• How about 10?
• What about more than 10
correct?
34%
65%
97%
Does Vocabulary relate to how well students
do on the state test?
60
Vocabulary and Science ISAT
Meet
DNM
Likely
NOT to Meet
40
Likely
to Meet Standards
30
20
10
VM Score
20
18
16
14
12
10
8
6
4
2
0
0
No. Students
50
The Measurement Pool
• What words will a successful student
know by the end of the school year?
• The student is likely to know some of
the key words before you begin.
• As the year progresses, more words
are taught, and more words are
learned
How can writing be assessed
at the classroom level
A first grade classroom
example, goals and data:
By March 19, 200X, when given a writing prompt, 90%
first grade students will write a complete sentence in
the language of instruction that directly includes the
prompt in the response with an explanation (quality).
By December 19, 200X, when given a writing prompt, 90% of first grade
students will write a complete sentence in the language of instruction.
7
88% met the goal
6
5
38%
4
50%
3
10%
2
1%
1
0
5
10
1%
15
20
25
30
35
40
45
50
55
60
65
Time 1
A complete sentence includes (1) a noun and a verb (2) begins with a
capital letter, (3) ends with punctuation, (4) has a space between
words. 5) To exceed means more than one sentence with 4 elements.
By January 19, 200X, when given a writing prompt, 90% of first grade students
will write a [part 1] complete sentence in the language of instruction that [part 2]
directly includes the prompt in the response (quality).
93% met part 1 of the goal
77% met part 2 of the goal
7
6
57%
5
36%
4
7%
3
2
High Quality Indicator
Beginning Quality Indicator
Quality absent
Insufficient quantity indicators
1
0
5
10
15
20
25
30
35
40
45
50
55
60
65
Time 2
A complete sentence includes (1) a noun and a verb (2) begins with a capital
letter, (3) ends with punctuation, (4) has a space between words. (5) more
than one sentence with 4 elements,
What do we need to do when not
everyone meets?
• Meet with peers / peer mentors
(Grade 1 Team meetings)
• Instructional support
• Increasing focus and intensity of
instruction
By January 19, 200X, when given a writing prompt, 90% of
first grade students will write a [part 1] complete sentence in
the language of instruction that [part 2] directly includes
the prompt in the response (quality).
93% met part 1 of the goal
77% met part 2 of the goal
By March 19, 200X, when given a writing prompt, 90% of 1st
grade students will write a complete sentence in the language
of instruction that directly includes the
prompt in the response with an explanation (quality).
7
6
78 %
68%
5
22%
4
3
2
High Quality Indicator
Beginning Quality Indicator
Quality absent
Insufficient quantity indicators
1
0
5
10
20
15
25
30
35
40
45
50
60
65
70
Time 3
A complete sentence includes (1) a noun and a verb (2) begins with a
capital letter, (3) ends with punctuation, (4) has a space between
words. (5) more than one sentence with 4 elements,
Goal setting at the Grade
level
A Kindergarten Center
example…
Making Instructional
Decisions based on Data
1. Identification
of Needs
2. Validate
Needs
What is
thethe
goal
of instruction?
does
student
need?what
Is
the
difference
between
How
implementation
integrity
be ensured?
What
will be
used?
Dowill
wematerials
have enough
data
to confirm
or
is expected
and what
is occurring?
Was
the intervention
plan
successful?
How
progress be monitored?
refutewill
a hypothesis?
5. Plan
Evaluation and
Modification
4. Plan
Implementation
3. Plan
Development
The Plan:
Three Tiers of Intervention
• Defined: A data – driven model for
differentiated instruction for all
students
• Focus on what works
– Scientifically Based Reading Instruction
– In the context of a self – correcting
model of decision-making
• Identifies / verifies need then
organizes support that students need
Three Tiers, with an Emphasis on
Prevention at Each Level
• Tier III (FEW students)
Tertiary Intervention
– Reduce complications, intensity,
severity of current problems
• Tier II (SOME students)
Secondary Intervention
– Reduce current cases of
academic and behavior problems
• Tier I (ALL students)
Primary Intervention
– Reduce new cases of academic
and behavior problems
Students who need more…
receive more
Tier 3:
Intensive Instruction
Intensity of service
Tier 2:
Strategic Instruction
Tier 1:
Benchmark Instruction
(Typical)
Intensity of Need
Tier 3:
Intensive Instruction
Intensity of service
Tier 2:
Strategic Instruction
Tier 1:
Typical Instruction
Intensity of Need
2004-2005 Meeting
Expectations: In Phonological Awareness
By June 1st 200X, when given a DIBELS Phoneme
Segmentation Fluency Probe, 95% of students will
orally identify more than 10 phonemes per minute
(Emerging phonological awareness or better).
Sixty-percent (60%) of Kindergarten students will
identify 35 or more phonemes per minute
(Established Phonological Awareness).
60
Three Tiers in action
50
PSF
Intensity of service
Student: Jen
• September: Tier 3
• October: Strategic Monitoring
• Successful intervention confirmed
with progress monitoring
• December: Tier 1
Corrects
40
NWF
30
20
ISF
10
LNF
0
Fall
Benchmark
Oct
Nov
Dec
Jan
Winter
Benchmark
Tier 3:
Intensive Instruction
Tier 2:
Strategic Instruction
Tier 1:
Typical Instruction
Intensity of Need
Feb
Mar
Monitoring progress
Think graphically
How much or how many?
What is your goal?
What should progress look like?
Keep changing
until
the data
Collect
and graph
data!
look the way you expect them
If data do not match the
to look!
expected course, change
something!
Collect enough data to be
reasonably confident that
you know baseline
PLEP
(e.g., the median of 3)
Time
If data don’t show growth
change something!
Leaving No Child Behind
• We agree with the goal of No Child Left Behind
– Safe schools
– Qualified teachers, etc.
– All children reading at grade level
• The practical reality is that while we may not be
able to demonstrate on level results on State
tests, SMART goals will show us which students
need more/different instruction to improve their
rate of progress.
• SMART goals will empower teachers and allow
them to celebrate their success with students.
BIG IDEAS
•
•
•
•
Determine your power standards
set smart goals
Monitor progress
If it’s not working, or it’s not working well
enough change it
BIG IDEAS
• There are things that we can monitor that can help
us see how we are going to do.
– Whether we are talking about student academic
achievement, appropriate behavior, or diffusion of
innovations – the first step is identifying specifically what we
are going to measure (Power Standards).
• We know that once we have the Basics,
– Some need more to “meet” expectations.
– And it is always nice to exceed.
• We can be reasonably confident when we answer the
question,
“How are we doing?” before answering
“How did we do?”
…and just for fun, how about a
high school example
Cumulative (estimated) probability of meeting
standards on P S A E
The relation between A C T and
meeting standards on P S A E
100%
90%
The odds of
meeting
standards on
PSAE for a
student who
scored 19 on
ACT is 58.7%
80%
70%
60%
59%
50%
40%
30%
20%
10%
0%
0
2
4
6
8
10
12
14
16
19
18
20
22
24
A C T Math Score (2004)
26
28
30
32
34
A C T Cut scores for
Reading and Science
Reading
Predicted
ACT SCORE probability
2
0.0%
5
0.0%
6
0.0%
8
0.1%
9
0.2%
10
0.4%
11
0.8%
12
1.7%
13
3.4%
14
6.7%
15
12.9%
16
23.3%
17
38.4%
18
56.2%
19
72.5%
20
84.4%
21
91.7%
22
95.8%
23
97.9%
24
99.0%
25
99.5%
26
99.8%
27
99.9%
28
99.9%
29
100.0%
30
100.0%
31
100.0%
32
100.0%
33
100.0%
36
100.0%
Substantial jumps in
probability of meeting
standards for Reading and
Science (PSAE) were
obtained based on ACT as
well
Approximately 90% of
students who score 21 on
any given sub-test meet
standards on PSAE
Science
Predicted
ACT SCORE probability
Mean
7
0.0%
8
0.0%
9
0.0%
10
0.0%
11
0.0%
12
0.0%
13
0.0%
14
0.0%
15
0.1%
16
0.6%
17
2.6%
18
10.0%
19
31.8%
20
66.2%
21
89.2%
22
97.2%
23
99.3%
24
99.8%
25
100.0%
26
100.0%
27
100.0%
28
100.0%
30
100.0%
32
100.0%
33
100.0%
So what does this tell us?
FINDINGS
The probability of meeting PSAE standards increase by nearly 30%
by increasing from an ACT score of 17 to 18 in Reading; 18 to 19 in
Math; and 19 to 20 in Science. Approximately 90% of students
obtaining a score of 21 met standards regardless of the subject area.
These findings are reliable - Analysis of a data set representing more
than 10,000 11th grade students resulted in nearly identical findings
IMPLICATION
Although this is not causal (i.e., 19 does not cause PASE = Meet), the
dramatic increase in probability does compel us to investigate this
relation.
ACTION Question
Is there something that teachers can do to increase the chances of
students obtaining a score of 18,19,20 (for Reading, Math, Science)
on ACT?
FOCUS
We can use the practice ACT data to drive the focus of instruction!
On which type of item ought we
focus?
• All items were examined and sorted
according to difficulty.
• The raw score equating to 21 was
determined.
• Items were coded as Most difficult,
Difficult, FOCUS and Easiest
Suggestion: Use the actual practice test,
examine the items as a team and with
students to determine why those items
were missed. Don’t teach the test, but do
With which students ought
we
focus
and
on
what?
All scale scores were computed.
•
• Scores were compared with probabilities to
designate the risk status of students
– regarding the likelihood of meeting standards on PSAE
– based on data and a little guessing.
• Scores were sorted by class
– English, Science, Math, unidentified class and
– color-coded.
High Risk
Likely to be on academic warning
Indeterminate Risk
May need to assess further, strategize instruction
Low Risk
Maintain current program
Negligible Risk
Likely to meet PSAE if it were given today
HR
IR
LR
NR
Suggestions
• Use the data to begin to strategize
who needs how much of what.
• The more you analyze the actual test
the more you will be able to assist
your students in being ready for the
concepts that are being tested