COSA Assessment Conference
Mickey Garrison
Tony Alpert
Jon Wiens
Beth L aDuca
 Need a question about preliminary results!
 What data are essential to ensuring continuous
improvement? (strand 2)
 Where can administrators find state assessment and report
card data? (strand 1)
 What tools can be used to help focus analysis, planning and
evaluation? (strand 2)
Broadly Explore Successes & Challenges
Start with OAKS
results to narrow
the focus for
digging deeper at
the classroom
level.
Winnow Data
Infer Cause/Effect
Relationships
SMART
Goals
Hypothesize Improvement
Strategies
Use student and
adult data collected
locally to dig deeper
and gain clearer
insights.
3
District A
District B
Reading, 2008-09, Gender
Math, 2008-09, LEP
 Subtest or Strand scores are based on
fewer items
 Fewer items = greater measurement
error = lower reliability of scores
 Use with caution
 Understand this limitation
 Triangulate—look for convergent
evidence to develop interventions or
instructional plans for students.
http://www.ascd.org/publications/educationalleadership/feb09/vol66/num05/Unraveling-Reliability.aspx
 Measures of error for group scores tell you how
much confidence you can have that you’ve
pinpointed the group’s actual performance
(provides a range or band of scores)
 Influenced by size of group and the size of the
range of the scores of students in the group
 Student level—provides a range of values that
represent the range within which a student would
score again.
 Influenced by number of items a student
answers for a particular score category
 The more items answered, the smaller the
measure of error.
Compare the size of
measurement error for
total mathematics vs.
each of the five
subscores
Broadly Explore Successes & Challenges
Winnow Data
Infer Cause/Effect
Relationships
Use simple or complex
data tools to collect and
organize data and gain
insights from your data.
SMART
Goals
Hypothesize Improvement
Strategies
Use process tools to
winnow data, analyze
for meaning, set
SMART goals,
hypothesize
improvement
strategies and facilitate
decision making.
14
1.
2.
3.
4.
5.
Collect and chart data
Analyze strengths and obstacles
Establish goals: set, review, revise
Select instructional strategies
Determine results indicators
What impacts effectiveness of data teams and
use of data to inform instruction?
What roadblocks exist to prohibit
effectiveness?
Calculations &
Estimation
Measurement
Statistics &
Probability
Algebraic
Relationships
Geometry
Grade 10
235.7 235.4 234.9 236.2
236.3
Grade 8
235.0 235.4 235.4 235.7
235.6
Grade 5
222.7 224.2 224.2 223.9
225.0
Grade 3
210.3 210.8 212.1 209.7
212.8
Calculations &
Estimation
Measurement
Statistics &
Probability
Algebraic
Relationships
Geometry
Grade 10
236.2 235.9 235.2 236.7
236.9
Grade 8
235.6 235.5 235.4 235.9
235.8
Grade 5
223.4 224.5 225.2 224.6
225.2
Grade 3
211.1 211.9 212.5 210.6
212.1
Vocabulary
Read to
Demonstrate Develop an
Perform a
General
InterTask
Understanding pretation
Examine
Content &
Structure –
Info. Text
Examine
Content &
Structure –
Lit. Text
Grade 10
238.4 239.0 239.5 238.9 238.2 238.6
Grade 8
234.3 234.6 234.9
Grade 5
223.1 223.2 225.6 223.4 223.6 222.6
Grade 3
213.2 212.5 213.9 212.6
234.5 233.8 234.5
NA
NA
Vocabulary
Read to
Demonstrate Develop an
Perform a
General
InterTask
Understanding pretation
Examine
Content &
Structure –
Info. Text
Examine
Content &
Structure –
Lit. Text
Grade 10
239.1 239.7 240.2 239.2 239.2 238.9
Grade 8
234.2 234.2 235.0
Grade 5
223.6 223.5 225.3 223.8 223.5 222.9
Grade 3
213.4 213.2 214.1 213.4
234.4 233.9 234.6
NA
NA
Antecedents
(organization)
Instructional
Strategies
• Structures and learning conditions that precede and predict results.
These may include planning time, focused resources, intentional
professional development.
• Factors that have a strong correlation with results.
• Teacher-student practices that engage students in thinking. These
practices require training, practice, and ongoing professional development
to implement with mastery.
• These do not necessarily influence student cognition
Causes
(implementation)
• Adult behaviors and routines that create specific effects or results in
student behavior or achievement. These may include classroom
management routines, classroom schedule routines, transition routines, etc.
• These do not necessarily require extensive training to implement with
mastery.
Effect
• Student learning which can be measured by formative, interim, and
summative assessments.
• The measures do not have to be extensive or an additional layer but rather an
intentional use of existing materials and resources
2008-2009
Test
Subject Grade
OAKS
Math
8
OAKS
Math
8
NAEP
Math
8
NAEP
Math
8
OAKS Reading
8
OAKS Reading
8
NAEP Reading
8
NAEP Reading
8
Sub-Group
Mean St. Dev. Mean Dif. Effect Size
Hispanic Students 230.3
White Students
9.2
236.6
11.2
Hispanic Students 263.8
32.8
White Students
290.0
34.1
Hispanic Students 229.5
8.0
White Students
235.5
8.6
Hispanic Students 247.0
33.0
White Students
268.9
31.2
6.3
0.58
26.2
0.77
6.0
0.71
21.9
0.70
Average
Effect
Size
Percentile
Gain
Number of
Studies
Identifying Similarities & Differences
1.61
45
31
Summarizing & Note Taking
1.00
34
179
Reinforcing Effort & Providing
Recognition
.80
29
21
Homework & Practice
.77
28
134
Nonlinguistic Representation
.75
27
246
Cooperative Learning
.73
27
122
Setting Objectives & Providing
Feedback
.61
23
408
Generating & Testing Hypotheses
.61
23
63
Cues, Questions, & Advance Organizers
.59
22
1251
Instructional Strategies
Marzano, Pickering, Pollock
Triangulate!
23
Triangulation!
Achievement of Results
Lucky
Leading
Losing
Ground
Learning
Understanding Antecedents of Excellence
* Source: The Leadership and Learning Center
25
What’s
Working?
1.
2.
3.
4.
5.
Lucky
Leading
Losing
Learning
Evidence that Convinces
1.
2.
3.
4.
5.
26
www.oregondataproject.org
Mickey Garrison
541-580-1201
mickey@oregoneesc.org
28