Local Accountability Professional
Development Series
Building a Local Accountability System with Standards, Assessments,
and Standards-Based Instruction
LEARNING INNOVATIONS AT WESTED
WestEd.org
Overview of Series
Session
Objectives
November Session
Module 1
 Understand the value of a comprehensive local
February 14-15 Session
Module 2 & 5
Unpacking Standards
Creating Reliable and
Valid assessments.
 “Unpack” essential standards by examining the
accountability system
 Learn about essential or power standards
 Begin to think about and identify power standards
 Re-write essential standards in student-friendly
language
 Begin creating assessment calendars
content/concepts, skills, and level at which to teach
and assess (Bloom’s Taxonomy)
 Understand elements of reliable and valid tests
 Begin process of creating informative assessments to
assist with instructional programs and practices to
support improved Student Learning Objectives
WestEd.org
Fill out the initial Pre-Assessment…
Do the best that you can…
If you are unable to write a paragraph
at this time, feel free to write a list of
question you might have.
WestEd.org
Seasonal Partners
When looking for partners…
Find folks you have not worked with in
the past couple of weeks if possible.
WestEd.org
Backward Design
“To begin with the end in mind means
to start with a clear understanding
of your destination. It means to
know where you’re going so that
you better understand where you
are now so that the steps you take
are always in the right direction.”
From The Seven Habits of Highly Effective People by
Stephen R. Covey
WestEd.org
Assumptions about a Local Accountability
System and Data-Driven Instruction
 Collaboratively building
a local accountability
system enhances
school/district culture
 Limiting and ‘unpacking’
essential standards
supports the
development of
formative assessment
practice, differentiated
instruction, and
improved student
performance and
understanding
 Timely, targeted
assessment data
provides teachers with
the opportunity to inform
instruction and
individualize learning
opportunities
 Immediate access to
and use of data
significantly influences
teacher pedagogy and
student performance
WestEd.org
How your work on assessments connects
with your development of SLOs and the
development of related compensations
 Identifying mastery
learning objectives is
essential in creating
effective performance
assessments
 Benchmark (interim)
testing is the “glue” for
an effective local
accountability system
 Formative Assessment
practice informs
differentiated instruction
and provides
opportunities for all
students to mastery
concepts and skills, and
perform at higher
achievement levels
WestEd.org
Benchmark and
Formative
Assessments to
support student
outcomes
Building Your
Assessment
System
Step
1
Identifying
and Using
Essential
Standards
Student
Learning and
Achievement
StandardsBased
Instruction
Using Data
Monitoring
Progress
WestEd.org
Step
2
Benchmark and
Formative
Assessments to
support student
outcomes
Building Your
Assessment
System
Identifying
and Using
Essential
Standards
Student
Learning and
Achievement
StandardsBased
Instruction
Using Data
Monitoring
Progress
WestEd.org
Today and Tomorrow’s Agenda
Unpacking Standards – the big picture, content & skills
Determining Rigor – New Bloom’s Taxonomy and Webb’s Depth
of Knowledge
Practice Unwrapping Essential Standards
Elements of developing valid and reliable assessments
Work sessions - Unpacking essential (Power) standards
- Writing standards in student friendly language
- Developing essential (Power) standards assessments
Used by permission by WestEd for presentations
within the Northern Humboldt District
WestEd.org
10
Four Essential Questions
• What do we want our students to know?
• What will it look like when we get there?
• How will we know when they got there?
• What will we do to get them there?
WestEd.org
Four Essential Questions
• What do we want our students to know?
• What will it look like when we get there?
• How will we know when they got there?
• What will we do to get them there?
WestEd.org
Local Accountability System
Six Elements to consider for implementation.
WestEd.org
Instruction
Formative
Assessment
Reporting
and Using
Results
Standards
Leadership
Professional Development
WestEd.org
Essential Standards
Find your Fall partner and read page 35.
1. What does the
author mean by
“guaranteed and
viable”?
3. At what point
can we say that
sufficient
“opportunities to
learn” were
provided?
*Be prepared to share your
answers with the group.
2. What does the
author mean by
“opportunity to
learn”?
WestEd.org
Why do we need essential standards?
“If American educators were to
adequately cover all of the
knowledge identified in the current
set of standards for the core subject
areas, it might take as much as 22
years of schooling (literally!) within
the current structure.”
From Awash in a Sea of Standards by Robert J.
Marzano and John S. Kendall
WestEd.org
Why do we need essential standards?
Or to put it another way…..
We would be going to school from
kindergarten through grade 21!
WestEd.org
Essential (or Benchmark) Standards
 Subset of state
standards
 Prioritized in terms of:
 Taught on calendar
 Leverage
 Mastery “guaranteed”
 Formative assessments
 Intervention
opportunities
 Readiness
 Not license to eliminate
 Endurance
 Required for state
assessments
 Vertical and
horizontal alignment
other standards from
curriculum
WestEd.org
Essential (or Benchmark) Standards
What makes a standard essential?
 It has endurance. Will this provide students with
knowledge and skills that will be of value beyond a
single test date?
 It provides leverage. Will this provide knowledge
and skills that will be of value in multiple disciplines?
 It ensures readiness for the next level of learning.
Will this provide students with essential knowledge and
skills that are necessary for success in the next grade or
next level of instruction?
 It is regularly assessed by the state. Is this a
standard that students have failed to master?
WestEd.org
Standards Based Curriculum, Instruction, and Assessment
WestEd.org
Essential (or Benchmark) Standards
What considerations need to be made to
identify essential standards?
 Consider teaching order
 Re-write in student friendly language
 Consider allotment of time to teach to
mastery
 Review previous performance data
 Sequence across grade levels
 Vertical and horizontal alignment
WestEd.org
Assessment Beliefs……
(Card activity)
 Count off by (Total number of participants




divided by 3)
Work in teams of three
find the statement(s) that resonate for you
Try to reach consensus as a group
Be prepared to report out to the whole group
WestEd.org
Local Accountability (Return to your seats)
 Read Element 3, “Assessments” on pages 5-7.
 As you read, underline or highlight key quotes,
points, or phrases.
WestEd.org
23
Local Accountability Assessment
 Go around your table, giving each person a
chance to share a favorite quote or idea.
 Decide upon one quote or idea that you’d like to
share with the whole group. Give a reason why
that quote resonates for your group.
 Write your quote and reason on the chart paper
provided.
Used with permission from Learning Innovations at WestEd for Northern Humbolt presentations
WestEd.org
24
Self – Scoring Guide
Pages 22-27
22-27
WestEd.org
Where are you now….. ?
Review pages 21 & 22 and then as a group at
your table, identify where you would place
yourself on the rubrics on pages 23 & 25 & 27
for Assessments, Un-packed standards…..
And Common Formative Assessments.
Be prepared to share your findings………
WestEd.org
26
Quick review of student friendly language
See Handouts
WestEd.org
Unpacking Standards is critical in the
process of identifying foundational
skills……
After data analysis has helped to
define an instructional focus and
Learning Objectives.
See Module 5, Page 15
WestEd.org
Performance or Other Assessments
Competency, Course or Learning
Objective
Competency, Course or Learning
Objective
Formative and Common Formative Assessment(s) to pre-assess
student understanding, inform and individualize instruction
(inform intervention support), and determine mastery.
Standard
1
Standard
2
Standard
5
Standard
3
Standard
6
Standard
9
WestEd.org
WestEd.org
WestEd.org
Module 5: Unpacking Standards
Building the foundation for instruction guided by local assessments
WestEd.org
Backward
Design
Standards
Based
Instruction
Planning
Develop tiered
interventions –
Reteach/Enrich
Use these
assessments in
collaboration
with students to
track
improvement
over time
Used by permission by WestEd for presentations
within the Northern Humboldt District
Select set of
Essential
Standards
Unwrap
Standards
Standards
Based
Curriculum,
Instruction,
and
Assessment
Monitor
implementation
and make
adjustments
Deconstruct
them into
classroom
achievement
targets that form
the foundation
of learning
Create highquality
assessments
of the
classroom
targets
WestEd.org
33
Used by permission by WestEd for presentations
within the Northern Humboldt District Districts
34
WestEd.org
Workshop Essential Questions
1. How are grade level expectations (standards)
unpacked? Why do we need to unpack them?
Used by permission by WestEd for presentations
within the Northern Humboldt District
WestEd.org
35
Rationale
(See page 17 Module 5)
Unpacking grade level expectations will provide a
simple and highly effective way to manage
standards.
Unpacked grade level expectations can help you
address the appropriate cognitive depth for
classroom instruction and assessment.
Used by permission by WestEd for presentations
within the Northern Humboldt District
WestEd.org
36
Definition
“Unpacking standard expectations” means
to identify the concepts and skills found in
the state standards.
IN OTHER WORDS. . .
Used by permission by WestEd for presentations
within the Northern Humboldt District
WestEd.org
37
It means to examine the standards to determine
exactly what students need to:
1. know (the content/concept)
2. be able to do (the skill) considering
3. New Bloom’s (identifying the level of rigor to teach
students the concepts and skills)
And
4. Webb’s Depth of Knowledge (to focus on how deeply a
student has to know the content in order to respond)
Used by permission by WestEd for presentations
within the Northern Humboldt District
WestEd.org
38
Content & Skills
Level of Rigor
•Nouns
•Verbs
•New Blooms
•Webb’s DOK
Parts of an
Unpacked
Standard
Big Ideas
Used by permission by WestEd for presentations
within the Northern Humboldt District
Essential
Questions
WestEd.org
39
What Are Concepts?
Abstract ideas that point to a larger
set of understandings
Concepts = the important nouns
Used by permission by WestEd for
presentations within the Northern Humboldt
District
WestEd.org
40
What Are Skills?
Skills = the verbs
When we “unpack” skills in
a standard, we are looking
for the verbs.
Used by permission by WestEd for presentations
within the Northern Humboldt District
WestEd.org
41
Linear
Non-linear
Patterns
Models
Tables
Sequences
Graphs
Problem situations
Words and symbols
Expression
Equation
Identifies
Generalizes
Writes
Extends
Using
Math Concepts & Skills
Used by permission by WestEd for presentations
within the Northern Humboldt District
WestEd.org
42
Text Features
Main Idea
Supporting details
Organizing
information
Questions
Inferences
Information
Informational text
Stating
Organizing
Generating
Synthesizing
Evaluating
Drawing
Inferences
Reading Concepts & Skills
Used by permission by WestEd for presentations
within the Northern Humboldt District
WestEd.org
43
Used by permission by WestEd for presentations
within the Northern Humboldt District
WestEd.org
44
R:LT:4:2.2: Analyze and interpret elements of
literary texts, citing evidence where appropriate
by- describing main characters’ physical
characteristics or personality traits; or providing
examples of thoughts, words, or actions that
reveal characters’ personality traits
Work in groups of 2 or 3 and use the graphic organizer–
1. Write the standard above in the box at the top
2. Write the nouns in the concepts box
3. Write the verbs in the skills box
Used by permission by WestEd for presentations
within the Northern Humboldt District
WestEd.org
45
Bloom’s Original
Taxonomy
Revised Taxonomy
Separate dimension
Knowledge
Remembering
Comprehension
Understanding
Application
Apply
Analysis
Analyze
Synthesis
Evaluation
C
O
G
N
I
T
I
V
E
Evaluate
Create
Used by permission by WestEd for presentations
within the Northern Humboldt District
WestEd.org
46
Categories of the Knowledge Dimension
There are 4 categories of knowledge.
Factual Knowledge
Procedural
Knowledge
Conceptual
Knowledge
Metacognitive
Knowledge
Used by permission by WestEd for presentations
within the Northern Humboldt District
WestEd.org
47
What kind of knowledge is it?
1. Pizza Pie =
Fractions
2. Colleague
scripts my
lesson = I
reflect
3. Jeopardy
Game with
Content
Facts
4. Training
home
workers on
ornament
construction
WestEd.org
Bloom’s Original
Taxonomy
Revised Taxonomy
Separate dimension
Knowledge
Remembering
Comprehension
Understanding
Application
Apply
Analysis
Analyze
Synthesis
Evaluation
C
O
G
N
I
T
I
V
E
Evaluate
Create
Used by permission by WestEd for presentations
within the Northern Humboldt District
WestEd.org
49
Categories of the Cognitive Processes
There are 6 categories of the Cognitive
Processes.
Remember
Understand
Apply
Analyze
Evaluate
Create
Used by permission by WestEd for presentations
within the Northern Humboldt District
WestEd.org
50
Remembering means to retrieve
relevant knowledge from longterm memory and is essential for
meaningful learning and problem
solving.
1. Recognizing
2. Recalling
Used by permission by WestEd for presentations
within the Northern Humboldt District
WestEd.org
51
Understand means to construct
meaning from instructional messages,
including oral, written, and graphic
communication.
1. Interpreting
2. Exemplifying
3. Classifying
4. Summarizing
5. Inferring
6. Comparing
7. Explaining
WestEd.org
52
Apply involves using procedures to
perform exercises or solve problems.
1. Executing
2. Implementing
Used by permission by WestEd for presentations
within the Northern Humboldt District
WestEd.org
53
Analyze involves breaking material
into its constituent parts and
determining how the parts are related
to one another and to an overall
structure.
1. Differentiating
2. Organizing
3. Attributing
Used by permission by WestEd for presentations
within the Northern Humboldt District
WestEd.org
54
Evaluate is defined as making judgments
based on clearly defined criteria and
standards. The criteria most often used
are quality, effectiveness, efficiency, and
consistency.
1. Checking
2. Critiquing
Used by permission by WestEd for presentations
within the Northern Humboldt District
WestEd.org
55
Create involves putting elements together to
form a coherent of functional whole. In other
words, drawing upon elements from many
sources and putting them together into a
structure or pattern relative to one’s own
prior knowledge. (It may or may not include
originality or uniqueness.)
1. Generating
2. Planning
3. Producing
Used by permission by WestEd for presentations
within the Northern Humboldt District
WestEd.org
56
Used by permission by WestEd for presentations
within the Northern Humboldt District
WestEd.org
57
Practice determining Bloom’s
R:LT:4:2.2: Analyze and interpret elements of literary texts,
citing evidence where appropriate bydescribing main characters’ physical characteristics or
personality traits; or providing examples of thoughts, words,
or actions that reveal characters’ personality traits
Using the standard below, determine the Knowledge and
Cognitive Process boxes to check on the graphic organizer.
Using the Taxonomy Table on page ___ of the handout,
determine how you would check boxes for this standard.
Used by permission by WestEd for presentations
within the Northern Humboldt District
WestEd.org
58
• Norman Webb’s Depth of
Knowledge Levels (1997, 2003)
• Descriptive not a taxonomy
• Focuses on how deeply a student
has to know the content in order
to respond
• Interpreting and assigning depth
of knowledge levels to content
standards Kentucky Department of
Used by permission by WestEd for presentations
within the Northern Humboldt District
Education
WestEd.org
59
Used by permission by WestEd for presentations
within the Northern Humboldt District
WestEd.org
60
• Focus is on specific facts,
definitions, details, or using
routine procedures
Used by permission by WestEd for presentations
within the Northern Humboldt District
Kentucky Department of
Education
WestEd.org
61
• Focus is on applying skills (in a familiar/typical
situation) and concepts, relationships (compare,
cause-effect), main ideas – requires deeper
knowledge than definition; explaining how or
why; making decisions – estimating, interpreting
in order to respond; one right answer
Kentucky Department of
Education
Used by permission by WestEd for presentations
within the Northern Humboldt District
WestEd.org
62
• Focus is on need for reasoning and planning in
order to respond (e.g., write an essay, apply in
new/novel situation);
complex and abstract thinking is required;
often need to provide support for reasoning or
conclusions drawn;
more that one “correct” response or
approach is often possible
Used by permission by WestEd for presentations
within the Northern Humboldt District
Kentucky Department of
Education
WestEd.org
63
• Requires complex reasoning, planning, and
thinking generally over extended periods of
time for the investigation or to complete the
multiple steps of the assessment item.
Students may be asked to relate concepts
within the content area and among other
content areas or to real-world applications
in new situations.
Used by permission by WestEd for presentations
within the Northern Humboldt District
Kentucky Department of
Education
WestEd.org
64
• Handout - Ceiling Levels
of DOK
• Read and highlight key
points and phrases for
each DOK level
Used by permission by WestEd for presentations
within the Northern Humboldt District
WestEd.org
65
Handout
• Kentucky Released
Items
• We will work in groups
of 3 – 4
• Choose the DOK level
for your question and
write your reason(s) for
choosing the level
Used by permission by WestEd for
presentations within the Northern Humboldt
District
WestEd.org
66
Used by permission by WestEd for presentations
within the Northern Humboldt District
WestEd.org
67
Check the appropriate box(s) on the Webb’s
DOK on the Graphic Organizer.
R:LT:4:2.2: Analyze and interpret elements of literary
texts, citing evidence where appropriate bydescribing main characters’ physical characteristics
or personality traits; or providing examples of
thoughts, words, or actions that reveal characters’
personality traits
Used by permission by WestEd for presentations
within the Northern Humboldt District
WestEd.org
68
Closure
Look back at your “Unpacked Standards”,
ask yourself:
1.
Could you put away the original standard(s)confidently, plan instruction and
assessment using only your graphic organizer, knowing that you have
faithfully captured every important concept and skill students need to learn?
2. Would other educators identify the same
concepts and skills?
If the answer to both questions is yes, congratulations!
You have successfully “Unpacked a Standard”!
Used by permission by WestEd for presentations
within the Northern Humboldt District
WestEd.org
69
Two Buck Summary
On your white board,
please write a two buck
summary for unpacking
the standard. Each
word is worth 10 cents.
WestEd.org
Where to Find the Standards…
Content Standards:
http://www.cde.ca.gov/be/st/ss/
Curriculum Frameworks:
http://www.cde.ca.gov/be/st/fr/
Common Core Standards:
http://www.corestandards.org/the-standards
WestEd.org
Skills for Teachers to Implement
Standards-based Instruction
Turn to page 31 – Module 2
Alternately read sections of this article to each other
following along while someone else is reading.
Highlight or underline sections that resonate for you.
WestEd.org
72
Developing Benchmark Assessments
Read page 33 on your own.
As you encounter sections you….
Understand, mark with a check,
Question…. Mark with a ?
Think important…… Mark with a +
When you are done, read page 33 and discuss with
your neighbor.
WestEd.org
73
Flag Tag……
(Sorting activity)
 Find your Spring partners and join one other Spring




partner pair
Work in teams of four
Order the assessment questions from least difficult
to most difficult
Try to reach consensus as a group
Be prepared to defend your decision using your
understanding of Bloom’s taxonomy, Webb’s DOK
and other considerations.
WestEd.org
Why do we do this?
Two reasons….
Universal design….
Matching the appropriate level of
assessment items to the rigor and
‘cut scores’ we have identified.
WestEd.org
Performance Levels
• Turn to page 36 and 37
• Box 1 show the achievement distributions
comparing conventional, mastery and tutorial
approaches to learning and instruction.
• Box 2 notes the six key characteristics of
effective assessments.
•Box 3 shows that a comprehensive school
program is needed to inform practice and
program decisions leading to students
performing at the proficient levels.
WestEd.org
Performance Levels
The purpose of Benchmark and Formative Assessments is;
• to provide timely appropriate feedback to inform
instruction,
•to diagnose and move students up levels, and
• to find out what students have and have not learned at a
proficient level in time to intervene.
WestEd.org
Performance Levels
Why do we suggest 4 to 5 performance levels?
• Performance levels become the foundation for
scoring scales defined by the district.
• The performance levels help to group students
for targeted interventions for improvement.
• A purpose of assessment becomes the
identification of student performance to match
targeted instructional strategies and assist each
student in mastery of district benchmark and
state standards.
WestEd.org
Setting Performance Levels
These have often already been set by the DOE
with ATI Galileo for current benchmark/interim
assessments.
How through what process would you choose
different performance levels within the district?
WestEd.org
LUNCH
We will return in 60 minutes…
At……..
WestEd.org
Monache High School
Read pages 10 & 11
Write down any question that come up for you as you
read this story.
Cynthia Lee to share her experience as principal
involved in a similar effort.
WestEd.org
81
Test Construction - Universal Design
House of the Future, Disneyland
Architects, Richard Hamilton and Marvin Goody
1957-1967
WestEd.org
Creating Assessments - Guidelines Packet
Local Accountability Professional Development Series
Learning Innovations at WestEd
WestEd.org
Taking a test - Franzipanics
Individually, take the test on page
47, then join with a partner if you
finish early and discuss what you
think are the correct answers.
Hint: There is one correct answer for
each question.
Why did we take this test?
WestEd.org
Developing Quality Classroom Assessments
Read pages 51 & 52
Discuss at your table the questions on page 53.
We will discuss as a whole group the need for rubrics
when utilizing performance based assessments.
(See page 46)
WestEd.org
85
Framing of Day 2 -
End of Day 1
We will start off tomorrow using a rubric to evaluate some
writing samples. (Performance task)
You will also be creating new and/or evaluating and
modifying existing assessments you already use. (Galileo
allows you to do this). Bring standards, materials and
assessments that you would like to work with.
Please complete today’s evaluation before leaving.
Thanks!
WestEd.org
86
WestEd.org
8th Grade English and Writing
(performance) Assessment
 Read a student writing sample and using the 6
Traits Rubric, score the sample as best you can in
all areas on the scale of 1 to 5.
Compare your results with a partner
If you have time, look at an additional sample
and go through the same procedure of scoring
the sample by yourself, and comparing your
results to your partner’s findings.
Be prepared to discuss as a whole group.
See p. 46 – Building Performance Assessments
WestEd.org
89
WestEd.org
WestEd.org
WestEd.org
WestEd.org
WestEd.org
Taking a test - Franzipanics
Individually, take the test on page
47, then join with a partner if you
finish early and discuss what you
think are the correct answers.
Hint: There is one correct answer for
each question.
Why did we take this test?
WestEd.org
Developing Quality Classroom Assessments
Read pages 51 & Skim 52
As you read, underline or highlight key quotes,
points, or phrases.
Discuss at your table the questions on page 53.
WestEd.org
97
Local Accountability (Return to your seats)
 Read Element 3, “Assessments” on pages 5-7.
 As you read, underline or highlight key quotes,
points, or phrases.
WestEd.org
98
Formative Assessment – Making it
happen in the classroom
 Read and ✔ (understand), ? (question) or +
(important).
When you are done, discuss with your summer
partner.
Be prepared to discuss as a whole group.
100
WestEd.org
Ms. Toliver’s Classroom
 Review the Group 1, 3, and 4 questions on the
handout.
As you watch the video…. Take notes on the
different formal and informal assessment
practices you are observing in her classroom
When you are done, discuss with a neighbor.
Be prepared to discuss as a whole group.
101
WestEd.org
WestEd.org
WestEd.org
Work Period…..
Start by ‘unpacking’ the standard(s) you will work
with…
Continue work on Student Friendly Language….
Begin work creating or reviewing assessments that
will inform your new accountability system….
WestEd.org
Grant Information and
updates from Bob
11:15 to Noon
WestEd.org
LUNCH
We will return in 60 minutes…
At……..
WestEd.org
Work Period Continued…..
Continue work on Student Friendly Language….
Begin work creating or reviewing assessments that
will inform your new accountability system….
WestEd.org
End of Day 2
Final Questions……
Post Assessment Workshop Assessment.
Please complete today’s evaluation before leaving.
Thanks!
110
WestEd.org
Thank you!
If you have any questions, please contact:
Steve Hamilton
Tel: 781.481.1104
shamilt@wested.org
WestEd.org