A Closer Look at the
SMARTER Balanced Assessments:
Secondary Mathematics
Cheryl Avalos
Avalos_Cheryl@lacoe.edu
Los Angeles County Office of Education
Agenda:
• Item Types
• Depth of Knowledge
• Technology
• Next Steps
Try the Problem
• Sale Prices – Grade 7
• The Envelope – Grade 11
Item Design Considerations
• The Four Claims
• SBAC screened standards for on-demand
measurability and Depth of Knowledge (DOK)
–Level 1 (Recall)
–Level 2 (Basic Skills and Concepts)
–Level 3 (Strategic Thinking and
Reasoning)
–Level 4 (Extended Thinking)
4
Item and Test Design
5
Item Types
•
•
•
•
•
Short Answer
Constructed-response
Extended constructed-response
Performance Events
Technology-enhanced
6
Short Answer Items
• Used to cover a broad range of content.
• Assess defined problems with limited
solutions
• Should take relatively little time to answer
• Efficient to score
Extended Response
– Assess knowledge and skills not easily
accessed with selected response or constructed
response (e.g., oral presentations, exhibitions,
product development, extended written
response).
– Students are often asked to justify answers.
– Require 10-20 minutes to complete.
– Require careful scorer training and calibration.
Performance Task Items
• Collections of questions and real-world tasks connected
to a single theme or scenario:
– A student may be presented with reading material on
a given topic, and asked questions about the reading
– That may be followed by reading on the same topic,
but from a different perspective, with questions on
that reading, plus some compare/contrast questions
– Finally, the student may be provided additional
readings on the same, or an associated topic, and
asked to take a position or provide an argument
– Each performance task could take place across more
than one sitting of 45-60 minutes each
9
Sample Assessment Items
Mathematics Draft Content Specifications
http://www.k12.wa.us/SMARTER/Resources.aspx
Mars Tasks
http://www.insidemathematics.org/index.php/tools-forteachers/7th-grade-math/mars-tasks-scoring-rubrics-aanalysis
10
Teaching with Depth
An Understanding of
Webb’s Depth of Knowledge
Cognitive Demand
What is Depth
of Knowledge (DOK)?
• A scale of cognitive demand (thinking) to align
standards with assessments
• Based on the research of Norman Webb,
University of Wisconsin Center for Education
Research and the National Institute for Science
Education
Webb’s Four Levels of
Cognitive Complexity
•
•
•
•
Level 1: Recall and Reproduction
Level 2: Skills & Concepts
Level 3: Strategic Thinking
Level 4: Extended Thinking
DOK Level 1:
Recall and Reproduction
• Requires recall of information, such as
a fact, definition, term, or performance of
a simple process or procedure
• Answering a Level 1 item can involve
following a simple, well-known procedure
or formula
Recall and Reproduction DOK Level 1
Examples:
• Calculations
• Determine the perimeter or area of
rectangles given a drawing or labels
Depth of
Knowledge
Skills/Concepts: DOK Level 2
• Includes the engagement of some mental
processing beyond recalling or reproducing a
response
• Items require students to make some
decisions as to how to approach the question
or problem
• Actions imply more than one mental or
cognitive process/step
Constructed Response
– Complex thinking skills, comparison and contrast,
cause and effect, patterns, conflicting points of view,
categorizing, summarizing, interpreting information
– Time required is 5 to 10 minutes.
– Scored using a rubric
– Scoring training and calibration are advised
Skills/Concepts: DOK 2 Examples
• Classify plane and three dimensional figures
• Graphing linear equations
• Solving word problems
Strategic Thinking: Level 3
• Requires deep understanding exhibited through
planning, using evidence, and more demanding
cognitive reasoning
• The cognitive demands are complex and
abstract
• An assessment item that has more than one
possible answer and requires students to justify
the response would most likely be a Level 3
DOK Level 3: Strategic Thinking
Examples:
Solve a multiple-step problem and provide support with
a mathematical explanation that justifies the answer
First problem that we did today was identified as
Level 3
Extended Thinking: Level 4
• Requires high cognitive demand and is very complex
• Students are expected to make connections, relate
ideas within the content or among content areas, and
select or devise one approach among many alternatives
on how the situation can be solved
• Due to the complexity of cognitive demand, DOK 4
often requires an extended period of time
Extended Thinking: DOK 4
Examples
• Gather, analyze, organize, and
interpret information from multiple
(print and non print) sources to draft
a reasoned report
•
A Word of Caution
Verbs are not always used
appropriately...
DOK is about what follows the
verb... What comes after the verb
is more important than the verb
itself.
Same Verb—Three Different DOK Levels
DOK 1- Describe three characteristics of two quadrilaterals.
(Requires simple recall)
DOK 2- Describe the difference between convex and
concave polygons. (Requires cognitive processing to
determine the differences in the two polygon types)
DOK 3- Describe a model that you might use to represent the
relationships that exist within the set of polygons. (Requires
deep understanding of polygons and a determination of
how best to represent it)
DOK is not about difficulty
For Example…
• Adding is a mental process.
• Knowing the rule for adding is the intended outcome that
influences the DOK.
• Once someone learns the “rule” of how to add, 4 + 4 is
DOK 1 and is also easy.
• Adding 4,678,895 + 9,578,885 is still a DOK 1 but may
be more “difficult.”
Technology
Technology
33
Technology Required for CCSS
– Testing is completed through computer.
See sample problems.
– Computer Adaptive Testing
What is Computer
Adaptive Testing (CAT)?
• Adapts to the student’s ability level
• Assessment “engine” delivers short series of
moderately difficult grade items
• Depending on student’s initial performance delivers
items that are either more or less difficult
• Process continues until student’s performance level
is determined.
35
Benefits of Computer
Adaptive Testing (CAT)
Faster results
•Turnaround in weeks compared to months today
Shorter test
length
•Fewer questions compared to fixed form tests
Increased
precision
Tailored to
student ability
Greater
security
Mature
technology
•Provides accurate measurements of student growth over
time
•
Item difficulty based on student responses
•Larger item banks mean that not all students receive the
same questions
•GMAT, GRE, COMPASS (ACT), Measures of Academic Progress
(MAP)
36
Paper and Pencil Option
• Will there be a paper and pencil option?
• It is likely that a paper and pencil option will be
provided initially. This decision is made at the
state level. There will an additional cost at the
district level to select this option as there are
inherent costs such as printing, shipping, scoring,
etc.
37
Tablets vs. Computers
• There are current challenges to test item
security for online assessment with tablets, ereaders, and smartphones.
• Testing software used by computers typically
shuts down keyboard commands, such as “print
screen” and “copy,” to ensure test security. Tablet
applications don’t have this capability.
• SBAC will conduct further development and
testing.
38
*With media taking a prominent role in both the gathering of
information and the presentation of new knowledge products, it is
recommended that schools be equipped with:
•
•
•
•
•
•
projection/presentation stations
internet connectivity
media creation stations
writing centers
internet stations to support distance learning course offerings
video and still cameras with computer software for media
production
***Recommendations from the NYLA (New York Librarian Association)
IT Technology
Readiness Tool
• Online dynamic and interactive
technology readiness tool
• Helps SEAs and LEAs evaluate
current technology and
infrastructure in terms of SBAC
implementation readiness
• It is anticipated tool will be ready
in spring 2012
• NCS Pearson will develop
40
By embracing technology and encouraging
students to explore its various forms,
educators also have a responsibility to teach
students to use technology in a responsible
manner.
2 Things To Remember…
• Digital Citizenship will be constantly
changing with the times as new tools are
developed and used.
• As within any society, we need a structure
which people need to honor so that we
are respectful to each other.
How Do We Start Implementing Digital
Citizenship In Our Classrooms and Where Can
We Find Support ?
 http://cybercitizenship.ning.com
 http://www.onguardonline.gov
 http://GetNetWise.org
 http://CommonSenseMedia.org
 http://iKeepSafe.org - Parents
 http://NetSmarts.org
 http://WiredSafety.org
Equity Issues
•
•
•
•
Availability of Technology
Accessibility of Technology
Integrating it into the Curriculum
Teacher knowledge and comfort
What can we do now?
• Integrating new problem types.
– POD, POW, POM
• Assigning projects
• Implementing the Eight Mathematical Practices
– Inside Mathematics
– MARS Problems
• Ask for justifications
Technology in the Classroom
• Banana Hunt (cool-mathgames.com)
• Adopted materials
• Geogebra (geogebra.org)
Resources
• Full text of the Common Core California Standards:
http://www.scoe.net/castandards/index.html (Outside
Source)
• Information about the common core:
http://www.corestandards.org/ (Outside Source)
• Information about the common core including
implementation timelines: http://www.cde.ca.gov/ci/cc/
• SBAC information: www.k12.wa.us/SMARTER (Outside
Source)
• Center for K-12 and Performance Management at ETS:
http://www.k12center.org/publications.html (Outside Source)
• SBAC FAQ:
http://www.cde.ca.gov/ta/tg/sa/documents/qasbac11a12.pdf