A Closer Look at the
SMARTER Balanced Assessments:
Preparing for Elementary Mathematics
Anthony Quan
quan@gseis.ucla.edu
Teacher Education Program, UCLA
Agenda:




Item Types
Depth of Knowledge
Technology
Next Steps
Try This
Assessment Items
•
•
•
•
•
•
Multiple choice/Selected response
Short-answer
Constructed response
Extended response
Performance Events
Technology-enhanced
Depth of Knowledge
“He who learns but does not think, is
lost.
He who thinks, but does not learn is
in great danger.”
Confucious
Depth of Knowledge (DOK)
No Child Left Behind (NCLB) requires
assessments to “measure the depth
and breadth of the state academic
content standards for a given grade
level”.
(U.S. Department of Education, 2003, p. 12)
What is Depth
of Knowledge (DOK)?
• A scale of cognitive demand (thinking) to align
standards with assessments
• Based on the research of Norman Webb,
University of Wisconsin Center for Education
Research and the National Institute for Science
Education
• Defines the “ceiling” or highest DOK level for
each Core Content standard for the state
assessment
• Guides item development for state assessments
DOK is about complexity
• The intended student learning outcome
determines the DOK level.
• Every objective in the science and
mathematics frameworks has been assigned a
DOK level.
• Instruction and classroom assessments must
reflect the DOK level of the objective or
intended learning outcome.
Depth of Knowledge
• Level 1
• Conduct basic mathematical calculations
• Label locations
• Perform routine procedures
• RECALL
• Determine the perimeter or area of rectangles
given a drawing or labels
Depth of
Knowledge
Depth of Knowledge
• Level Two
• Solve routine multiple-step problems
• Identify patterns
• Organize, represent and interpret data
• SKILLS AND/OR CONCEPT
• Classify plane and three dimensional figures
Depth of Knowledge
• Level Three
• Apply a concept in other contents
• Support ideas with details and examples
• STRATEGIC THINKING
• Solve a multiple-step problem and provide support
with a mathematical explanation that justifies the
answer
Depth of Knowledge
• Level Four
• Apply mathematical model to illuminate a
problem or situation.
• Analyze and synthesize information from multiple
sources
• Design a mathematical model to inform and solve
a practical or abstract situation
• EXTENDED THINKING
Depth of Knowledge
• So what is the point of knowing Depth of
Knowledge?
• This helps us understand what the outcomes of
our lessons should be.
• We need to prepare students to think more
conceptually
Making Sense & Worthwhile Tasks
“What are our Kids really being asked to
do?”
“How are we keeping up with
Cognitive Demand?”
Why Depth of Knowledge?
Mechanism to ensure that the intent of the standard
and the level of student demonstration required by that
standard matches the assessment items (required under
NCLB)
To ensure that teachers are teaching
to a level that will promote student
achievement
DOK is
NOT...
•
Bloom’s taxonomy
•
the same as difficulty
•
about using “verbs”
DOK is about intended outcome,
not difficulty
DOK is a reference to the complexity of mental
processing that must occur to answer a question,
perform a task, or generate a product.
• Adding is a mental process.
• Knowing the rule for adding is the intended outcome
that influences the DOK.
• Once someone learns the “rule” of how to add, 4 +
4 is DOK 1 and is also easy.
• Adding 4,678,895 + 9,578,885 is still a DOK 1 but
may be more “difficult.”
Let’s look at DOK in Math
Activities
Remember DOK is...
…descriptive
…focuses on how deeply a
student has to know the
content in order to respond
…NOT the same as difficulty.
…NOT the same as Bloom’s
Taxonomy
What is Computer
Adaptive Testing (CAT)?
• Adapts to the student’s ability level
• Assessment “engine” delivers short series
of moderately difficult grade items
• Depending on student’s initial performance
delivers items that are either more or less
difficult
• Process continues until student’s
performance level is determined.
Benefits of Computer
Adaptive Testing (CAT)
Faster results
•Turnaround in weeks compared to months today
Shorter test
length
•Fewer questions compared to fixed form tests
Increased
precision
Tailored to
student ability
Greater
security
Mature
technology
•Provides accurate measurements of student growth over
time
•
Item difficulty based on student responses
•Larger item banks mean that not all students receive the
same questions
•GMAT, GRE, COMPASS (ACT), Measures of Academic Progress
(MAP)
27
Tablets vs. Computers
• There are current challenges to test item
security for online assessment with tablets, ereaders, and smartphones.
• Testing software used by computers typically
shuts down keyboard commands, such as “print
screen” and “copy,” to ensure test security. Tablet
applications don’t have this capability.
• SBAC will conduct further development and
testing.
IT Technology
Readiness Tool
• Online dynamic and interactive technology
readiness tool
• Helps SEAs and LEAs evaluate current
technology and infrastructure in terms of SBAC
implementation readiness
• It is anticipated tool will be ready in spring 2012
• NCS Pearson will develop
System Portal for
Online Delivery
• System Portal contains information about Common Core State
Standards, Consortium activities, web-based learning communities,
and assessment results
• Dashboard gives parents, students, practitioners, and
policymakers access to assessment information
• Reporting capabilities include static and dynamic reports, secure and
public views
• Item development and scoring application support educator
participation in assessment
• Feedback and evaluation mechanism provides surveys, open
feedback, and vetting of materials
Digital Etiquette
• Are there suggestions for preparing
technology enhanced responses?
• YES. Let’s take a look at what can be done.
*According to the Common Core, “The standards require that students gain,
evaluate, and present increasingly complex information, ideas, and evidence
through listening and speaking as well as through media.”
*With media taking a prominent role in both the gathering of information and the
presentation of new knowledge products, it is recommended that schools be
equipped with:
•
projection/presentation stations
•
internet connectivity
•
media creation stations
•
writing centers
•
internet stations to support distance learning course offerings
•
video and still cameras with computer software for media production
***Recommendations from the NYLA (New York Librarian Association)
By embracing technology and encouraging students to explore
its various forms, educators also have a
responsibility to teach students
to use technology
in a responsible manner.
2 Things To Remember…
Digital Citizenship will be constantly changing with the times as new
tools are developed and used.
As within any society, we need a structure which people need to
honor so that we are respectful to each other.
How Do We Start Implementing Digital
Citizenship In Our Classrooms and Where Can
We Find Support ?
•
•
•
•
•
•
•
http://cybercitizenship.ning.com
http://www.onguardonline.gov
http://GetNetWise.org
http://CommonSenseMedia.org
http://iKeepSafe.org - Parents
http://NetSmarts.org
http://WiredSafety.org
Technology in the Classroom
• Banana Hunt (cool-mathgames.com)
• Adopted materials
• Geogebra (geogebra.org)
Equity Issues
•
•
•
•
Availability of Technology
Accessibility of Technology
Integrating Technology into the curriculum
Teacher knowledge and comfort