Measuring What Matters:
Technology & the
Assessment of all Students
Jim Pellegrino
Discussion Topics

Assessment and the National Educational
Technology Plan

Research & Conceptual Foundations

Towards a New Generation of TechnologyEnabled Assessments

Some Final Thoughts About Technology
& the Assessment of All Students
NETP: What is it?



Response to Congressional
mandate for five-year plan for
educational uses of
technology
Plan for transforming
education with technology in
response to urgent need to
remain competitive in a global
economy
Reflection of increased
understanding of how to
support learning and of
growing capabilities enabled
by technology
The Five Goals
Teaching
Assessment
Infrastructure
Productivity
Learning
Assessment





Measure what matters
Improve learning in the
moment; don’t just
measure it after the fact
Use technology to
measure complex
competencies
Create electronic
learning records, similar
to electronic medical
records
Aggregate and analyze
data on a system level
Discussion Topics

Assessment and the National Educational
Technology Plan

Research & Conceptual Foundations

Towards a New Generation of TechnologyEnabled Assessments

Some Final Thoughts About Technology
& the Assessment of All Students
QuickTime™ and a
TIFF (LZW) decompressor
are needed to see this picture.
Assessment as a Process of
Reasoning from Evidence



cognition
 theory or model of how
students represent knowledge
& develop competence in the
domain
observations
 tasks or situations that allow
one to observe students’
performance
interpretation
 method for making sense of
the data
observation
interpretation
cognition
Must be
coordinated!
Discussion Topics

Assessment and the National Educational
Technology Plan

Research & Conceptual Foundations

Towards a New Generation of TechnologyEnabled Assessments

Some Final Thoughts About Technology
& the Assessment of All Students
Advantages of Technology for
Science, Math & Literacy Assessment










Present authentic, rich, dynamic, interactive environments
Support access to collections of information, expertise
Present phenomena difficult or impossible to observe and
manipulate in classrooms
Represent temporal, causal, dynamic relationships “in action”
Allow multiple representations of stimuli and their
interactions (e.g., data generated during a process)
Allow overlays of representations, symbols, natural language
Allow student manipulations/investigations, multiple trials
Allow student control of pacing, replay, reiterate
Capture student responses during research, design, problem
solving
Allow use or simulations of a range of tools and supports
(internet, productivity, domain-based)
Life Science Simulation
In the experiment that you just analyzed, the amount of alewife was set to 20 at the beginning.
Another student hypothesized that the result might be very different if she started with a larger
or smaller amount of alewife at the beginning.
Run three experiments to test that hypothesis. At the end of each experiment record your data
by taking pictures of the resulting graphs.
After three runs, you will be shown your results and asked if it makes any difference if the
beginning amount of alewife is larger or smaller than 20.
Examples of Formative
Assessment with Feedback
•
•
•
•
Diagnoser
ASSISTments
SimScientists
(FACET Innovations, LLC)
(Worcester Polytechnic Institute)
(WestEd)
Science
Grades 6-12
Web-based
Diagnoses
“Facets” of
student
understanding
• Provides counter
examples and
additional
lessons
•
•
•
•
Math
Grades 4-10
Web-based
Tutors students
through
assessment
items
• Reports to
teachers on 90
skills
•
•
•
•
Science
Grades 6-8
Web-based
Simulations of
science
phenomena
used to assess
students
• Reports to
students and
teachers
Features and Advantages
Formative
Multimedia
Scoring
Reporting
• Diagnostic
• Feedback and Coaching
• Animations/Video
• Simulations
• Can show things impossible in P&P
• Can assess skills like inquiry
• Natural Language Processing
• Immediate
• Detailed summative reports
• Reports for instruction
Discussion Topics

Assessment and the National Educational
Technology Plan

Research & Conceptual Foundations

Towards a New Generation of TechnologyEnabled Assessments

Some Final Thoughts About Technology
& the Assessment of All Students
NETP Assessment Goal
Our education system at all levels will leverage the
power of technology to measure what matters and use
assessment data for continuous improvement.
o Design, develop, and adopt assessments that give students,
educators, and other stakeholders timely and actionable feedback about
student learning to improve achievement and instructional practices.
o Build the capacity of educators and educational institutions to use
technology to improve assessment materials and processes for both
formative and summative uses.
o Conduct research and development that explores how gaming
technology, simulations, collaboration environments, and virtual worlds
can be used in assessments to engage and motivate learners and to
assess complex skills and performances embedded in standards.
o Revise practices, policies, and regulations to ensure privacy and
information protection while enabling a model of assessment that
includes ongoing student learning data gathering and sharing for
continuous improvement.