Assessing
Computational
Thinking
Eric Snow
Center for Technology in Learning
SRI International
Sangeeta Bhatnagar
Evergreen Valley High School
Overview
• What is computational thinking (CT)?
• Key CS/CT assessment efforts to date.
• Current CT Assessment Work at SRI
International:
• Planning for the Assessment of Computational
Thinking (PACT 1)
• ECS-Standards Alignment
• Principled Assessment of Computational Thinking
(PACT 2)
What is Computational Thinking?
• Popularized by Jeanette Wing in ACM Viewpoints Article
(2006)
• “Universally applicable attitude and skill set everyone, not just
computer scientists, would be eager to learn and use.”
• Characteristics of CT:
•
•
•
•
Conceptualizing, not (just) programming
A way that humans, not computers, think
Ideas, not (just) artifacts
Draws on existing conceptions of mathematical, engineering
(systems), and inquiry-based thinking
• Not just for computer science/scientists
Key CS/CT Assessment Efforts to
Date
• CS Principles Computational Thinking Practices (College
Board, 2011)
•
•
•
•
•
•
•
Connecting computing
Developing computational artifacts
Abstracting
Analyzing problems and artifacts
Communicating
Working effectively in teams
Aligned with new CS AP principles course
• CS Concept Inventory (Herman, Loui, & Zilles, 2010)
• FCS1 Assessment (Elliott Tew, 2010)
• SUNY CS1-2 Exam (Decker, 2007)
Planning for the Assessment of
Computational Thinking (PACT 1)
NSF-CE21 Planning Grant (9/2011 – 6/2012)
Principal Investigators: Eric Snow, Marie Bienkowski
Project Goals
Advance field of assessment in computer science and
computational thinking.
Leverage interdisciplinary expertise to begin creating an
assessment framework and design templates for
computational thinking (CT) aligned with the Exploring
Computer Science curriculum.
Planning for the Assessment of
Computational Thinking (PACT 1)
Project Plan
December 2011
Core expert group meets to begin creating an assessment framework and
design templates for computational thinking (CT) aligned with the
Exploring Computer Science curriculum
January 2012
Debrief core expert group meeting and circulate draft framework and
design templates for internal review. Develop project dissemination web
site. Begin planning for next stage of CT assessment design and
development work
February – September 2012
Release project dissemination website, draft standards alignment
document, invite CE21 community feedback on assessment framework,
design templates and standards alignment work. Plan for next stage of CT
assessment design and development work
Focal KSAs form the core of design templates that specify what we want
to measure, the behaviors that will count as evidence of what we want to
measure, and the types of tasks we can use to elicit the desired behaviors.
Planning for the Assessment of
Computational Thinking (PACT 1)
ECS - CS/CT Standards Alignment
• Objectives
• Gain a better understanding of how ECS CT practices and learning
objectives were related to relevant national and state standards
(e.g., CSTA, Common Core).
• Provide value to ECS team, especially teachers and others who
had to convince stakeholders that ECS helps students learn
valuable knowledge and skills
• Worked with local ECS teacher to develop draft document
that shows the alignment between ECS Unit learning
objectives and CT practices, and CSTA, ISTE, Common Core,
Next Generation Science/Engineering, and CA and IL state
learning and CTE standards.
• Currently under review/revision
ECS - CS/CT Standards Alignment
Planning for the Assessment of
Computational Thinking (PACT 1)
Web Site URL:
http://pact.sri.com
Please note:
• The site is still under construction.
• PACT materials will continue to be posted over coming
months. Please email me if you would like to get an update
on when materials are posted.
• Feel free to send comments and feedback.
Principled Assessment of Computational
Thinking (PACT 2)
NSF-CE21 Special Project (9/2012 – 8/2014)
Principal Investigators: Eric Snow, Marie Bienkowski
*proposal currently “Recommended” for funding
Project Goals
Advance field of assessment in computer science and
computational thinking.
Leverage materials and partnerships developed under PACT
1 to design, develop and validate assessments of
computational thinking for the ECS curriculum and,
eventually for the larger CT community.
Principled Assessment of Computational
Thinking (PACT 2)
Year 1
• Extend and finalize domain analysis/modeling started
in PACT 1; focus on ECS units 1-4, possibly unit 5 (“big
data”).
• Develop pilot assessment tasks for unit and
summative assessments
• Expert panel review of pilot tasks
• Pilot summative assessment (include subset of unit
tasks), including think-aloud interviews
• Data analysis and assessment task revisions
• Advisory panel
Principled Assessment of Computational
Thinking (PACT 2)
Year 2
• Expert panel review of summative and unit
assessment tasks
• Field test summative assessment, pilot unit
assessments, including think-aloud interviews
• Data analysis and final assessment task revisions
• Advisory panel
Methodological approach focused on assessment
design/development, and on collecting validity
evidence based on test content, internal structure,
response processes, and relations to other variables.
Computational Thinking Resources
• National Research Council. (2004). Computer science:
Reflections on the field. Washington, DC: The National
Academies Press.
• National Research Council Committee on the Workshops on
Computational Thinking. (2010). Report of a workshop on the
scope and nature of computational thinking. Washington, DC:
National Academy Press.
• National Research Council Committee on the Workshops on
Computational Thinking. (2010). Report of a workshop on the
pedagogical aspects of computational thinking. Washington,
DC: National Academy Press.
• CS Standards from International Society for Technology in
Education (ISTE) and CS Teacher’s Association (CSTA) (2011)
THANK YOU!