HOT
Assessment Practices
P. Gibson Ralph
Department of Theatre &
Music Studies
Credit Where Credit Is Due
• Plenary Speaker 2014 STLHE Conference
• “Why you can pass tests and still fail in the
real world
• Eric Mazur
– Balkanski Professor of Physics and Applied Physics
at Harvard University
– Area Dean of Applied Physics
• Thesis - Assessment fails to focus on the skills
that are relevant in life in the 21st century.
Flash Cards – Rote Memory
35% Retained
after
1 week
• 21% retained after two weeks
• 18% retained after three weeks
• 17% retained after two months
Purpose of Assessment
•
•
•
•
•
•
•
Rate students
Rate professor and course
Motivate students to keep up with work
Provide feedback on learning to students
Provide feedback to instructor
Provide instructional accountability
Improve teaching and learning
High Order Thinking
HOT
Bloom’s Revised 21st Century Taxonomy
Problem Solving
• Most problems are low order thinking skills
– Given - a problem
– Procedure for solving known
– Outcome is unknown
• A computer can do this
Authentic Assessment/Testing
• Real world tasks
– 21st Century approach
– Computers & robots handle manufacturing
• The outcome is know
• The process or solution to the problem is
unknown
– Humans solve problems
So what’s the problem?
Grading is
incompatible with
REAL
problem solving
Creating Authentic Assessment
• Any assessment question whose solution is
available on Google is not Authentic
• Problem with known outcome & unknown
solution
• The HOT process
– Creating/ Evaluating/ Analyzing
• Make assumptions
• Develop a Model
– Applying the model
Mazur Example
How Long a Wait?
• Assumptions
– People shop for approximately two hours
– Assume people leave at regularly spaced intervals
• Model
– Wait time = shopping time/number of spaces
• Application of the model
– If average shopping time is 2 hours
– Number of spaces is 20
– Wait time = 120minutes/20spaces
Second Impediment
YOU DIDN’T
COVER THIS IN
CLASS!!!!
Avoid the LOT Temptation
• Asked to remove assumptions and provide
data
• Asked to provide formula
Remembering
Ralph What If’s
• Dramatic literature
– If Racine had written Hamlet how would it be
different.
• Scenic Design
– Equipped with Sabatini’s Pratica di fabricar scene
e machine ne' teatri and $400 produce an effect to
revel Banquo’s ghost in Hamlet.
Grading Options
• External review
• Peer and self assessment
• Calibrated Peer Review
– http://cpr.molsci.ucla.edu
Possible Assessment Approaches
• Imitate real life
– Open book
– Immediate Feedback Assessment Technique
• Focus on feedback not ranking
• Focus on skills not content
Future – Perhaps
Reexamine
The
Course
Backward Design
• Begin with Desired Outcomes
• Determine what is Acceptable Evidence
• Select the Instructional Approach
Understanding by Design: Grant P. Wiggins, Jay McTighe 2005