Designing Electronic Homework for
Learning First, and Assesment
Second
Bill Vining
Beatrice Botch
Roberta Day
SUNY College at Oneonta
UMass Amherst
UMass Amherst
Development of the OWL
Electronic Homework System
Philosophy
Needs
How it all Works
Philosophy
Exams and Quizzes are for
evaluation.
Homework is for learning.
Needs to Support Learning
• Graded Homework questions
• No “penalty” for trying
• Feedback on how to answer each question
• A way to ensure that sometime, eventually,
they really get it right on their own
• Extra Support
– Teaching materials aimed at concepts
– Detailed, step-by-step help for the more
troubled among us
Needs to Support Teaching
• Comprehensive coverage
• Strong accuracy and pedagogical vetting
• Flexibility
• Ease of use:
– Assigning work
– Obtaining grades
– Managing Students
• It needs to work
Needs to Support Learning
• Graded Homework questions
• No “penalty” for trying
• Feedback on how to answer each question
• A way to ensure that sometime, eventually,
they really get it right on their own
• Extra Support
– Teaching materials aimed at concepts
– Detailed, step-by-step help for the more
troubled among us
Homework Question
The No-Penalty
Mastery Method
Answer
Correct Answer
and Feedback
The Necessity of Parameterization:
incorrect
correct
Mastery Credit
Three Dimensions of Parameterization
Numbers
Chemistry
Questions
Support I: Question-Specific Feedback
Support II: Additional Help for
Concepts and Problem Solving
Concept Simulations and
Exercises
Homework Question
Step-by-step
tutorial
Answer
Correct Answer
and Feedback
incorrect
correct
Mastery Credit
Newly
generated
question
Tutorials: Numerical
Tutorials: Graphical
Concept Simulations
Multimedia Exercises
Support III: Integration of Materials
Concept Simulations and
Exercises
Textbook and
Learning Resources
Homework Question
Step-by-step
tutorial
Answer
Correct Answer
and Feedback
incorrect
correct
Mastery Credit
Newly
generated
question
Support III: Integration of Materials
Organization: Chapter Folders
Organization: Assignments
Organization: Units
Organization: Questions
Instructor Control
• Assignment Dates/ Retroactively Changeable
• Individual Extensions
• Scoring Cutoffs/Number of Tries
• Show Answers/Feedback
• Full Grade and Reporting Functions
• Full Question Editing and Authoring
OWL: The Current State
• Courses:
–
–
–
–
–
–
General Chemistry
Organic Chemistry
GOB
Introductory
Liberal Arts
Gen Chem and Organic “Prep”
• Users
> 300 schools
> 75,000 students
Over 500,000 student users to date
OWL: The Current State
Content: General Chemistry
> 600 Assignments
> 1200 Units
~ 700 Tutors
~ 100 Simulations
~ 150 Multimedia Exercises
Including 20-40 Text End-of-ChapterSpecific Questions per Chapter.
OWL: The Current State
Evaluation Studies
Multiple Studies, None Perfect, General
Conclusions:
Can raise grades by ~ letter grade
Most effective for C/D students
Question Repeation/Feedback is
essential
Electronic Homework Hints
and Tips
Be a believer, even if you’re not
Assign early, assign often
Assign intro- how to assignments
Put some slack in the system
The Content/Software Team
Chemistry
UMass CCBIT OWL Creators
Roberta Day
Beatrice Botch
Bill Vining
Barbara Stewart
Peter Lillya
Steve Hixson
Susan Young
Steve Battisti
Cindy Stein
Dave Hart
Thomson Learning
Cowtown Productions
Eric Mulvihill
Jesse Charette
Bill Rohan
Lisa Lockwood, David
Harris, Michelle Julet,
Gayle Huntress, Amee
Mosely, Lisa Weber
Electronic Book