Calibrated Peer Review™ (CPR):
Using Technology to Get the Most
Out of a Writing Assignment
Professor Laurie S. Starkey
Chemistry Dept., Cal Poly Pomona
Calibrated Peer Review™
What is CPR?
• Web-based tool which enables peerreviewed writing assignments
• Students, faculty log in with password at
http://cpr.molsci.ucla.edu
to access assignments
• Use and access are currently free; servers
are maintained by UCLA
2
CPR Assignment Components
Stage 1: Text Entry Stage
• Essay assignment is given online
– Links to supporting materials, relevant web
sites, animations, etc., are provided
– Student Instructions, Guiding Questions and
Writing Prompt are also available
– Essay must be formatted using simple HTML
tags (can Preview before submitting)
• Student submits essay online before text
entry deadline (CPR verifies word count)
3
CPR Stage 2: Calibration Stage
• Begins only after text entry deadline
• Student reads, evaluates 3 Calibration Essays
– faculty-generated essays are of varying quality
– Evaluation: Student answers 10-12 Calibration
Questions for each essay and assigns 1-10 score
• Student views Calibration results
– Do Student’s answers match Instructor’s answers?
– Feedback is provided for each Calibration Qn
– Student can retake Calibration after failing
• Student becomes “Expert” Reviewer
4
Sample Calibration Questions
• Content (accurate, complete)
– Does the essay conclude that “150.0 m” has four
significant figures?
– Does the essay identify George Washington as the
first president of the United States of America?
– Are there scientific errors in the essay, such as
incorrect definitions or misleading statements?
• Style (coherent, integrated, grammar)
– Does the essay have a descriptive topic sentence?
– Is the author’s argument easy to follow? (Y/N)
– Are there spelling errors? (None, Some, Many)
5
CPR Stage 3: Review Stage
• Begins after passing Calibrations Stage
• “Review”= Answer Questions, Score 1-10
• Student reviews three classmates’ essays
– Anonymous and randomly assigned
– “Classmates” may include the instructor, TA’s,
grad students, faculty, etc.
• Student performs a self-evaluation
(reviews his or her own work)
6
CPR Stage 4: Results Stage
• Did Student “master” review of peers?
– Student’s 1-10 rating is compared to average
– Instructor sets maximum allowable deviation
• Student views detailed reviews of his/her
own text
– Four reviews have been performed: three peer
reviews plus Self-Assessment
– Compare answers for each Calibration Question
– View feedback provided by peers
7
Computing CPR Assignment Grade
• Student’s Text Entry (30 points)
– Calculates weighted average score based on reviews
– Score of 10 = 30 pts; Score of 5 = 15 pts
• Student’s Calibrations (3 x 10 points each)
– Pass = 10 pts; Pass after retake = 5 pts; Fail = 0 pts
• Reviews performed by Student (3 x 10 points ea)
– Within allowable deviation = 10 pts; exceeds = 0 pts
• Student’s Self-Assessment (10 points)
– Within allowable deviation = 10 pts; exceeds = 0 pts
8
Advantages to using CPR
• Getting more out of a writing assignment
– Time on task: write, read 6 essays, self-reflect
– Evaluates and promotes reading
comprehension and critical thinking
– Ideal for content-heavy assignments
• Lots of feedback, little Instructor’s time
– Peers are trained to be reviewers
– Utilizes time outside of class
• Provides experience using HTML
9
Is CPR right for me?
• Ideal for online courses or source materials
• Assignment runs about 2 weeks
• Not suitable for every writing assignment
– Develop Calibration Qns with Y/N answers
• Need significant time for initial investment
(authoring a CPR assignment)
10
How do I begin using CPR?
• Library contains over 60 CPR assignments
– Chemistry, Biology, Biochemistry, Geology
• Attend FCSC workshops, Winter 2002
– Introduction to CPR
– Authoring CPR Assignments
• Attend UCLA workshop, July 17-19, 2002
11
Acknowledgments
• UCLA workshop, Summer 2001
– Arlene A. Russell, Krista Motschiedler Brand
• Cal Poly Faculty Computing Support Center
• Cal Poly Pomona Chemistry Department
12