PeerWise
Why not let them teach each other?
Gidi Shemer
Biology Department, UNC-Chapel Hill
Anatomy and Physiology ??
Active learning
Class-response system
PeerWise
Overview
- PeerWise: student and instructor
- How I use PeerWise in my class
- Student perception
100.00%
80.00%
60.00%
40.00%
20.00%
0.00%
Yes No
- Future research
- PeerWise in upper-level courses
Yes No
Fall 11
Spring 12
Qn. 1
Yes No
Yes No
Fall 11
Spring 12
Qn. 2
Yes No
Fall 11
Yes No
Spring
12
Qn. 3
PeerWise- What is it?
Students create multiple choice questions
and share them with peers
Students view, answer, and give feedback
to their peers
Asking questions and peer assessmentkeys to success in research… and learning
Dear students,
Your peerwise assignments for this section are:
1. To create at least one multiple choice question about the
muscular system. Recall questions are OK, but you are
encouraged to write a question that assesses a deep
understanding and/or application of the material
2. To answer at least 3 of your peers’ questions and to
provide them with a detailed feedback
Creating a question
http://peerwise.cs.auckland.ac.nz/at/?unc_edu
Reviewing questions
Reviewing questions
Reviewing questions
Providing an answer
Reviewing questions
Providing feedback
Instructor view
Student feedback
90.00%
80.00%
70.00%
60.00%
Survey (2 semesters,
351 students in total)
50.00%
40.00%
30.00%
20.00%
10.00%
0.00%
Yes No
Yes No
Fall 11
Spring 12
Qn. 1
Yes No
Yes No
Fall 11
Spring 12
Qn. 2
Yes No
Yes No
Fall 11
Spring 12
Qn. 3
Have the PeerWise activities increased your ability to find and
use resources for answering questions?
Have the PeerWise activities increased your ability to organize
and express new ideas and concepts?
Have the PeerWise activities increased your ability to put
together ideas and/or concepts from different sources?
Student feedback
Thinking about designing your own question, what was the most difficult part for you?
“It was most difficult to come up with original questions that used a combination of
information and were not straight out of the lectures”
"I think the most frustrating thing is when you think you know a subject
and when you have to write a question [and] you don't know it at all. You
have to go back and you have to look at other research. I could not
necessarily write a good question until I understood like every single
aspect of the PowerPoints on the point I would present."
Trying to develop a challenging question and answers that were not obviously wrong..
Peerwise was the best supplemental tool I have encountered in any science class so
far! seriously.
“Thinking of a question that Professor Shemer would ask.”
Student feedback
Sense of community and support with other students in and out of class...
"Yes because of Peerwise, we rely on each other for answers and not just for
answers, but we need explanations as well, so we rely on each other's
knowledge and we have to talk about things to be able to understand them."
“The most difficult part was knowing if other students questions and answers were
precise and correct. Peerwise helped me learn when I made my questions mostly.“
“I had trouble accepting other students' questions. VERY good way to learn have to really comprehend in order to make a question.”
PeerWise activity is correlated with
academic performance
Deny et al, 2008
Issues to consider
Should you give your input?
How many?
When?
Grading?
Small, upper-level classes
A great tool for journal clubs
So, why am I happy with PeerWise?
- The instructor =
- Students become more engaged
- Students develop critical thinking
- Students have a question bank with > 500 practice
questions
- You have a have a question bank with > 500 exam
questions
Acknowledgments
Center for Faculty Excellence
Andrea Reubens
Bob Henshaw
Biology
Joey Patterson
SWoRD
(Scaffolded Writing and Reviewing in the Discipline)
Peer Grading Program
http://www.pantherlearning.com/bl
og/sword/
Kelly A. Hogan
Department of Biology
UNC- Chapel Hill
Deadline 1
Deadline 2
Deadline 3
Students submit draft
Students review draft
• Each student reviews (3-6 other students)
Students “backevaluate” each of
their reviewers
• How helpful was the review?
Repeat if desired
Basic Flow of SWoRD
The % of reviews
and back-evaluations
the student
completed
The grade given by the
reviewers (with adjustments
to weight some reviewers
more than others if needed)
The grade combining:
1) how accurately a
student reviews papers
relative to others and
2) how helpful their
review was to the
student
Task
Grade
Writing
Grade
Overall
Student
score
Review
Grade
“SWoRD’s copyrighted
algorithms have been
validated over a decade of
Why use peer grading? Students:
• Practice the skill of writing.
• Learn how to evaluate writing by providing feedback to
their peers, which can improve the students’ ability to
evaluate their own writing (Wooley et al. 2008).
• Receive feedback (from peers), which allows them to
revise drafts more successfully (Cho and Schunn 2007).
• Improve their writing just as much as those who
received feedback from a single instructor (for a
review, Topping and Ehly 1998), and in some cases,
students improved more from peer feedback (Cho and
Schunn 2007).
Why I was drawn to SWoRD
• A new kind of assessment with no extra resources
(free*, no TAs needed for grading)
• Writing assignments = higher level thinking (e.g.
synthesis, evaluation)
• Opportunities for students to practice writing in
our large department is rare.
• Opportunity to shift final grade weight off exams
(Exams = 90% of grade in past three semesters.)
• Chance to address science literacy goals.
* SWoRD has just added fees
How I am using SWoRD in my pilot
semester:
GOALS: To address scientific literacy by asking students to draw
conclusions from popular media/ data sets and to practice the
skill of evaluation.
OBJECTIVES WITH THE TWO ASSIGNMENTS:
• State the potential dangers in believing "assumptions" without valid
scientific data.
• Determine if there is a bias in news media articles through the evaluation
of sources within the article.
• Form an opinion on a biological issue and support it with reasoning.
• Identify the limitations of a scientific study and their impact on how we
interpret results.
Assignment 1
1. List three people/groups that support the use of the ACTN3 gene test in society.
Do you find these sources/quotes to be really persuasive in support for the use of the gene test, why
or why not? Explain how credible you think each of these sources are (i.e. who would you believe
more on this topic—a plumber or a geneticist?)
2. List three people/groups that DO NOT support the use of the ACTN3 gene test in society.
Do you find these sources/quotes to be really persuasive against the use of the gene test, why or why
not? Explain how credible you think each of these sources are.
3. On the first New York Times article, do you think the author, Juliet Macur, has an opinion or is she
presenting a neutral story? Why? (For example, does she present less credible people on one side of
the story compared to the other? Does she miss an important viewpoint? Or does she balance her
supporting claims on each side equally?) Defend your claim with supporting evidence.
4. Consider the studies that have been done on the ACTN3 gene and write persuasively as to why you
would or would not have a future child tested. (Tip: use facts/quotes/examples to write persuasively.)
5. The studies on ACTN3 discussed here have limitations. What are the limitations/flaws in design
specifically? What kind of consequences might these limitations have on drawing conclusions about
the predictive value of the ACTN3 gene test?
Instructor can
provides examples
or an entire model
paper to students:
Alternate text
from a biased
media news
source.
“Note that the author doesn’t just
state generalities, he cites specific
names and credentials of people. In
this case, he does this to show that
these are NOT credible people. If used
in a different way, quoting people and
giving their credentials can help
support information.”
“Numbers and graphs
themselves don’t make the
article credible. You must
look at the graphs and the
values to evaluate for
yourself.”
Provide the grading rubric to students before
writing their essays
Go back to the article and find the three examples the student gave for people/groups who support
the ACTN3 test. Are the student’s examples consistent with people/groups who support the use of the
ACTN3 test and does the student justify why they provide support? Choose the best answer from the
choices below.
o Yes, the examples provided are consistent with people who support the gene test. AND, the student
clearly justifies his or her thoughts on this.
o Somewhat. All or most examples provided are consistent with people who support the gene test and the
student makes some justifications.
o The example(s) given by the student might or might not be consistent but the student does not clearly
justify why he or she thinks so. The student has not gone beyond listing examples.
Did the student discuss whether author Juliet Macur had an opinion or presented a neutral story?
o Student provided a well-written and thorough explanation of Macur's stance. The answer was wellbacked
with evidence from the article.
o Student effectively discusses Macur's opinion but provides a somewhat weak amount of supporting
evidence from the article OR student provides some evidence that does not really support their
claim.
o Student effectively discusses Macur's opinion but does not provide good supporting evidence from
the article OR student provides evidence that does not support their claim.
o Student discusses the opinion but provides no real discussion of supporting evidence for their
statement.
o Student fails to comment on Macur's opinion
7 evaluation questions with
points assigned
(0-7 points for each question)
Writing grade = 75%
Qualifish76
This student’s grade:
Does the system discriminate high achievers
vs. low achievers (based on exam
performance?)
86%
79%
Total
SWoR
D
grade
64%
Low
High
achievers achievers
Low
Achievers
(outliers
removed)
n = 15
For each
group
PROS
CONS
• Interface is very simple for student •
• Grace periods with or without
penalties
•
• No major technical problems with
uploading or server crashing etc.
• Writing assignment graded without •
instructors; assignment would not
have been possible otherwise
•
• A re-do of the assignment (with
different reviewers) led to fairly
similar scores for students who re- •
did.
• Students wrote some very thoughtful
evaluations and took assignment
seriously.
Interface is awkward for
instructor
Students had a lot of trouble
understanding the process and
partial grades.
Students missed deadlines and
couldn’t jump in late
Some students will have less
reviews if their peers don’t
complete review
Students are wary about their
grade determined by peers