Hankins – IBIS Faculty Report
1. Provide an overview of your course and what you redesigned for the IBIS project
A foundation course for English and Liberal Arts majors in the literature
concentration, English 258 introduces students to literary criticism and theory and builds
skills of critical thinking and writing in English Studies. Emphasis is on the application
of principles and methods of literary theory to selected texts, and examining and
responding to texts from a variety of critical perspectives.
My project targeted the following department-provided SLO: students will write
well-developed, coherent essays that use critical theory to interpret a text. I assessed how
online delivery of peer review writing workshops impacts students’ abilities to utilize
critical theory in their written work. Students were placed in peer review groups during
the first week of class. Students exchanged drafts of written work and commented on
those drafts using Word review tools and the myCourses discussion board. During online
class sessions, students used the myCourses discussion board to discuss their comments,
the challenges of applying the assigned theory to a literary text, and strategies for meeting
those challenges. These peer review sessions were organized around weekly response
papers and two formal papers.
Midterm course evaluations overwhelmingly stated that students were not finding
the online sessions helpful. Following this evaluation I modified the course so that
students continued to post drafts and comments online, but the discussion of those drafts
took place in face to face sessions.
2. Explain the data you collected
A – Surveys
I administered a survey at the beginning of the semester evaluating student attitudes
toward peer review. At the end of the semester my mentor administered a survey
evaluating student attitudes toward the online peer review sessions. See Appendix 1 and
Appendix 2 for quantifiable survey results.
B – Grades
I also compared grades for online workshops to grades for written work. See Appendix 3
for relevant grade averages.
C – Control Group
I was also able to compare grades for written work from a previous semester that did not
incorporate online sessions.
3. Provide a data analysis and your conclusions about the course experience. The
central question is did the blend positively or negatively or not at all affect student
learning? This is the core of your report.
A – Surveys
The final student survey showed that students felt they received accurate feedback
from their peers during online peer review, that face-to-face sessions prepared them for
the online sessions, that instructor evaluation of online sessions had a positive impact on
Hankins – IBIS Faculty Report
the effectiveness of online sessions, and that online sessions increased their
understanding of instructor expectations and assignment guidelines.
Despite these positive effects, students did not find the workshops to improve
their ability to write an essay using critical theory to interpret a text. The numbers were
split on whether the review sessions improved the group’s ability to write essays using
critical theory.
Most students felt the myCourses platform was an impediment to learning—
students overwhelmingly found the myCourses discussion board to be an ineffective tool
for practicing the written application of critical theory. The chat room feature also
proved ineffective as most students were unable to use it. These findings also reflect the
information gathered by my mentor in her final evaluation of the course.
The qualitative data I collected in the surveys also suggests that although students
did not believe the online workshops improved their learning, they did find some benefit
in receiving written comments from peers online. For example, one student stated “I
preferred online because I think the comments were thought out instead of thought of on
the fly.” Another student observed that “it was easier to formulate my thoughts online
and I felt we never got off topic, like you can when face-to-face. You also have a record
of the discussion.” The same student noted “people who have a difficult time getting
their thoughts out in person or get nervous/shy get more involved online and its really
helpful for them, they can get ideas across without pressure.” Another student noted that
online sessions allowed for more emphasis on “information relating to improving
understanding of [applying] theory to text.” This, of course, was one of my primary
goals for the online sessions. These comments summarize the comments of about 7 of
the surveys; for these students, online sessions allowed for more focused discussion,
more articulate feedback on drafts, and equal participation.
The negative comments emphasized the limitations of the myCourses platform
and expressed general dislike of online conversations. The new blend I instituted
midsemester seems to have addressed these concerns, as many students noted that the
online comments combined with face-to-face discussions were the most helpful parts of
the peer review system. One challenge I did not successfully address was expressed by
students who were unhappy with their peer group’s participation. The success of these
review groups depends on each member committing to the process. Although online
work accounted for 15% of the final grade, clearly I need to implement policies that more
effectively guarantee a positive experience for every committed student.
B – Grades
Despite the students’ assessment that online peer review sessions did not improve
their ability to use critical theory to interpret a text, there might be a correspondence
between grades for online sessions and written work. The four lowest averages for online
work (C- or below) corresponded to the four lowest scores for written work (B- or
below). Students who received at least a C+ for online work received at least a B for
written work.
C – Control Group
Comparing this semester’s grades to the previous semester’s grades shows some
improvement, especially in response paper averages. The response paper average for the
Hankins – IBIS Faculty Report
semester without online review sessions was 2.946; with online sessions the response
paper average was 3.353. Formal paper grades showed a slight improvement. In the
previous semester the average for the midterm paper was 3.0 and the average for the final
paper was 2.873. In the blended semester the average for the midterm paper was 3.116
and the average for the final paper was 2.889. These improvements, however, are likely
a result of my own increased ability to write more precise assignments and grading
rubrics. Additionally, these improvements might simply reflect the added peer review
component, not necessarily the online component.
One number that I do think is a direct reflection of the online sessions is that in
the previous semester, of the 17 students who completed the course, 5 students failed to
complete the 4 required response papers. This semester, of the 19 students who
completed the course, not a single student failed to complete all 5 required response
papers. Although students were not convinced of the value of the online sessions, as an
instructor, I witnessed a tremendous improvement in the responsibility students took for
their work when asked to participate in online group workshops.
Conclusions
Given these results, I’ve concluded that the online peer review workshops may
have increased student learning, but there is not any significant quantitative data that
convincingly reflects this speculation. In the future, I will continue to ask students to
post drafts and comments online, but I will reserve online sessions for discussions
focused on specific key content questions rather than writing assessment.
I’d also like to note that I noticed a negative attitude toward online work from the
‘millennials’ when I introduced the grant project at the beginning of the semester. Some
students never overcame this resistance. Some of these negative attitudes are directly
linked to the positive benefits of online work that instructors observe—online peer review
is more demanding (thus the resistance) than face-to-face peer review because students
are forced to precisely articulate their thoughts in writing, the online record preserves
their level of investment in each review session, and even the most reticent or disengaged
students must participate in order to receive credit. Without discounting students’
assessment of their experience, I do continue to believe that instructors are able to
observe benefits that students might not see. I need to do a better job of convincing
students of these benefits.
4. Provide a section outlining your peer mentor experience. When did you meet?
What guidelines did you set up? What worked well? What didn't work well?
Professor Anupama Arora and I discussed my plan over the summer via email and
telephone. She also provided me with sample assessment materials. Professor Arora
observed the myCourses site and administered the final survey to students at the end of
the semester. She also asked students to give her their reflections of the course.
Hankins – IBIS Faculty Report
Appendix 1 – Survey Administered 7 September 2012
1) On a scale of 1-5 (1 being the lowest) indicate how confident you feel in your ability
to write an essay that uses critical theory to interpret at text.
Average – 3.7857
2) On a scale of 1-5, indicate how confident you feel that peer review will improve your
ability to write an essay that uses critical theory to interpret a text.
Average - 3.666667
3) On a scale of 1-5, indicate how helpful you have found peer review writing workshops
in the past.
Average - 3.381
4) On a scale of 1-5, indicate how confident you feel in the feedback you receive from
your peers during writing workshops.
Average - 3.190476
5) On a scale of 1-5, indicate how willing you are to significantly revise your writing
based on feedback from a peer.
Average - 3.52381
6) How much time do you usually spend reading and commenting on a peer’s draft?
(Assume the draft is about 5 pages long.)
a. less than 30 minutes
4
b. 30-60 minutes
13
c. 1-2 hours
4
d. more than 2 hours
0
7) How would you describe the drafts you usually submit for peer review?
a. very rough (argument undetermined, prose deteriorates into notes, less than half the
required length, no research incorporated) 0
b. rough (argument attempted but vague and unfocused, fully in prose but supporting
paragraphs not fully developed, no clear transitions between ideas, research inserted but
not properly cited)
10
c. complete (argument is clearly stated, all supporting paragraphs are fully developed,
clear transitions between ideas, research properly cited)
10
Hankins – IBIS Faculty Report
8) What do you find most helpful about peer review?
9) What do you find least helpful about peer review?
Appendix 2 –Survey Administered by Mentor on 10 December 2012
1) Online peer review workshops increased my understanding of instructor expectations
and assignment guidelines.
Strongly agree 1
Agree 10
Disagree 5
Strongly disagree
2) Online peer review workshops improved my ability to write an essay that uses critical
theory to interpret a text.
Strongly agree 1
Agree 5
Disagree 10 Strongly disagree
3) Online peer review workshops improved my peer review group’s ability to write
essays that use critical theory to interpret a text.
Strongly agree 1
Agree 8
Disagree 7
Strongly disagree
4) Face-to-face class sessions provided the preparation I needed to offer my peers
accurate feedback on their written applications of critical theory.
Strongly agree 7
Agree 8
Disagree
Strongly disagree 1
5) The myCourses Discussion Board was an effective tool for peer review.
Strongly agree Agree 4
Disagree 11 Strongly disagree 1
6) The myCourses Discussion Board was an effective tool for practicing the written
application of critical theory.
Strongly agree Agree 7
Disagree 8
Strongly disagree
7) I received accurate feedback from my peers during online peer review.
Strongly agree 3
Agree 12
Disagree
Strongly disagree 1
8) The instructor evaluation of online peer review sessions had a positive impact on the
effectiveness of those sessions.
Strongly agree 1
Agree 11
Disagree 4
Strongly disagree
9) How have your attitudes toward peer review work changed over the course of the
semester?
10) What was useful or not useful about the online peer review system?
11) How did your experience with online peer review sessions this semester compare to
your experience with face-to-face peer review sessions in the past?
Hankins – IBIS Faculty Report
12) What could you and your group members have done differently to increase the
effectiveness of online peer review sessions?
13) What could the instructor have done differently to increase the effectiveness of
online peer review sessions?
14) Further comments are welcome below:
Appendix 3 – Relevant Grade Averages
Response paper
Midterm
Average for
Average for
average (5 total)
paper
Final paper written work
online work
3.3
3.7
3.7
3.566666667
4
3
3
3.3
3.1
4
3.3
3
3
3.1
4
3.7
4
3.3
3.666666667
4
4
4
3
3.666666667
4
2.7
3.7
2.3
2.9
3.7
3.3
3
3
3.1
3.7
3.3
3.3
2
2.866666667
3.7
3.7
3
3.3
3.333333333
3.7
4
3.3
3.7
3.666666667
3.7
4
3.3
3
3.433333333
3.7
3.7
3
3.3
3.333333333
3.7
3.7
3.3
3.7
3.566666667
3.3
3.3
3.7
3
3.333333333
2.7
3.3
3.3
2.3
2.966666667
2.3
2.7
2.7
2.7
2.7
1.7
3.3
2.3
2
2.533333333
1.7
2.7
2.3
2.3
2.433333333
1.7
2.7
1.3
2
2
1.3
3.352631579 3.115789474 2.88947368