Kimberly J. Banks and Laurel Harris
Assessment Institute Spring 2013
Assessment Report
English Department
EN-102 English Composition II: Introduction to Literature
General Education Objective
6. Differentiate and make informed
decisions about issues based on multiple
value systems
Course Learning Outcome
Learning Outcome #5:
Differentiate relevant evidence throughout all
writing tasks, including written texts, visual images,
electronic media and such primary sources as
observations, interviews, and surveys.
Assessment Activity
Tools
We used the attached rubric to evaluate whether and how well students have achieved the EN102 course learning outcome #5 and the relevant general education learning outcome #6.
Assignments
Six faculty members contributed several randomly selected artifacts of student writing in
response to a graded, substantive assignment that asks students to select and integrate relevant
evidence into their writing.
Overview of Assignments
1. Students write a 3-4 page paper by identifying a difficult aspect of a short story they've
selected, assessing at least two hypotheses and incorporating at least one classmate's
opinion either in support or as a way to challenge any one of their hypotheses.
2. Students will write a 1600-2000 word essay comparing two plays and incorporating
evidence from at least three library research sources.
3. Students will write a 3-4 page paper synthesizing two plays and a critical essay.
4. Students will write a 3-4-page essay defining and applying specific literary concepts to
the close reading of a poem.
5. Students will write a 3-4 page paper arguing for how a work of their choice should be
incorporated into the course based on the theme “Death, Disease, and Disorder.” They
will incorporate evidence from the new text and at least two texts discussed in class.
6. Students will write a 3-4 page essay interpreting one text (story or poem which we read
in class) with reference to two secondary sources written about the chosen text.
EN-102 Course Assessment
Page 1
Data Collection Instrument
A four-point rubric with six specific criteria was used to assess each writing sample. The rubric
is designed to measure student achievement in terms of manipulating evidence. This exercise
was designed to assess the reliability of the rubric for measuring what it purports to measure.
Analysis and Summary of the Assessment Results
We modified the rubric as follows:
• We rewrote aspects of the “Outstanding” column of our original rubric to make it
consistent with the kinds of writing that would appropriately qualify for the category.
• We realized that the rubric does not provide a way to account for the quality of evidence
marshaled to support a writer’s claims. So we adjusted the first row to reflect this.
• We removed “multiple sources” from the rubric as explained below.
• We combined rows 4 and 5 as they were more accurately evaluated together.
We determined that to accurately assess student proficiency with this outcome, we would need to
collect batches of papers in individual classes next year rather than a completely randomized
collection of samples. The assignment itself is not necessary for evaluation whereas student
proficiency can only be determined within the context of peer performance on the same
assignment.
We concluded that the department needs a way to measure a student’s rhetorical savviness within
the context of specific genres. Not all genres require students to manipulate evidence in the same
way nor do all genres define evidence in the same way.
We realized that the way we framed the task introduced an unnecessary condition for evaluation.
We explained the learning outcome as follows: “Students will be able to select, explain, analyze,
synthesize, and integrate evidence from multiple sources.” The condition that students should use
“multiple sources” caused a problem in the scoring. The general education objective emphasizes
“multiple value systems” while the learning objective for the course emphasizes “differentiating
evidence.” Simply using “differentiating evidence” as a condition for evaluation is insufficient
because it does not account for the multiple steps students must follow to accomplish this task
proficiently. At the same time, in the context of EN 102 English Composition II, students have to
produce many close readings. Such readings require a careful manipulation of evidence but only
within the context of one source. In order to more accurately score student proficiency in both
the course and general education outcomes, the new course learning outcome will thus be as
follows: “Students will be able to select, explain, analyze, synthesize, and integrate evidence.”
We tentatively propose that at the end of EN 101 English Composition I, students should be
developing in at least three categories of the rubric and satisfactory in at least three categories.
Consequently, we tentatively propose that, to demonstrate progress, at the end of EN 102 English
Composition II, all students who have taken EN 101 and EN 102 in a two-semester sequence
should be satisfactory in all categories of the rubric.
EN-102 Course Assessment
Page 2
Assessment Results and Effects on Student Learning Outcomes
For the first two criteria, one scorer averaged above the “Satisfactory” criteria with 3.06 in the
first category and 3.02 in the second (Table 1) while the second scorer averaged 2.92 for both
criteria (Table 2). Both scorers averaged approximately the same score for the third criterion
(2.79 for the first scorer and 2.77 for the second). In the fourth category, the first scorer’s
average was .13 points lower than the second (2.62 versus 2.75), a trend even more pronounced
in the fifth category (2.55 versus 2.77). In the final category, the first scorer’s average was higher
than the second (2.83 versus 2.54).
Given that the 48 essays scored did not represent a statistically significant sample of EN-102
students, the results were used to revise the assessment instrument as described above and as
attached to prepare for a more comprehensive assessment using the revised rubric to be
conducted in the fall of 2013. For example, the scorers interpreted the “Outstanding” columns
differently for the first two criteria and these columns were adjusted to clarify the category. In
addition, the scorers interpreted the rubric very differently for the final fourth and fifth criteria.
The first scorer determined her score based on the specification “multiple sources” while the
second scorer considered this specification less important in instances when the assignment
specifically involved close reading. To this end, both the learning outcome and rubric have been
revised to clarify the expected outcome and its criteria.
Table 1 Score Breakdown (48 Essays)
Rubric for Learning Outcome: “Students will be able to select, explain, analyze, synthesize, and integrate
evidence from multiple sources.”
Criteria
Outstanding
(4)
Satisfactory
(3)
Developing
(2)
Unsatisfactory
(1)
Totals
Average
Scores
15
22
10
1
147
3.06
Select appropriate evidence from
multiple sources
11
27
10
0
145
3.02
Explain and analyze evidence
from multiple sources
11
17
19
1
134
2.79
Synthesis of evidence from
multiple sources
6
19
22
1
126
2.62
Integrate evidence from multiple
sources
8
13
25
2
123
2.56
16
11
18
3
136
2.83
67
109
104
8
811
2.82
23.3%
37.5%
36.1%
2.8%
Present author’s unique
perspective and identify other
perspectives
Use MLA citation style
Totals
Percentages
EN-102 Course Assessment
Page 3
Table 2 Score Breakdown (48 Essays)
Rubric for Learning Outcome: “Students will be able to select, explain, analyze, synthesize, and integrate
evidence from multiple sources.”
Criteria
Outstanding
(4)
Satisfactory
(3)
Developing
(2)
Unsatisfactory
(1)
Totals
Average
Scores
8
29
10
1
140
2.92
Select appropriate evidence from
multiple sources
6
32
10
0
140
2.92
Explain and analyze evidence
from multiple sources
7
23
18
0
133
2.77
Synthesis of evidence from
multiple sources
7
23
17
1
132
2.75
Integrate evidence from multiple
sources
6
25
17
0
133
2.77
6
17
22
3
122
2.54
40
149
94
5
800
2.77
13.9%
51.7%
32.6%
1.7%
Present author’s unique
perspective and identify other
perspectives
Use MLA citation style
Totals
Percentages
In the fall of 2013, ten faculty members will use the revised rubric to evaluate a statistically
significant and representative sample size of roughly 600 student essays in both EN 101 and EN
102.
Appendices
Appendix A:
Original Scoring Rubric
Appendix B:
Revised Scoring Rubric
Appendix C:
Assignment #1 with Two Sample Papers
Appendix D:
Assignment #2 with Two Sample Papers
Appendix E:
Assignment #3 with Two Sample Papers
Appendix F:
Assignment #4 with Two Sample Papers
Appendix G:
Assignment #5 with Two Sample Papers
Appendix H:
Assignment #6 with Two Sample Papers
EN-102 Course Assessment
Page 4
Appendix A
Rubric for Learning Outcome: “Students will be able to select, explain, analyze, synthesize, and integrate evidence from multiple sources.”
Criteria
Present author’s
unique perspective
and identify other
perspectives
Outstanding
Satisfactory
Developing
Unsatisfactory
Persuasively presents a
unique perspective and is
able convincingly
characterize other
perspectives
Accurately presents a
unique perspective and is
able to adequately
characterize other
perspectives
Presents a unique
perspective or identifies
other perspectives
Is unable to present a unique
perspective or identify other
perspectives
Select appropriate
evidence from
multiple sources
Selects and contextualizes
convincing evidence from
more than one source
Selects appropriate evidence
from more than one source
Selects appropriate evidence
from only a single source or
inappropriate evidence from
more than one source
Selects no evidence or
inappropriate evidence
throughout the essay
Explain and analyze
evidence from
multiple sources
Persuasively explains and
convincingly analyzes
evidence from more than
one source
Satisfactorily explains and
analyzes evidence from
more than one source
Attempts explanation and
analysis of most evidence,
but explanation or analysis
may not be fully relevant
Explains and analyzes most
evidence unsatisfactorily
Synthesis of evidence
from multiple sources
Synthesizes (connects)
evidence persuasively and
insightfully from more than
one source
Synthesizes (connects)
evidence adequately from
more than one source
Attempts synthesis
(connection) of evidence
from more than one source,
but connections may be
incomplete
Presents evidence without
synthesis
Integrate evidence
from multiple sources
Convincingly and
insightfully integrates
(connects sources to own
ideas) evidence from more
than one source
Integrates (connects sources
to own ideas) evidence
adequately from more than
one source
Attempts integration
(connection to own ideas) of
evidence, but connections
may not adequately be
explained
Integrates most evidence
inadequately
Use MLA citation
style
Uses MLA citation style
flawlessly
Uses MLA citation style
adequately
Attempts but inadequately
uses MLA style
Fails to use MLA citation
style
EN 102 Course Assessment
Spring 2013
page 5
Appendix B
Revised Rubric for Revised Learning Outcome: “Students will be able to select, explain, analyze, synthesize, and integrate evidence.”
Criteria
Present author’s
unique perspective
and identify other
perspectives
Precisely presents and
supports a unique
perspective and is able
convincingly characterize
other perspectives
Outstanding
Accurately presents and
supports a unique
perspective and is able to
adequately characterize
other perspectives
Presents a unique
perspective or identifies
other perspectives
Is unable to present a unique
perspective or identify other
perspectives
Select appropriate
evidence
Selects and contextualizes
convincing evidence
Selects and contextualizes
appropriate evidence
Selects some appropriate
and some inappropriate or
unclear evidence
Selects no evidence or
inappropriate evidence
throughout the essay
Explain and analyze
evidence
Convincingly explains and
analyzes evidence
Satisfactorily explains and
analyzes evidence
Attempts explanation and
analysis of most evidence,
but explanation or analysis
may not be fully relevant
Fails to explain or analyze
evidence or explains and
analyzes irrelevant evidence
Synthesize and
integrate evidence
Synthesizes and integrates
evidence insightfully
Synthesizes and integrates
evidence adequately
Attempts synthesis and
integration of evidence, but
connections may be
incomplete
Fails to synthesize or
integrate evidence
Use MLA citation
style
Uses MLA citation style
accurately most of the time
Uses MLA citation style
adequately
Attempts but inadequately
uses MLA style
Fails to use MLA citation
style
EN 102 Course Assessment
Satisfactory
Spring 2013
Developing
Unsatisfactory
page 6
Appendix C/page 1
EN 102 Course Assessment
Spring 2013
page 7
Appendix C/page 2
EN 102 Course Assessment
Spring 2013
page 8
Appendix C/page 3
EN 102 Course Assessment
Spring 2013
page 9
Appendix C/page 4
EN 102 Course Assessment
Spring 2013
page 10
Appendix C/page 5
EN 102 Course Assessment
Spring 2013
page 11
Appendix C/page 6
EN 102 Course Assessment
Spring 2013
page 12
Appendix C/page 7
EN 102 Course Assessment
Spring 2013
page 13
Appendix C/page 8
EN 102 Course Assessment
Spring 2013
page 14
Appendix C/page 9
EN 102 Course Assessment
Spring 2013
page 15
Appendix C/page 10
EN 102 Course Assessment
Spring 2013
page 16
Appendix C/page 11
EN 102 Course Assessment
Spring 2013
page 17
Appendix C/page 12
EN 102 Course Assessment
Spring 2013
page 18
Appendix C/page 13
EN 102 Course Assessment
Spring 2013
page 19
Appendix C/page 14
EN 102 Course Assessment
Spring 2013
page 20
Appendix D/page 1
EN 102 Course Assessment
Spring 2013
page 21
Appendix D/page 2
EN 102 Course Assessment
Spring 2013
page 22
Appendix D/page 3
EN 102 Course Assessment
Spring 2013
page 23
Appendix D/page 4
EN 102 Course Assessment
Spring 2013
page 24
Appendix D/page 5
EN 102 Course Assessment
Spring 2013
page 25
Appendix D/page 6
EN 102 Course Assessment
Spring 2013
page 26
Appendix D/page 7
EN 102 Course Assessment
Spring 2013
page 27
Appendix D/page 8
EN 102 Course Assessment
Spring 2013
page 28
Appendix D/page 9
EN 102 Course Assessment
Spring 2013
page 29
Appendix D/page 10
EN 102 Course Assessment
Spring 2013
page 30
Appendix D/page 11
EN 102 Course Assessment
Spring 2013
page 31
Appendix D/page 12
EN 102 Course Assessment
Spring 2013
page 32
Appendix D/page 13
EN 102 Course Assessment
Spring 2013
page 33
Appendix D/page 14
EN 102 Course Assessment
Spring 2013
page 34
Appendix D/page 15
EN 102 Course Assessment
Spring 2013
page 35
Appendix E/page 1
EN 102 Course Assessment
Spring 2013
page 36
Appendix E/page 2
EN 102 Course Assessment
Spring 2013
page 37
Appendix E/page 3
EN 102 Course Assessment
Spring 2013
page 38
Appendix E/page 4
EN 102 Course Assessment
Spring 2013
page 39
Appendix E/page 5
EN 102 Course Assessment
Spring 2013
page 40
Appendix E/page 6
EN 102 Course Assessment
Spring 2013
page 41
Appendix E/page 7
EN 102 Course Assessment
Spring 2013
page 42
Appendix E/page 8
EN 102 Course Assessment
Spring 2013
page 43
Appendix E/page 9
EN 102 Course Assessment
Spring 2013
page 44
Appendix E/page 10
EN 102 Course Assessment
Spring 2013
page 45
Appendix F/page 1
EN 102 Course Assessment
Spring 2013
page 46
Appendix F/page 2
EN 102 Course Assessment
Spring 2013
page 47
Appendix F/page 3
EN 102 Course Assessment
Spring 2013
page 48
Appendix F/page 4
EN 102 Course Assessment
Spring 2013
page 49
Appendix F/page 5
EN 102 Course Assessment
Spring 2013
page 50
Appendix F/page 6
EN 102 Course Assessment
Spring 2013
page 51
Appendix G/page 1
EN 102 Course Assessment
Spring 2013
page 52
Appendix G/page 2
EN 102 Course Assessment
Spring 2013
page 53
Appendix G/page 3
EN 102 Course Assessment
Spring 2013
page 54
Appendix G/page 4
EN 102 Course Assessment
Spring 2013
page 55
Appendix G/page 5
EN 102 Course Assessment
Spring 2013
page 56
Appendix G/page 6
EN 102 Course Assessment
Spring 2013
page 57
Appendix G/page 7
EN 102 Course Assessment
Spring 2013
page 58
Appendix G/page 8
EN 102 Course Assessment
Spring 2013
page 59
Appendix G/page 9
EN 102 Course Assessment
Spring 2013
page 60
Appendix G/page 10
EN 102 Course Assessment
Spring 2013
page 61
Appendix H/page 1
EN 102 Course Assessment
Spring 2013
page 62
Appendix H/page 2
EN 102 Course Assessment
Spring 2013
page 63
Appendix H/page 3
EN 102 Course Assessment
Spring 2013
page 64
Appendix H/page 4
EN 102 Course Assessment
Spring 2013
page 65
Appendix H/page 5
EN 102 Course Assessment
Spring 2013
page 66
Appendix H/page 6
EN 102 Course Assessment
Spring 2013
page 67
Appendix H/page 7
EN 102 Course Assessment
Spring 2013
page 68
Appendix H/page 8
EN 102 Course Assessment
Spring 2013
page 69
Appendix H/page 9
EN 102 Course Assessment
Spring 2013
page 70
Appendix H/page 10
EN 102 Course Assessment
Spring 2013
page 71
Appendix H/page 11
EN 102 Course Assessment
Spring 2013
page 72