Advice for Evaluating Student Work Using Rubrics
This new version of the Wabash Study will be the first time we have asked participating
our institutions to use rubrics to evaluate student work and teaching practices. Some
of you may already have extensive experience in using rubrics to gather and review
student work. For you, this document will most likely be redundant with what you already
know.
The goal or “outcome” of assessment is not measuring student learning, it is improving
student learning. Measuring student learning is a means to that outcome, not the
outcome itself. The most important question you need to consider in using rubrics to
evaluate student work is “How will what we learn from using rubrics improve the quality
and impact of teaching on our campus?” We suggest you consider the process of using
rubrics as a professional development activity disguised as the evaluation of student
work. As you build the activities consider how each step can be structured so that the
faculty and staff you are engaging will walk away with lessons they can apply to their
work with their students.
Like most forms of assessment, using rubrics to review student work requires significant
planning and careful execution. Below, we will review a number of details that we
recommend you consider in planning the rubric-based assessment component of the
Wabash Study.
Timeline
The year-long schedule we have suggested for collecting and evaluating student
work using rubrics is exceptionally tight. It will require steady work throughout the
next year to meet the deadline of reviewing documents by the summer of 2011.
Step One – Collecting Student Work
We suggest you consider the following questions prior to collecting student work.
● How will you inform your institution’s IRB about your project?
○ IRBs at most institutions will “exempt” using rubrics to evaluate student
work as long as the evaluation is for assessment purposes, students’
names are removed from the work, and you do not plan to use the
findings for publication. However, there are few constants in the world of
IRBs, so you should check with yours as soon as possible to find out what
they would like you to do.
● What specific question do you want to address in using rubrics to evaluate
student work? This breaks down into three related questions:
○ What outcomes are you targeting for the Wabash Study?
○ What part or parts of the institution are engaged in promoting those
outcomes?
○
Are you trying to gauge how much students change or whether
they reach some minimal threshold or proficiency on the outcomes?
● Are the assessment questions you are trying to address by reviewing student
work relevant to faculty and staff teaching interests?
○ If you focus on a question that is important to faculty, it will be easier to
get them involved in the project—both in helping you collect student work
and in helping with the rubric workshops.
○ One way of generating faculty and staff interest is to stress the fact
that reviewing student work allows them to get information they cannot
get from simple surveys or from standardized assessment measures.
○ Consider framing the assessment question in terms of “What can we
learn about the kinds of assignments that are most likely to promote high
student performance on outcome X?” rather than “How much do our
students gain on outcome X?”
● What should you collect?
○
○
What kinds of student work are best suited to addressing the previous
questions?
We have assumed that most of you wish to review students’ written work
and that is what we will focus on below.
■ But we are happy to work with you if you are interested in
considering other types of student work. Moreover, many of
the same issues we discuss below will arise regardless of
what kind of student work you are collecting and reviewing.
● Who will you target?
○ It is important to identify your target student population early.
○ Are the student groups you target the right students to address in your
assessment questions?
○ Can you contact and work with those students via specific courses?
○ Aim to identify and contact the largest group of students you can with as
little intrusion as possible.
■ Work with leaders in academic and student affairs to vet and get
support for your paper collection plans.
● What type(s) of papers should you collect that will help you assess one or more
of your institution’s learning outcomes?
○ Should they include:
■ Thesis-driven, argumentative, or case-building papers?
■ Papers or reports from student capstone experiences?
■ Creative writing?
■ Structured reflections on internships?
■ Documents in which your institution asks students to reflect on
their curricular experiences?
■ References to sources or bibliographies?
■ Papers that focus on personal experiences?
○ How long should the papers be to demonstrate the kinds of outcomes you
are assessing?
■ How many pages will you specify in your call for student work?
■ When thinking about the range for collection, consider the length
of papers students typically produce on your campus. If you want
10- to 12-page papers, but your students rarely write papers that
length, it will be hard to get an adequate number of documents.
○
Will you collect papers by contacting students directly or by
contacting faculty and staff in specific courses and programs?
■ If you collect papers through faculty and staff
○ It is important to talk with them early on about the rubric
project, including the types of documents you’re looking for
and what you hope to learn from the evaluation of student
papers.
○ Engaging your writing center director or other staff and
faculty who support student writing at your campus can be
helpful. They can talk to their colleagues and encourage
them to help collect papers.
○ Faculty and staff who support student writing may also
have a sense of which courses to target as well as the
types of assignments in those courses.
■
If you collect papers from students
○ There are upsides and downsides. It is usually more laborintensive to collect work directly from students, but the
benefit is that they can email papers directly to you—
there’s no middleman.
○ Students are often receptive to submitting papers
as long as project leaders explain why this project is
important to the institution and how students can help.
■ Make it real for your students; tell them what you’ll
do with the results.
■ Consider engaging your student government and
commit to reporting and discussing the results with
them.
○ Consider incentives. A Teagle collaborative paid students
$20 if they submitted a paper and took the CLA. One
institution also offered to provide students with a statement
about the study for their resumes.
○ It may be effective to involve instructors and ask them to
speak with students about submitting their work. Students
may be more willing to submit work when their faculty ask
them to.
● What information will you collect in addition to the papers? We recommend you
include:
○ The assignment prompt.
○ The department and course from which the assignment came.
○ The proportion of the course grade for which the assignment counted.
■ Assignments that count for more of a student’s grade tend to be of
higher quality. This factor is especially important to consider it you
want to examine how students change over time.
○ Were drafts/revision required?
■ Papers that have required drafts tend to be better.
○ Was the paper peer reviewed or workshopped in class?
Step Two – Reading and Evaluating Student Papers
● Who will read the papers and use rubrics to assess them?
○ As you collect the papers, think of the time you’ll need for people to read
through and evaluate them. More papers, or longer papers, will require
more people and/or more time.
○ It is also crucial to create enough time for faculty and staff to discuss what
they are learning from the papers. This is where the learning takes place.
■ We recommend you design the reading/evaluation sessions in
reverse—start by setting up time and space for discussion and
then work on figuring out how much time you need to set aside for
reading papers.
■ We believe it is better for faculty and staff to read fewer papers
and spend more time talking about what they are learning than
vice versa. Think of this as a faculty and staff development
activity.
● How will you respond to the following concerns that faculty and staff may raise?
○ Concern about being asked to read papers outside of a person’s area of
expertise.
○ Fear of having our assignments read by other faculty.
■ Your responses to these concerns will depend on exactly what
outcomes you’re trying to assess. Above all else, it is important to
structure and frame this work as a faculty and staff development
opportunity, and to build the process so that faculty and staff have
a chance to learn from one another. The evaluation of student
work is a means to that broader end. Embedding professional
development into this and every other assessment process
is the best way to make assessment impact the classroom.
● What rubrics will you use?
○ Your rubric(s) should be specific to the question you’re asking.
○ If you plan to use generic rubrics, like the AAC&U VALUE rubrics, it may
be important to modify them for your campus. Consider them as starting
points for faculty and staff discussions about what matters, not as readyto-use products.
○ Even though you may spend a considerable amount of time developing
the rubrics, leave time for faculty and staff who will use the rubrics to
make final modifications.
■ However, it is important to limit the extent of these modifications
to avoid taking up too much time. No rubric, or any assessment
method, is perfect. It has to be “merely” good enough to help
us learn something about how our students are learning.
Example – The following example is from a six-institution Teagle Collaborative that
engaged in significant formative assessment of student writing and critical thinking. The
reading sessions for this collaborative were held during the summer (note: summer work
typically requires a stipend).
● The sessions were two and a half days long and food was provided for
participants.
● There were four readers from each institution plus deans and provosts for a total
of 30 readers.
○ Our recommendation would be to employ more readers and have them
read fewer papers, rather than vice versa. Since this is a professional
development activity, engaging more people translates into more
development.
● One of the collaborative leaders estimated the time needed for the reading
sessions as follows:
●
●
●
It was important to have norming sessions for working with the rubrics to get all
of the readers “on the same page.” They brought in an outside expert to lead the
norming sessions. Inviting an expert helped to legitimize the experience and the
use of the rubrics.
Each paper was read twice. If the scores weren’t close, the paper was read by
a third reader. Because of norming sessions, they did not have to go to a third
reader very often.
After the reading sessions, project leaders asked readers for feedback
about the sessions and about what they learned from reading the papers.
○ We recommend you summarize this feedback on a short report and send
it to the readers for additional comments. Once they have vetted it, you
can include your summary in your reports about the findings from the
sessions.
Other questions to consider
● Who will be responsible for tabulating the scores from the reading
sessions and entering this information into a spreadsheet or database?
● Who will be responsible for entering additional information for each document—
e.g., the type of paper, proportion of grade it was worth—into the spreadsheet or
database?
● Who will be responsible for any data analyses?
●
●
●
●
Who will write summary reports about what you learned?
How will findings be reported out to the readers and the rest of the campus
community?
What will you do to engage people on campus who are responsible for faculty
development with the findings?
Who will be responsible for examining NSSE or other surveys and assessment
data for information about factors that may influence student performance on
their papers?
○ We recommend that such information should be included in any reports.
The goal of assessment is improvement; hence, reports about student
learning should be connected with evidence about the practices and
conditions that may enhance learning.
We would like to thank Jon Christy (Luther College) and Carol Rutz (Carleton College)
for their advice in putting together this document. Any mistakes or poor suggestions are
ours alone.
The following documents from the Teagle collaborative project “Measuring Intellectual
Development and Civic Engagement through Value-Added Assessment” with Augustana
College, Alma College, Gustavus Adolphus College, Illinois Wesleyan University,
Luther College, and Wittenberg University are included at the end of this document as
examples:
●
●
●
●
Writing rubric
Critical thinking rubric
Letter to Luther College seniors requesting papers
Letter to Luther College faculty requesting papers
From Teagle collaborative “Measuring Intellectual Development and Civic Engagement through Value-Added
Assessment” with Augustana College, Alma College, Gustavus Adolphus College, Illinois Wesleyan University,
Luther College, and Wittenberg University.
ORGANIZATION
EVIDENCE
ARGUMENT
MAIN IDEA/
THESIS
Teagle Study Writing Rubric
Unacceptable
1
Overall position is
not evident. Topic
as expressed is
superficial, trite, or
clichéd.
Beginning
2
Overall position is
evident, but often
simplistic. Topic is
simplistic and onedimensional.
Competent
3
Overall position is
clear with a sense of
developed ideas.
Topic is interesting
and significant, but
not deeply explored
in needed areas.
Skilled
4
Overall position is
clear and
developed. Topic is
interesting,
significant, and is
engaged from
several angles.
No argumentative
structure is evident.
Ideas are
unconnected.
Argument structure
is rudimentary.
Claims are
repeated rather
than developed.
Few objections are
addressed or with
misrepresentations.
Some claims are
supported by valid,
reliable evidence,
but support is
inconsistent,
making the paper
less than
convincing.
Argumentative
structure is evident
but sometimes
simplistic. Objections
are addressed but
formulaically.
Argumentative
structure is evident.
Objections are
taken seriously and
typically addressed
fair-mindedly.
Claims are typically
supported by valid,
reliable evidence
from credible
sources, making the
paper for the most
part convincing.
Claims are almost
always supported
by valid, reliable
sources, so that the
paper is generally
convincing.
The paper is
organized, though
simplistically.
Paragraphs are
occasionally
incoherent, without
strong topics sentences and clear
development.
An introduction
and conclusion are
attempted but are
perfunctory or
formulaic. The
introduction
includes an overly
general thesis, and
the conclusion
simply restates that
thesis.
The introduction sets
a context for the
paper, states a thesis,
though in a
predictable way.
Paragraphs are
usually clear with
serviceable topic
sentences,
development, and
information. Main
points are logically
structured.
Transitions provide
coherence, but may
be formulaic (first,
second, third, etc.).
The conclusion
summarizes the
paper, but does not
explore implications
or significance.
The introduction
sets the context for
the paper, states a
clear thesis.
Paragraphs are
coherent with
strong topic
sentences, developing systematically so that
meaning is clear.
Main points are
clear and logically
structured. Transitions provide a
sense of coherence.
The conclusion
summarizes the
paper and makes
some effort to
explore
implications and
significance.
Claims are not
supported by
reliable evidence
from credible
sources, making the
paper unconvincing.
Ideas appear
unconnected.
Several paragraphs
are incoherent,
lacking clear topic
sentences and
developed by
restatement; they
may contain
irrelevant
information. Paper
shows serious lack
of unity and
coherence.
Introduction and /or
conclusion may be
weak, trite, or
nonexistent.
June 2006
Exemplary
5
Overall position is
well-articulated
and developed with
appropriate
evidence Topic is
interesting,
significant and
intellectually
challenging with
multiple facets
addressed.
Argumentative
structure is clearly
evident. Objections
are taken seriously
and addressed fairmindedly and with
great skill.
Claims are
supported by
reliable, valid
evidence from
credible sources
and effectively
synthesized in a
very convincing
manner.
The introduction
skillfully captures
reader attention
and sets the context
for the paper. Paragraphs are coherent
with apt topic
sentences,
developed so
meaning is especially clear and easy
to follow. The
thesis is clear and
effective. Main
points are clear and
logically
structured. Transitions provide a
sense of coherence. The
conclusion summarizes and explores
implications and
significance.
READABILITY
CONVENTIONS
Numerous errors in
grammar, usage,
spelling and
punctuation
seriously impede
meaning. Necessary
documentation is
missing.
OVERALL
IMPRESSION
Awkward phrasing,
unskilled or
inappropriate
voice/tone, and
unsophisticated
and/or imprecise
vocabulary hinder
understanding.
The writer is unable
to construct and
present a significant
position.
Paragraphing and
overall organization
hinder
effectiveness. Ideas
are asserted rather
than developed or
are seriously
underdeveloped.
Language is
substandard with
errors at the
sentence level.
Below college-level
writing
Awkward phrasing,
unskilled or inappropriate voice/
tone, and
unsophisticated
and/or imprecise
vocabulary distract
from the paper’s
ideas.
Several errors in
grammar, usage,
spelling, and
punctuation distract
the reader and
impede meaning.
Problems with
needed
documentation
exist.
The writer presents
a significant
position that falls
short of being
convincing.
Frequent though
not pervasive
problems at the
sentence-level.
Paragraphing is
inconsistent.
Overall
organization and
support are
rudimentary.
Remedial l college
level writing.
Phrasing is generally
competent;
voice/tone and
vocabulary are
generally suitable for
the paper’s ideas and
only occasionally
work against its
ideas.
Errors in grammar,
usage, spelling, and
punctuation are
noticeable, but do not
seriously impede the
reader.
Documentation is
usually correct.
Clear phrasing,
appropriate shifts
in voice and tone,
and vocabulary
enhance the
paper’s ideas.
Skilled phrasing,
appropriate shifts
in voice and tone,
and apt word
choice create an
inviting paper.
There are
occasional errors in
grammar, usage,
spelling, and
punctuation that do
not impede the
reader.
Documentation of
sources is correct.
There are very few
or no mechanical
errors in the paper.
Documentation of
sources is correct.
The writer presents a
significant position
that is generally
convincing, but has
some weak-nesses.
Paragraphs are
typically organized
and add to the
development of ideas.
Support is good, but
sometimes
inadequate.
Organization is
evident but
sometimes simplistic.
There are occasional,
but not overly
distracting, sentencelevel errors,
Acceptable if
sometimes uninspired
college work.
The writer presents
a significant and
thoughtful position
that is for the most
part convincing.
Paragraphs are well
organized and
contribute to the
development of
ideas. Support is
good with
infrequent soft
spots. Organization
is clear. Sentencelevel errors are
infrequent. Skilled
college-level
writing.
The writer presents
a significant and
thoughtful position
that is convincing
and at times
thought-provoking.
Paragraphs are
skillfully organized
and add to the
development of
ideas. Support is
sound with rare or
no soft spots.
Sentence-level
errors are rare to
non-existent.
Strong collegelevel writing.
For this paper, how appropriate was the rubric?
Not appropriate
Somewhat
appropriate
Mostly appropriate
Very appropriate
*Inter-rater reliability of .938 (Cronbach’s alpha) for faculty reader scoring at Alma College (June 2006)
Teagle Scoring Guide for Critical Thinking
May 2007
This guide sets out several habits of mind that students must develop in order to succeed at critical thinking in their academic work. Academic work
includes but is not limited to such activities as argumentation; interpretation; developing proofs, theorems, and case statements; model building;
analysis; and creative projects. Faculty in various disciplines are invited to adapt this scoring guide to fit the contexts of their disciplines, the courses
they teach, and the assignments that they present in those courses. Each habit of mind identified below is accompanied by descriptors for a range of
performance exhibited in actual student work.
1. Problem: Recognizes from readings, experience, data, or observation a problem, question, or issue to address.
Emerging
1
2
Does not identify a problem, question, or
issue or identifies an inappropriate
problem, question, or issue. May be
confused or represent the problem,
question, or issue inaccurately. Does not
establish problem’s significance/relevance.
Developing
3
4
Identifies a problem, question, or issue and
presents it clearly, if simply. May
recognize some of the nuances, but does so
inconsistently. Acknowledgment of
problem’s significance/relevance is too
simple.
Mastering
5
6
Identifies the main problem, question, or
issue, as well as embedded or implicit
ones; and identifies them clearly,
addressing their relationships to each other.
Recognizes the nuances of the problem,
question, or issue, including the relevance/
significance.
2. Central/Main Idea: identifies and presents an approach and position to address the problem/issue raised.
Emerging
Developing
Mastering
1
2
The project displays no central approach or
controlling idea, or that idea remains
unimportant to the work.
3
4
Presents an approach/controlling idea that
addresses the issue or problem, though
sometimes in an unsophisticated or
simplified way.
5
6
Presents an approach/controlling idea that
addresses the issue or problem raised, in a
complex, sophisticated way.
From Teagle collaborative “Measuring Intellectual
Development and Civic Engagement through ValueAdded Assessment” with Augustana College, Alma
College, Gustavus Adolphus College, Illinois Wesleyan
University, Luther College, and Wittenberg University.
3. Perspective(s): Identifies and considers salient perspective(s), position(s), and context(s).
Emerging
Developing
Mastering
1
2
Deals superficially with a single
perspective. Even when applicable, fails to
acknowledge other possible salient
perspectives. Lacks a sense of fairness and
open-mindedness. May not be aware of
having a perspective or may not present an
appropriate perspective.
3
4
Maintains a single perspective. When
applicable, acknowledges other possible
salient perspectives. Is mostly fair and
open-minded. When appropriate to subject,
student demonstrates some awareness of
his or her own perspective and its influence
on the approach to the task.
5
6
Skillfully conveys a single perspective and,
when applicable, addresses and
accommodates all other salient perspectives
well. Is consistently fair and open-minded.
When appropriate to subject, student shows
a deep and detailed awareness of his or her
own perspective and its influence on the
approach to the task.
4. Supporting Data/Evidence: Includes supporting data/evidence and assesses its quality.
Emerging
1
2
Provides very little data/evidence to
support its position, or the data/evidence
selected is low quality or irrelevant. Does
not seriously assess support, to distinguish
among fact, opinion, and value judgments.
Developing
3
4
Provides data/evidence to support its
position; some data/evidence is low
quality or irrelevant. Attempts, though
sometimes mistakenly, to assess support,
to distinguish among fact, opinion, and
value judgments.
Mastering
5
6
Provides ample evidence to support its
position; almost all data is high quality and
clearly relevant. Clearly assesses support,
distinguishing among fact, opinion, and
value judgments.
5. Depth of thought: Deeply engages in the work.
Emerging
1
2
Demonstrates little engagement
with the work. The treatment remains
shallow, over-simplified, and limited in
focus and usefulness. Exhibits little or no
ability to deal with ambiguity.
Developing
3
4
Engages the work, in places pushing
the treatment to greater depth and
complexity, approaching it with a spirit of
exploration, or expanding the focus as
needed to do the work justice. Level of
complexity throughout is adequate but in
need of greater development. Acknowledges
ambiguity.
Mastering
5
6
Engages the work fully, pushing to achieve
full depth and complexity, fully exploring
and where necessary expanding the
boundaries of the work. Treatment is
complex, sophisticated, imaginative, and
nuanced. Acknowledges and effectively
manages ambiguity.
6. Reasoning: employs logic to construct a cogent argument/statement.
Emerging
1
2
Work has obvious flaws in logic/analysis.
Developing
Mastering
3
4
5
6
Work is generally sound, but has some flaws Work is very sound, with no flaws or only
in logic/analysis.
minor flaws in logic/analysis.
7. Development: Strategically organizes and styles the work.
Emerging
1
2
Employs seemingly random and/or
inappropriate organization, and, where
applicable, genre, and/or medium.
Developing
3
4
Makes some sensible choices of
organization. Where applicable, selects an
appropriate genre and/or medium.
Mastering
5
6
Makes appropriate choices of organization
and, where applicable, genre and/or medium.
While coherent, also engages with and tests
rules or boundaries of the work.
8. Conclusions/Consequences: identifies and assesses strengths and weaknesses of choices, conclusions, implications, and consequences.
Emerging
Developing
Mastering
1
2
Fails to identify conclusions, implications,
and consequences of the issue or the key
relationships between the other elements of
the problem, such as assumptions, contexts,
data, evidence, organization, genre, medium.
Seems unaware of limits of evidence and
conclusions.
3
4
Identifies some conclusions, implications,
and consequences and/or fails to spell out
conclusions, implications, and consequences
as clearly as possible. Some awareness of
the limits of evidence and conclusions.
5
6
Identifies and discusses conclusions,
implications, and consequences considering
assumptions, context, data, evidence,
organization, genre, medium. Objectively
reflects upon their own assertions, including
limits of evidence and conclusions.
9. Holistic Rating. What rating would you give this work as a whole?
Emerging
1
(Absent )
Developing
2
3
Mastering
4
5
Emerging (Recognizable) (Inconsistently Developing (Competent but (Sometimes Mastering
Competent)
unsophisticated) sophisticated)
1
3
4
5
2
6
(Frequently
sophisticated)
6
Letter to Luther College seniors requesting papers for the Teagle
collaborative project "Measuring Intellectual Development and
Civic Engagement through Value-Added Assessment."
Dear xxxx,
Could you use $20.00? If so, read on . . .
We are writing to ask your help to improve the teaching and learning about writing and critical thinking at Luther. You
may or may not remember that as a first-year student, you contributed to a study that Luther participated in that was
designed to assess student growth in these two areas. You submitted your first and last Paideia I papers, and some of
you also took a timed test over in Olin during September of 2005 (the CLA—Collegiate Learning Assessment). Luther
was one of six colleges participating in the study, and the participation rate from Luther students was the highest among
all the colleges.
The point of this study to assess student growth, and we are writing the final chapter now that you are seniors. We
hope that you are willing to let us see a senior-year sample of your writing and critical thinking, by submitting another
paper and taking the CLA during the spring semester.
The type of paper we are looking for is, first and foremost, one that you are reasonably proud of as a writer, which
means it is a paper that represents serious thought and preparation for—not something you dashed off the night before
it was due! It should be a thesis-driven paper, with evidence beyond personal experience, and include a section that
lists sources (i.e., a bibliography). The length we are looking for is anywhere from 5-12 pages. Here are some
additional criteria for the paper:
addresses a problem, question, or issue;
identifies and presents the writer’s own perspective and position on the issue;
demonstrates a willingness to make connections between student’s existing knowledge as well as common
knowledge and knowledge gained from sources;
considers multiple perspectives, alternative approaches, ambiguity;
acknowledges materials from other sources;
makes effective judgments and/or choices in developing the work.
We’re writing now so that we can capture a paper from your senior year, either from fall or your anticipated spring
semester schedule. We will follow up this letter with an e-mail request asking whether you choose to participate. If
you respond yes, we will be in contact with you to help you determine the best paper to submit.
We realize this will take time and energy for you to help us out, so it’s reasonable for you to ask, “well, what’s in it for
me?” Two things: if you submit a paper and take the CLA test in the spring of 2009, you’ll receive $20.00, and if your
CLA score is in the upper 50% of Luther students, you’ll receive an additional $20.00. In addition, the college writing
director, Nancy Barry, will provide you with a statement about this study that you can use as part of your resume for
perspective employers—to show actual evidence that you are someone who cared enough about your writing and
critical thinking to participate in a long-term study is something that is likely to impress future employers, and we’re
happy to authenticate to them that you did this.
Thanks very much for thinking of this, and we genuinely appreciate your time and help in providing these materials.
Your work is completely confidential when it is assessed by readers—there are no identifying marks on it at all (either
your name or Luther College), and we will ask for your permission before including your paper and CLA scores in the
pool.
We’ll be sending a follow-up e-mail very shortly with a form that you can use to reply. For now, please accept our
appreciation for your previous participation, and your help in gathering this year’s work as well.
Sincerely
Nancy Barry
Luther Writing Director
Jon Christy
Director of Assessment/Institutional Research
To:
From: Jon Christy, Eric Westlund, Nancy Barry
Date: February 16, 2007
Re: Collecting a course paper from select junior and senior students
Letter to Luther College faculty requesting
papers for the Teagle collaborative project
"Measuring Intellectual Development and
Civic Engagement through Value-Added
Assessment."
Luther College, as part of a consortium of six colleges, is participating in a Teagle Foundation research
study of how various institutional characteristics relate to student learning. The three areas of study are
writing, critical thinking/analytical reasoning, and civic engagement. Participating colleges in the Teagle
research study are Alma College, Augustana College, Gustavus Adolphus College, Illinois Wesleyan
University, Luther College, and Wittenberg University. From the Luther faculty, Eric Westlund, Nancy
Barry, and Craig Mosher are assisting with the study. This year, year two of the grant, the Teagle study
focus is on critical thinking/analytical reasoning. Faculty participants have decided the best process to
use to review critical thinking for the purposes of this study is via a collection and review of student
papers.
We are asking for your help in collecting papers. You have been selected because you are teaching
either a Paideia Capstone course or a “writing-intensive” course in which there are junior and/or senior
students enrolled. Please be assured that these papers are being collected as an assessment tool for
evaluating student critical thinking/analytic reasoning, not your course or your role as the instructor of the
course. All names of students and instructors will be removed from any material before being reviewed,
and students do have the right to not participate in the study if they choose. Our hope, however, is that
they will recognize that this study will be useful to the college and will be willing to submit their work. We
are asking instructors to help with the collection of the papers because it provides students with a larger
context for participating, and (we hope) makes the collection of the papers a bit more orderly.
Here’s what we need: for each of the students listed below (these students are junior or senior
students on your course roster who started at Luther as first-year students), one clean copy of an
argumentative or case-building paper 5-15 pages long, along with the accompanying paper
assignment. These papers should be thesis driven, with evidence beyond the personal, and
include a section listing paper sources (i.e., bibliography). Papers unlikely to serve our purposes
are: fiction (short stories, novel chapters, poems, etc.); personal narratives; close readings of
literary texts that do no other contextualizing or that do not use sources; literature reviews; topic
proposals; lab reports; or proofs.
Student Name
Student Name
Student Name
Student Name
Student Name
Student Name
Student Name
Student Name
Student Name
The Teagle Study Critical Thinking rubric (in its current draft format) consists of the following categories:
Recognizes from readings, experience, data, or observation a problem, question, or issue to
address.
Identifies and presents the student’s own perspective and position.
Demonstrates a willingness to make connections to student’s existing knowledge as well as
to common knowledge and knowledge gained from source materials.
Considers multiple perspectives, alternative approaches, ambiguity.
Engages with the task at hand.
Acknowledges intellectual debts
Makes effective judgments and/or choices in developing the work.
To make this as easy as possible, we envision that when students hand in an assignment matching the
above requirements, they would simply provide two copies of the paper—one for you and one with a
signed consent form. When you have the entire group of papers collected, you would attach the paper
assignment to each and send it to Jon Christy, 234B Dahl Centennial Union.
If you have any questions about the type of paper that might be appropriate, please contact Eric Westlund
at wester01@luther.edu (1711) or Nancy Barry at barrynan@luther.edu (1591). If you have questions
about the format or procedures for the study, contact Jon Christy at chrijo01@luther.edu (1016).
Thanks very much for your help and participation in this work!