DOLCE:
Developing Off/On-Line
Computer Ethics
http://csethics.uis.edu/dolce/
partially supported by the
National Science Foundation
CCLI-DUE 9952841
1
Grading Essays in Computer Science:
Rubrics Considered Helpful
Barbara Moskal
Keith Miller
Laurie King
Mathematical & Computer
Sciences
Colorado School of Mines
Golden, CO 80401
Computer Science
Univ. of Illinois-Springfield
P.O. Box 19243
Springfield, IL 62794-9243
Math & Computer Science
College of the Holy Cross
One College Street
Worcester, MA 01610
2
Rubrics Considered Helpful:
Outline
• Brief overview
• Brief practice grading session
3
Computer Science Ethics:
Important Issues
• Responsible Computer Use
• Harm from Unreliable
Software/Malicious Attacks.
• Professional Decisions
• CSAB and ABET
4
Classroom Challenge:
Assessing Students
• Writing is important
– What?
– So What?
– Now What?
• Grading Essays is different from
grading in a technical course
5
Rubrics:
Descriptive Scoring Schemes
• Pre-Defined Scheme makes
Evaluation more Objective
• Holistic versus Analytic
• Analytic: Separate Specific
Criteria of Interest
6
Rubrics:
Details and Definitions
• Task Specific versus General Criteria
• Validation: Evidence to support that
the manner in which assessment
information is used is appropriate.
• Rater Reliability: Consistency of
Assessment Scores between raters.
7
Rubrics: An Example
Written Communication (General Criteria)
Mechanical (Sentence Level)
Inadequate (1 pt.): Sentences
and paragraphs are difficult to
read and understand due to
poor grammar or mechanics.
Needs Improvement (2 pts.):
The essay contains numerous
grammatical and mechanical
errors.
Adequate (3 pts.): The essay
contains minimal grammatical
or mechanical errors.
Excellent: (4 pts.): The essay
is clear and concise and the
grader found no grammatical
or mechanical errors.
Clarity (Paragraph Level)
Inadequate (1 pt.): There appears
to be no organization of the
essay’s contents.
Needs Improvement (2 pts.):
Organization of the essay is
difficult to follow due to a
combination of inadequate
transitions and a rambling format.
Adequate (3 pts.): The essay can
easily be followed. A combination
of the following is apparent: Basic
transitions are used. A structured
format is used.
Excellent: (4 pts.): The essay can
easily be followed. A combination
of the following is apparent:
Effective transitions are used. A
polished format is used.
Organization (Whole Essay)
Inadequate (1 pt.): Thoughts
in the writing do not appear
organized or logical. No
organization visible.
Needs Improvement (2 pts.):
Some thoughts are
discernible, but the essay
confuses the reader. It is
unclear the direction the essay
will take.
Adequate (3 pts.): Most
thoughts appear logically, but
the essay is listless, flat, or
slightly muddled.
Excellent (4 pts.): Essay
presents its ideas eloquently,
logically, and clearly. Writer
leads the reader gracefully.
8
Rubrics: An Example
Technical Content (General Criteria)
Followed the Assignment's Directions
Inadequate (2 pts.): The paper has
no apparent relation to the directions
of the assignment.
Needs Improvement (4 pts.): Some
of the paper follows the directions.
Adequate (6 pts.): Most of the paper
follows the directions.
Excellent: (8 pts.): The paper follows
the directions precisely. (i.e. the
sections are labeled, directions for
finding the article are clear, all
required information, etc.)
Explains the Technical Issue
Inadequate (2 pts.): Names the
technical issue, but technical issue is
not explained.
Needs Improvement (4 pts.):
Attempts to explain the technical issue,
but is misleading or inaccurate.
Adequate (6 pts.): Technical details
are accurate, but either incomplete or
rambling.
Excellent: (8 pts.): Technical
explanation is both concise and
complete in technical explanation.
Leads gracefully into ethical
discussion.
9
Rubrics: An Example
Technical Content (General Criteria)
Stakeholders Identified and Values at Stake
Explained
Inadequate (2 pts.): Does not identify who is
impacted by the ethical dilemma or how they are
impacted. Does not explain the values at stake.
Needs Improvement (4 pts.): Specifies either
who is impacted by the ethical dilemma OR how
they are impacted, but not both. Attempts to
explain the values at stake, but misses the mark.
Adequate (6 pts.): Specifies who is impacted by
the ethical dilemma AND how they are impacted.
Attempts to explain the values at stakes, but
leaves out important points.
Excellent: (8 pts.): Specifies who is impacted by
the ethical dilemma AND how they are impacted.
Clearly explains the important values at stake
and why they are ethically significant.
Conclusion: Justified Preferred
Position
Inadequate (2 pts.): Doesn’t pick a
position.
Needs Improvement (4 pts.): Picks a
position, but doesn’t justify it.
Adequate (6 pts.): Picks and tries to
justify position; Argument is not
convincing OR a convincing
justification is given, that has nothing
to do with the analysis stated.
Excellent: (8 pts.): Essay provides a
persuasive argument that clearly
supports the position. Even a reader
who disagreed with the position
before finds her/himself thinking
about the issue more carefully.
10
Rubrics: An Example
Technical Content (Task Specific Criteria)
Uses Utilitarian Theory in Analysis
Inadequate (2 pts.): None
discernable.
Needs Improvement (4 pts.): The
paper mentioned utilitarian ideas, but
they weren't used well.
Adequate (6 pts.): The paper
included an adequate utilitarian
analysis.
Excellent (8 pts.): The paper
included an unusually original or
particularly revealing utilitarian
analysis.
Uses Deontological Theory in Analysis
Similar to Utilitarian above.
Uses an Analogy
Inadequate (2 pts): None discernable
Needs Improvement (4 pts): The
paper included an analogy, but it was
not used well.
Adequate (6 pts): The paper included
an appropriate analogy (i.e. similarities
and differences are explained with
respect
to
the
original
situation/activity).
Excellent (8 pts): The paper included
an appropriate analogy which was
either unusually original or particularly
revealing.
11
Rubrics: Results
Highest
CONTENT:
Possible
Followed Directions
8
Technical Details
8
Stakeholders
8
Analysis: Analogy
8
Analysis: Utilitarian
8
Analysis: Deontological 8
Conclusion
8
WRITING:
Clarity
4
Mechanics
4
Organization
4
OVERALL SCORE:
100%
ESSAY 1
7.8
7.0
5.4
5.0
ESSAY 2
7.6
6.2
5.4
5.6
6.4
6.2
6.6
3.5
2.5
3.4
77%
3.8
3.4
3.4
82%
12
Rubrics: Conclusions
• Helps faculty member organize
thinking
• Helps students understand how
they will be evaluated
• Produces higher quality work
• Tool to track student development
13
•
•
•
•
•
•
Keith Miller
miller.keith@uis.edu
Laurie King
LA@cs.holycross.edu
Barbara Moskal bmoskal@mines.edu
Tracy Camp
tcamp@mines.edu
Deborah Johnson dgj7p@virginia.edu
Chuck Huff
huff@stolaff.edu
(supported by NSF grant DUE-9980768)
DOLCE home page:
http://csethics.uis.edu/dolce/
14
Activity: Assignment
Write a sentence that expresses an analogy
between a technology that includes computing and
a technology that does not include computing.
Your sentence should illustrate something germane
to ethics about the computer technology. Try to
write a sentence that makes the reader think; don't
restate the obvious. The sentence can include both
similarities and differences, but should emphasize
an aspect that has ethical significance.
15
Activity: Rubric
Inadequate (1
point)
Mechanical Misspelling and
Details
bad grammar.
Adequate (2
points)
Misspelling or
bad grammar.
Excellent (3
points)
No obvious
mechanical errors.
Analogy
None
discernable or
unrelated
analogy.
Analogy
An unusually
appropriate to the revealing analogy.
assignment.
Ethical
Content
No discernable
ethical content.
Some ethical
content.
Ethical content
impressive for a
single sentence.
16
Activity: Sentences
1. The Internert is like a bomb because it blasts away all competing
media.
2. MP3 files is similar to copyrighted books.
3. Just as the Model T Ford made autos accessible to many more
people, Palm Pilots will make computing accessible to many
more people, tearing down the digital divide.
4. Junk email is worse than junk mail from the U.S. Post Office
because junk emails are cheaper for the sender and more
annoying for the receiver.
5. Listening to a hacker on the topic of computer ethics is like
listening to Bill Gates talk about open source software. Self
justification is rarely enlightening.
17