Using Rubrics to Respond to and Evaluate Student Work What is a rubric? A rubric is a set of scoring guidelines for evaluating students’ work. A rubric indicates the criteria you use to judge student performance, and distinguishes different levels of quality in student work. Why bother with rubrics? There are three good reasons to use rubrics. 1. Good rubrics clearly identify the instructor’s expectations and how to meet those expectations. They help define the quality of work expected of students. 2. They can help students become more thoughtful judges of the quality of their own and others’ work. When used to guide self and peer assessment, the rubric can help to identify strengths and weaknesses in students’ own and one another’s work. 3. Rubrics can reduce the amount of time instructors spend responding to and evaluating student work. Instructors can use the language and categories of the rubric to give students feedback on the quality of their work. This can reduce the amount of time trying to explain the flaws and strengths in the work. Moreover, if students use rubrics to self-correct or peer-correct their work, instructors will receive more highly developed work. How to create a rubric. It takes time to develop a good rubric, but that’s time saved later on in the grading process. Here are a few pointers to get started. 1. Pick an assignment that you use regularly so you can refine and use the rubric again in the future. 2. Identify models of good and poor performance on the assignment. Pick a few of the best and worst pieces of student work. 3. Make a list of the characteristics or traits shared by the good pieces and a list of characteristics shared by the poor pieces. The goal is to develop a list of the criteria that distinguish good and poor work: a list of the dimensions that define quality in the work. 4. Define gradations of quality. The characteristics of the best and worst work provide anchor points. Fill in the middle levels of quality. You can make as many gradations or levels as you want. The trick is to identify clearly what distinguishes the “Best” work from the “Next Best” work and then the work that is next best and so on. 5. Create a rubric table. Most rubrics have a list of evaluation criteria, followed by descriptors of the characteristics of traits that constitute strong or weak performance in each level of quality. 6. Determine the scoring system. If you give a score for each criterion, you may want to weight the criterion scores differently, or you may simply use the criteria scores as indicators of performance quality without having them add up to an overall grade. (See examples.) 7. Practice using the rubric and revise as needed. First, you want to make sure that it does a satisfactory job of distinguishing levels of performance. Does it include all the essential dimensions that define quality? You can tell it is working well if you can read a set of student work and say with confidence that all the work in the top category really belongs there—that all the pieces have the same level of quality on the critical dimensions. If not, something is wrong. What if you apply the rubric and then discover that work in the same category actually looks different on the critical dimensions? You may not be clear yourself about what the actual dimensions are (e.g., you could be using some criteria that you have not yet made explicit) or you may be applying the criteria inconsistently (e.g., you may need to define the criteria more carefully so that you can use them without thinking twice about what they mean). How to use a rubric. 1. Distribute the rubrics to your students well in advance of the final due date. 2. Require the rubric to be attached to the paper to be evaluated. 3. Indicate the level of performance on each of the criteria; circle the appropriate descriptors for weaknesses and strengths; assign an overall grade—perhaps giving NO comments to the paper if it has received feedback during the composing process. Terry Beck & Bill Cerbin Follow Up to Responding to and Evaluating Student Work UWL Conference on Teaching & Learning, August 28, 2007 4. Use the rubric to give feedback on drafts as well to indicate areas of strength and weakness on final performances. have students self-assess using the rubric have peers fill out the rubric in a peer review use it yourself to give a this-is-the-grade-the-draft-would-get-if-submitted-in-thiscondition grade let the rubric indicate revision work needed (if a second submission is allowed) key your marginal comments on drafts to rubric criteria (e.g., a comment about weak development is preceded by—or even collapsed into—the sign Dv? to indicate that the problem is questionable development, when “Development” is a major evaluation criterion in the rubric). 5. Revise the rubric as you use it. Every time you evaluate a set of papers, make notes on features the rubric needs (or doesn’t need) and revise it before you use it again. (Same idea as #7 in “How to Create a Rubric”—but the point is that rubrics are always evolving.) For additional information about how to design and use rubrics, see: Wiggins, G (2004). Assessment as Learning a short essay that examines the importance and characteristics of effective feedback in learning. RubiStar is a free resource for teachers that hosts examples of rubrics. Although keyed to K-12 level you can find examples relevant to the college classroom. You can also use their templates to create your own rubrics. Stevens, D. & Levi, A. (2005). Introduction to Rubrics: An Assessment Tool to Save Grading Time, Convey Effective Feedback, and Promote Student Learning. Stylus Publishing, Sterling, VA. The publisher’s website has rubric downloads. Available in Murphy Library. Walvoord, B. & Anderson, V. (1998). Effective Grading: A Tool for Learning and Assessment. San Francisco: Jossey-Bass Publisher. Has many examples and practical advice about how to design more effective ways to evaluate and grade student work. Richlin, L. (2006). Blueprint for Learning: Constructing College Courses to Facilitate, Assess and Document Learning. Sterling, VA: Stylus Publishing. Very short, practical chapters on using classroom assessment techniques and rubrics. Wiggins, G. (1998). Educative Assessment: Designing Assessments to Inform and Improve Student Performance. San Francisco: Jossey-Bass Publisher. The definitive work on how to design rubrics—but accessible to a general audience. Available in Murphy Library. Wiggins, G. & McTighe, J. (1998). Understanding by Design. Alexandria, VA: Association for Supervision and Curriculum Development. Read this book! Available in Murphy Library. Note: Any of the rubrics on the following pages may be copied, then revised and edited to make them appropriate for your courses. Terry Beck & Bill Cerbin Follow Up to Responding to and Evaluating Student Work UWL Conference on Teaching & Learning, August 28, 2007 The rubric below uses a holistic analysis of three dimensions: 1) understanding, 2) clarity and coherence, and 3) mechanical correctness. I fill out this grade sheet and return it to the student with the graded paper. I may write brief comments on the paper keyed to the dimensions of the rubric. Bill Cerbin Performance Assessment Paper Name: I used the following dimensions to evaluate your paper. The “X” along each dimension indicates a qualitative judgment—not a quantitative one. I assigned a letter grade to the paper based on overall performance. Demonstrates understanding Evaluated Holistically Integrates concepts from the course material =====================Relies on unsubstantiated opinion, anecdotes Explains how and why strategy should improve learning Thinks with course material =====================Does not explain ===================== Clarity and coherence Does not think with the course material Evaluated Holistically Uses effective transitions to guide the reader ===================== No transitions Easy to follow the pattern of organization Hard to follow Parts fit together into a coherent whole Disjointed Mechanical correctness Evaluated Holistically Few errors in grammar, punctuation, spelling ===================== Multiple errors detract from the quality of the paper Overall grade: Comments: In order to strengthen this work you need to . . . Terry Beck & Bill Cerbin Follow Up to Responding to and Evaluating Student Work UWL Conference on Teaching & Learning, August 28, 2007 This is a general rubric for evaluating a project. An alternative to this general format is to use a rubric that corresponds to the structure of the work. For example, if the assignment was a research project there could be separate criteria for the sections of the report, (e.g., introduction, method, results, discussion) Bill Cerbin Rubric for Evaluating A Major Project in a Psychology Class Dimension Content and reasoning Exemplary Information is accurate. Proficient Information is accurate. Marginal Some factual errors, misconceptions, misinterpretations. Underdeveloped Serious factual errors, misconceptions, misinterpretations Uses appropriate evidence to analyze and support ideas. Minor weaknesses in one of the following areas: Basic reasoning is adequate but may be: incomplete, faulty or weak as a result of inadequate support and evidence. Serious weaknesses in reasoning. Uses little or no evidence to support ideas. Uses inappropriate evidence. Some inadequate or inappropriate evidence. Organization, coherence, focus Formal writing Ideas from sources are well integrated. Some weak integration of ideas. Lacks effective integration of ideas. Very weak integration of ideas. Clearly explains ideas. Lacks clarity in explanations. Some ideas are fuzzy, vague. Tangled explanations. Ideas are confused. Serious confusion, tangled explanations. Pattern of organization makes it easy to follow. Minor weakness in organization. Ideas are difficult to follow. Maintains clear focus throughout. Paper loses focus at times. Basic organization is appropriate, but some sections are disconnected, lacking in focus, tangled, hard to follow. Uses appropriate headings and transitions to guide the reader. Uses appropriate headings and transitions to guide the reader. Headings and transitions may be misleading. Lacks effective headings and transitions. Well adapted to the target audience. Well adapted to the target audience Not well adapted to the audience. Writing is not adapted to the audience. Uses appropriate style, punctuation, spelling, grammar. Uses appropriate style, punctuation, spelling, grammar. Some problems in style, grammar, punctuation. Serious problems in style, punctuation, grammar, punctuation. Paper is free of errors in spelling, grammar, punctuation. Contains very few errors in spelling, grammar, punctuation. Contains some errors in spelling, grammar, punctuation. Multiple errors in spelling, grammar, punctuation. Lacks coherence and focus, tangled Terry Beck & Bill Cerbin Follow Up to Responding to and Evaluating Student Work UWL Conference on Teaching & Learning, August 28, 2007 This is a complex rubric for students in a writing class, used for both instructor evaluation and peer review. —Terry Beck Writer: ____________________________________ Trait/Criterion Focus (Fc) Rating ___ Content/ Development (Dv) Rating ___ Structure (St) Rating ___ Prose Style (Pr) Rating ___ CEP=Conventionally Emphatic Positions: beginnings & endings, clauses. Conventions (Cv) & Presentation Rating ___ Evaluation Rubric Primary Reviewer: _______________________________ Level 1 Level 3 Primary focus &/or purpose are weak. Lower levels of focus not used or not linked to purpose Key ideas not clearly focused. The focus does not move beyond generalizations. Content is inadequate to purpose&/ or audience. Contexts for purpose/subject aren’t established.. Stories are sketchy or not used. Descriptive texture is very thin No analysis is evident. Reasoning/logic is insufficient Claims are not clear &/or inappropriate Support/examples are ineffective or missing. Alternate perspectives not examined. Other audience appeals (e.g., to audience needs &/or values) are ineffective or not used. Inadequate for the audience or unrelated to purpose. The thesis/primary focus is difficult to find. The structure is difficult to follow. The focus shifts abruptly, without clarity. Genre conventions are ignored. Introduction &/or ending are missing. or weak Style is inappropriate to genre, purpose, &/or audience. Diction is poorly chosen. The syntax is often awkward &/or rambling. Uses the “Official Voice.,” very wordy. Subordination &/or coordination are weak.. Figurative language is not used or poorly used. Coherence gaps are frequent, rhythms awkward. Very weak uses of emphasis, especially CEPs. The subject &/or audience tones are inappropriate. Grammar is often awkward or incorrect. Punctuation is often missing or incorrect. Spelling & typographical errors are frequent. Paragraphing doesn’t guide focus shifts. Citations &/or documentation are incorrect, incomplete, or missing. Format (margins, font, spacing, layout, quality of print, line spacing) is inadequate. Primary focus & purpose adequate but not strong. Lower level focuses weak or not linked to primary purpose Broad to specific focusing not followed. In places, the focus is inadequately sustained. Content is barely adequate to purpose and audience. Few contexts are introduced.. Stories told without detail & depth or w/o purpose. Descriptive texture is thin, lacks variety Analysis lacks depth &/or completeness. Reasoning/logic is not always sufficient Claims are sometimes inappropriate Support/examples are insufficient or weak. Alternate perspectives only weakly examined. Other audience appeals (e.g., to audience needs &/or values) are weak. Unoriginal but adequate to purpose and audience. The thesis/primary focus is weak &/or weak thesis. The structure is mechanical or awkward. Some focus shifts are smooth, others mechanical. A few genre conventions are not followed. Introduction & ending are adequate but dull. Style is bland, unsophisticated. Diction is sometimes imprecise. Some syntax is awkward, clichés. Wordy in places or generally wordy. Subordination &/or coordination are unsophisticated. Figurative language is rarely used. Coherence is weak or mechanical, lacks rhythm. Emphasis is applied inconsistently. The subject &/or audience tones are inconsistent. Grammar is sometimes conventional or incorrect Punctuation is sometimes unconventional/unclear. Spelling & typographical errors not corrected. Paragraphing sometimes inappropriate to the focus. Citations & documentation are somewhat flawed. Format (margins, font, spacing, layout, quality of print, line spacing) is problematic in places. Beck, English Level 5 Focus & purpose are engaging. The focus expansions /narrowing are engaging, developing rich detail & imagery. Content is effective for the purpose & the audience. The writer establishes a rich set of contexts. Stories are well chosen. Descriptive texture is rich & varied The analysis is complex & integrated. Reasoning/logic is compelling Claims are clear & appropriate Support/examples are strong Alternate perspectives well incorporated. Other audience appeals (e.g., to audience needs &/or values) are effective & appropriate. Enhances purpose and audience strategies. The thesis/primary focus is commanding. The structure is intriguing. Focus shifts add meaning. Genre is used creatively. The introduction is inviting & the ending satisfying. Crafted style enhances purpose, engages audience. Diction is sophisticated.. The syntax is lively and interesting. Every word counts. Subordination &/or coordination are sophisticated. Figuratively language is used effectively. The writing flows well; rhythms are effective. Important material is effectively emphasized. The subject & audience tones are effective. Grammar is conventional and correct throughout. Punctuation is clear & effective. All spelling and typing is correct.. Paragraphing sharply defines focus. Citations & documentation are conventional & complete. Format (margins, font, spacing, layout, quality of print, line spacing) is professional. Mindful/Rhetorical Qualities______ Overall Grade _____(This a holistic score—not an averaging of individual trait ratings) Revision Grade___ Comments: ________________________________________________________________________________________________________________________ 4 Terry Beck & Bill Cerbin Follow Up to Responding to and Evaluating Student Work UWL Conference on Teaching & Learning, August 28, 2007 This is a generic rubric created as a model for use in General Education classes by a General Education sub-committee, Terry Beck with Linda Dickmeyer, Mike Durnin, and Brad Seebach Writing Assessment Rubric Criteria Rhetorical Qualities Rating _____ N=Naive W=Weak C=Competent* P=Proficient S=Sophisticated Organization Rating _____ Development of Content & Reasoning Rating _____ Critical Questions Naive Does the writing have appropriate purposes? Does the writing make a transaction with its intended audience? Does the writer present herself appropriately and effectively? Is there a strong focus? A clear train of thought? i.e. a clear, strong thesis which is consistently developed? Or a narrative that is easily followed? Is there an appropriate introduction and ending? Are relevant contexts considered and explored? (Contexts may be historical, theoretical/conceptu al, philosophical, situational, etc.) Are claims supported with adequate evidence and reasoning? Has adequate research been done to support the purpose? The purposes are unclear, not interesting, or inappropriate. The needs and/or expectations of the audience are not met. The writer’s persona is inappropriate and/or ineffective Weak The thesis is not stated or difficult to find. The train of thought is difficult to follow. Transitions between ¶s are missing. Introduction &/or ending are missing. No contexts for the purpose and subject are established. Very little support is given for claims. Reasoning is missing, weak, or confused. There is little interesting detail or imagery. Relevant sources are not used. Proficient The purposes are weak. Strategies are inappropriate or inadequate to purposes (e.g., anecdotes instead of analysis). Some audience needs & attitudes are addressed, others ignored. The writer’s persona is marginally acceptable. The thesis is weak. The train of thought is mechanical or fails to emphasize important ideas. Transitions are weak or mechanical. Introduction &/or ending are weak, ineffective. Very little context is established or explored. Weak support supplied for claims. Reasoning is undeveloped. The descriptive texture is thin: few details or images. Sources are not well introduced or integrated. Sophisticated The purposes are consistent, appropriate, & interesting. The writing anticipates & meets most audience needs & attitudes and is appropriate to the context and situation. The writer’s persona is appropriate. The thesis is clear. The train of thought is easily followed. Most transitions are smooth. Intro & ending are appropriate & useful. Significant contexts are established for both subject & purposes. Solid support is given for claims. Clear reasoning. Detail & imagery are adequate. Sources are used to give meaningful support... The purposes are compelling & carried out in intriguing ways. The writing is sensitive to the context and situation and to the needs & attitudes of the audience, guiding their understanding very well. The writer’s persona is effective. The thesis is clear & strong, commanding reader attention. The train of thought is intriguing. Transitions are insightful. The introduction is inviting & the ending satisfying. The writer establishes a rich set of contexts. Supporting evidence is well chosen and abundant. Reasoning is compelling. Detail and imagery are rich and engaging. The writer has excellent command of relevant sources and integrates them effectively. *Note: A rating of “Competent” is given when the qualities fall between the descriptors for “Proficient” and “Sophisticated” Terry Beck & Bill Cerbin Follow Up to Responding to and Evaluating Student Work UWL Conference on Teaching & Learning, August 28, 2007 Criteria Description Prose Style Prose refers to the techniques and patterns the writer uses to cast her ideas and purposes into language. Effective prose is concise, clear, coherent, emphatic, and— above all— interesting to read. This category refers both to the correctness of the grammar, spelling, and punctuation and to the quality of the format: its consistency and appropriateness to the genre and the rhetorical situation. Rating _____ Conventions & Format Rating _____ Naïve Weak Sentence structure is awkward & wordy— even painful to read. Vocabulary is poorly chosen. Coherence gaps are frequent. The style is inappropriate. Errors in conventions &/or format make reading difficult. Punctuation is often missing or incorrect. Spelling errors are frequent. Citations &/or documentation are incorrect, incomplete, or missing. Proficient Sentence structures are bland; clichés abound. Vocabulary is awkward in places. Coherence gaps occur. The style is inconsistent. Control over conventions & format is inconsistent and/or distracting. Spelling problems are recurring. Some punctuation is wrong or awkward. Errors occur in citation &/or documentation. Sophisticated Sentence structure is clear and concise. Vocabulary is precise. The writing is coherent. The style is appropriate to the genre & purpose. Appropriate genre conventions & format are used. Spelling & typographical errors are quite minimal. Punctuation is clear & conventional. Citations & documentation are mostly conventional & complete. Sentence structure and vocabulary are lively and interesting. The style is completely suited to the genre & purpose. Conventions and format are used for interesting effects. Punctuation is clear & effective. All elements of format and mechanics are handled well. Citations & documentation are conventional & complete. Overall Rating _______ (Rated on a scale of 5, with 5 the highest: this is a holistic score—not an averaging of individual criterion ratings) Comments: 8/31/04 Terry Beck with Linda Dickmeyer, Mike Durnin, and Brad Seebach Terry Beck & Bill Cerbin Follow Up to Responding to and Evaluating Student Work UWL Conference on Teaching & Learning, August 28, 2007