The ARC Assignment Profile Assessment Rubric for Critical Thinking The Assessment Rubric for Critical Thinking (ARC) is a global rubric template developed for the College to provide a snapshot view of how student learning is being affected by the critical thinking QEP initiative. It is designed to assess a variety of student projects from a critical thinking perspective. For example, students in a composition class may be asked to complete a paper on a specific topic. This ARC rubric template can evaluate the student’s use of critical thinking skills in the development of the paper as opposed to specifically evaluating the quality of student’s writing skills. The ARC rubric template is designed to be flexible enough to address a number of student project modalities including written and oral communications. Validating the ARC The development of any quality rubric is a long and arduous process. The initial version of the ARC was developed by the College’s current team of faculty champions in conjunction with QEP staff and resources from their various disciplines. The initial administrations were just the beginning of the rubric development and validation process. Refinement of this initial instrument has included thorough reviews by faculty and assessment experts from around the college as well as a preliminary tests of the instrument on sample of discipline-specific, course projects. Faculty champions will determine the quality and usability of the rubric through the rating of student artifacts and will recommend modifications as needed. The ARC has been piloted and results were analyzed from a quantitative as well as a qualitative perspective to establish the quality, reliability, and validity of the assessment instrument. Based on these validation results, some additional refinements and modifications were made to the instrument to ensure the quality of the final standardized instrument. Rubric results will be reevaluated after each administration, and additional refinements and modifications may be made to the instrument as the assessment development and validation is intended to be an on-going dynamic process. Assignment Profile for ARC Page 1 of 5 Last Revised: 05/20/2010 ARC Assignment Profile The ARC Assignment Profile is designed to provide consistency and accuracy in the evaluation of the ARC as well as provide guidelines for the use of the assessment at the course level. The ARC is essential a ‘tool’ to evaluate critical thinking, but for a tool to be effective it must be used in the correct situation or ‘job.’ It would be inefficient to use a machete to conduct heart surgery. The purpose of the ARC Assignment Profile is to outline the most appropriate course assignment. 1. Participating faculty should have one assignment during the course that can be evaluated using the ARC scoring rubric. The course assignment could be a graded homework assignment or a major assessment for the course. 2. The course assignment for the ARC should include all of the elements of the rubric and should be aligned with the task outlined for each element. Assignments that only evaluate some of the elements or are not aligned with the specific ARC tasks will be considered incomplete. 3. Faculty may add additional discipline specific rubric elements (such as grammar and punctuation in a composition class), but should maintain the ARC elements as listed. 4. Students should be provided a copy of the assignment rubric (ARC and any additional discipline specific elements). The specific elements and tasks include: 1. Communication: Define the problem in your own words. 2. Analysis: Compare & contrast the available solutions within the scenario. 3. Problem Solving: Select one of the available solutions and defend it as your chosen solution. 4. Evaluation: Identify the weaknesses of your chosen solution. 5. Synthesis: Suggest ways to improve/strengthen your chosen solution (may use information not contained within the scenario). 6. Reflection: Reflect on your own thought process after completing the assignment. a. “What did you learn from this process?” b. “What would you do differently next time to improve?” 5. The evaluating scenario (selected or created) should be stated in such a manner to allow the student to address each of the tasks. The QEP team is willing to assist you with the creation of the scenario or identify possible sources of existing scenarios that could be used. A very basic sample scenario is provided at the end of the document. Assignment Profile for ARC Page 2 of 5 Last Revised: 05/20/2010 SPC’s Assessment of Critical Thinking (ARC) Scoring Template Rater (scorer) name: _____________________________Paper ID: _____________________Date: ____________________ Performance Element I. Communication Define problem in your own words. II. Analysis Compare & contrast the available solutions. III. Problem Solving Select & defend your chosen solution. Exemplary (4) Identifies the main idea or problem with numerous supporting details and examples which are organized logically and coherently. Proficient (3) Identifies the main idea or problem with some supporting details and examples in an organized manner. Developing (2) Identifies the main idea or problem with few details or examples in a somewhat organized manner. Emerging Not Present (1) (0) Identifies the main Does not identify idea or problem the main idea or poorly with few or no problem. details or states the main idea or problem verbatim from the text. Uses specific inductive or deductive reasoning to make inferences regarding premises; addresses implications and consequences; identifies facts and relevant information correctly. Thoroughly identifies and addresses key aspects of the problem and insightfully uses facts and relevant evidence from analysis to support and defend potentially valid solutions. Uses logical reasoning to make inferences regarding solutions; addresses implications and consequences; Identifies facts and relevant information correctly. Uses superficial reasoning to make inferences regarding solutions; Shows some confusion regarding facts, opinions, and relevant, evidence, data, or information. Makes unexplained, unsupported, or unreasonable inferences regarding solutions; makes multiple errors in distinguishing fact from fiction or in selecting relevant evidence. Identifies and addresses key aspects of the problem and uses facts and relevant evidence from analysis to develop potentially valid conclusions or solutions. Identifies and addresses some aspects of the problem; develops possible conclusions or solutions using some inappropriate opinions and irrelevant information from analysis. Identifies and Does not select and addresses only one defend a solution. aspect of the problem but develops untestable hypothesis; or develops invalid conclusions or solutions based on opinion or irrelevant information. Assignment Profile for ARC Page 3 of 5 Does not analyze multiple solutions. Score 4 3 2 1 0 N/A Comments: 4 3 2 1 0 N/A Comments: 4 3 2 1 0 N/A Comments: ARC Last Revised: 11/13/2008 Rater (scorer) name: _____________________________Paper ID: _____________________Date: ____________________ Performance Element IV. Evaluation Identify weaknesses in your chosen solution. V. Synthesis Suggest ways to improve/strengthen your chosen solution. VI. Reflection Reflect on your own thought process. “What did you learn from this process?” “What would you do differently next time to improve?” Exemplary (4) Insightfully interprets data or information; identifies obvious as well as hidden assumptions, establishes credibility of sources on points other than authority alone, avoids fallacies in reasoning; distinguishes appropriate arguments from extraneous elements; provides sufficient logical support. Insightfully relates concepts and ideas from multiple sources; uses new information to enhance chosen solution; recognizes missing information; correctly identifies potential effects of new information. Identifies strengths and weaknesses in own thinking: recognizes personal assumptions, values and perspectives, compares to others’, and evaluates them in the context of alternate points of view. Assignment Profile for ARC Proficient (3) Accurately interprets data or information; identifies obvious assumptions, establishes credibility of sources on points other than authority alone, avoids fallacies in reasoning; distinguishes appropriate arguments from extraneous elements; provides sufficient logical support. Developing (2) Makes some errors in data or information interpretation; makes arguments using weak evidence; provides superficial support for conclusions or solutions. Accurately relates concepts and ideas from multiple sources; uses new information to enhance chosen solution; correctly identifies potential effects of new information. Identifies strengths and weaknesses in own thinking: recognizes personal assumptions, values and perspectives, compares to others’, with some comparisons of alternate points of view. Inaccurately or incompletely relates concepts and ideas from multiple sources; shallow determination of effect of new information on chosen solution. Identifies some personal assumptions, values, and perspectives; recognizes some assumptions, values and perspectives of others; shallow comparisons of alternate points of view. Page 4 of 5 Emerging (1) Interprets data or information incorrectly; Supports conclusions or solutions without evidence or logic; uses data, information, or evidence skewed by invalid assumptions; uses poor sources of information; uses fallacious arguments. Poorly integrates information from more than one source to support chosen solution; Incorrectly predicts the effect of new information on chosen solution. Not Present (0) Does not evaluate data, information, or evidence related to chosen solution. Identifies some personal assumptions, values, and perspectives; does not consider alternate points of view. Does not reflect on own thinking Score 4 3 2 1 0 N/A Comments: Does not identify new information for chosen solution. 4 3 2 1 0 N/A Comments: 4 3 2 1 0 N/A Comments: ARC Last Revised: 11/13/2008 Sample Scenario: Deer Overpopulation: Three teenagers were seriously injured in a car accident when swerving to avoid a deer on a two-lane road near a small, rural town in Florida. The residents of the town have seen more and more deer enter the town’s populated areas over recent years. Local law enforcement has been called numerous times this year to remove the animals from backyards and neighborhood streets, and one deer even caused considerable damage as it entered a restaurant in town. The mayor has been charged by the city leaders to keep the town residents safe. Local crops have even been damaged by the animals. Some long time residents have requested that the hunting season and catch limits be extended in order to reduce the deer population. One city leader even proposed that the city purchase electronic devices to deter the deer from entering populated areas. Health concerns have recently been elevated as three deer carcasses were found at the edge of town and local law enforcement suspect that the animals had been poisoned. Assignment Profile for ARC Page 5 of 5 Last Revised: 05/20/2010