Draft 11/11/07 GLevy Assessment Definitions and Rubrics Some suggested definitions: Student Learning Outcome (SLO) – a description of some significant student performance or work-product that can be observed, and then evaluated against agreed-upon criteria. SLOs describe broad, overarching outcomes that the student will be able to demonstrate (in various ways) during and after participating in a focused learning experience (a course or program). SLOs “are clear and measurable statements that define what a student is able to DO at the completion of a course or program” (Fulks, 2004). These outcomes may be knowledge (cognitive outcomes), skills (general abilities or behaviors) or attitudes (affective behaviors) that demonstrate that learning has occurred, at a specified level of competency, as a result of a course or program. SLOs need to be appropriate to the students’ developmental and educational levels, and the nature of the course or program. Although the emphasis is on what the student can do upon completion of a course or program, these behaviors may sometimes be observed and evaluated throughout the student’s participation in a course or program. Some of the key features of the definition of SLOs include: 1. They focus on what the student can do or demonstrate. 2. They represent significant, overarching, key learning outcomes. That is, they reach across specific course content to describe general concepts, skills and attitudes that can be used or applied after completion of the course. 3. They use active verbs (see Bloom’s taxonomy of cognitive objectives). They do not use verbs such as understand, realize, or appreciate, because these terms cannot be directly assessed. Method of Assessment – an instructor-generated task and situation that allows the student to demonstrate his/her competency regarding the SLO. Competency Levels – an agreed-upon set of terms that designate different levels of student performance. An example of criterion-referenced, competency-based labels (without the outcome criteria that actually describe each level) would be - Exceeds Expectations, Meets Expectations, Shows Improvement, and Beginning Level. Generally, competency levels are not quantitative and cannot be assigned numerical values or averaged. Outcome Criteria – the characteristics of the performance or work product that describes a given competency level. The outcome criteria need to be complete and mutually exclusive from level to level. The student’s performance is compared to the outcome criteria and a competency level determined. Rubric – an approach to assessing cognitive or complex skills. A rubric usually involves a matrix of competency levels against knowledge or skill components, with the boxes of the matrix describing the criteria for determining whether the observed performance meets the designated competency level. A blank sample rubric format for a single learning outcome and a sample rubric for a learning outcome in general psychology are included below. Sample Assessment Rubric for Assessing a Single Student Learning Outcome Description of Learning Outcome Student Learning Outcome 1 (Describe) Method of Assessment (Describe) Competency Level Exceeds Expectations Outcome Criteria (Describe the specific outcome behaviors that match this Competency Level) Meets Expectations (Describe the specific outcome behaviors that match this Competency Level) Needs Improvement (Describe the specific outcome behaviors that match this Competency Level) Beginning Level (Describe the specific outcome behaviors that match this Competency Level) Sample Completed Rubric for a Single Student Learning Outcome (Psy 110 - General Psychology) Description of Learning Outcome Method of Assessment Competency Level Exceeds Expectations Meets Expectations Needs Improvement Beginning Level Student Learning Outcome 1 Discusses current theoretical conceptions of the human naturenurture question, by defining what is meant by nature and nurture, describing how these two concepts are related, and explaining how current scientific understanding of interactions between these two concepts can be applied to a practical example or social issue. A brief (5-10 minute) knowledge probe (Cross and Angelo, 1988). Outcome Criteria Clearly describes the question at a general, conceptual level. Correctly defines each term. Describes how nurture can influence the expression of generic predispositions, and also explains how genes can influence individual experiences. Clearly explains how current understanding of the question can be applied to several practical examples or social issues. Clearly describes the question. Correctly defines each term. Describes how nurture can influence the expression of generic predispositions, and also explains how genes can influence individual experiences. Explains how current understanding of the question can be applied to a practical example or social issue. Generally describes the question using concrete examples. Offers an incomplete definition of one or both of the terms, or reverses the definitions. Describes the idea that the two terms combine to influence development, and may describe influences in one direction, but cannot describe mutual interactions. Identifies one practical example or social issue related to the question, but does not explain how current understanding can be applied or why the question is important. Cannot describe the question or generally describes the question using only concrete examples. Cannot correctly define either of the two concepts. Cannot correctly describe how the two concepts are related. Cannot describe a practical example or social issue related to the question or explain why the question is important. Developing Student Learning Outcomes, Methods of Assessment and Outcome Criteria Developing Student Learning Outcomes, Methods of Assessment and Outcome Criteria is an iterative process, as suggested by the chart below. Developing Student Learning Outcomes, Methods of Assessment, and Outcome Criteria Student Learning Outcome Method of Assessment Outcome Criteria Try it out Which sections will use the same rubric? General Education Skill SLOs Section SLOs Section SLOs Section SLOs Section SLOs Section SLOs Section SLOs Sections that claim to teach a particular GenEd skill will use the same rubric Each GenEd group will have to develop SLOs, determine the method(s) of assessment and define the criteria for each outcome. This process will have to be extended to GenEd knowledge areas and specific discipline objectives. As I think about what we need to do, I’m overwhelmed. This is an enormous effort and a significant task for faculty and CAS. Competency Map After the SLOs for each GenEd skill and knowledge area are established, the college needs to develop a Competency Map for the GenEd Outcomes. A Competency Map helps identify which courses and programs make substantive contributions to the relevant Student Learning Outcomes. The map is simply a matrix which crosses references courses and programs against the SLOs for each GenEd outcome. The map is a necessary tool for planning which sections should participate in assessments for which GenEd outcomes (program-wide and college-wide).