The Hong Kong Institute of Education Outcomes Based Learning Unit DESIGNING AND DEVELOPÄ°NG LEARNING OUTCOMES 1. What are learning outcomes? Learning outcomes are statements that specify what learners will know or be able to do as a result of a learning activity. Outcomes are usually expressed as knowledge, skills, attitudes or dispositions. Learning outcomes provide direction in the planning of a learning activity. They help to: i. Focus on learner’s behaviour that is to be changed ii. Serve as guidelines for content, instruction, and evaluation iii. Identify specifically what should be learned iv. Convey to learners exactly what is to be accomplished 2. What are the characteristics of good learning outcomes? Learning outcomes have three distinguishing characteristics: i. The specified action by the learners must be observable. ii. The specified action by the learners must be measurable. iii. The specified action must be done by the learners. The ultimate trial when writing a learning outcome is whether or not the action taken by the participants can be assessed. If not, the outcome probably does not meet all three of the foloowing characteristics. i. Who is to perform? ii. What action they are to take? iii. What results would come from their actions? 3. Problems with unclear outcomes Some courses include learning outcomes which are unclear or represent elements of curriculum which is either content-based or beliefs and behaviours that are intrinsic, rather than some actions the participants will demonstrate. 1 Note the following examples: a.) Participants will understand the nine reasons for conducting a needs assessment. b.) Participants will develop an appreciation of cultural diversity in the workplace. If you ask a simple question ("Can it be assessed?"), you see readily that these learning outcomes have shortcomings. They are not measurable. The same outcomes can be modified by changing the action verbs. a.) Participants will list nine reasons for conducting a needs assessment. b.) Participants will summarize in writing their view points and feelings about cultural diversity in the workplace. Learners now have a much better idea of what is expected of them. 4. What is the importance of action verbs? Since the learner's performance should be observable and evidence-based, the verb chosen for each outcome statement should be an action verb which results in overt behaviour that can be observed andassessed. By creating learning outcomes using measurable verbs, you indicate explicitly what the student must do in order to demonstrate learning. Sample action verbs are: compile, create, plan, revise, analyze, design, select, utilize, apply, demonstrate, prepare, use, compute, discuss, explain, predict, assess, compare, rate, critique, etc. A. To measure knowledge (common terms, facts, principles, procedures), ask these kinds of questions: Define, Describe, Identify, Label, List, Match, Name, Outline, Reproduce, Select, State. Example: Upon successful completion of the course, students will be able to: "identify the scientific, social and humanistic aspects of environmental issues.” B. To measure comprehension (understanding of facts and principles, interpretation of material), ask these kinds of questions: Convert, Defend, Distinguish, Estimate, Explain, Extend, Generalize, Give examples, Infer, Predict, Summarize. Example: Upon successful completion of the course, students will be able to:" explain environmental problems and give examples from the perspectives of both environmental science and resource management." C. To measure application (solving problems, applying concepts and principles to new situations), ask these kinds of questions: Demonstrate, Modify, Operate, Prepare, Produce, Relate, Show, Solve, Use. 2 Example: Upon successful completion of the course, students will be able to: "choose which of the various environmental product certifications best apply to the use of sustainable products." D. To measure analysis (recognition of unstated assumptions or logical fallacies, ability to distinguish between facts and inferences), ask these kinds of questions: Diagram, Differentiate, Distinguish, Illustrate, Infer, Point out, Relate, Select, Separate, Subdivide. Example: Upon successful completion of the course, students will be able to: "calculate the cost differentiation after reviewing at least five ways green product ratings may need to evolve in order to truly address a sustainable future." E. To measure synthesis (integrate learning from different areas or solve problems by creative thinking), ask these kinds of questions: Categorize, Combine, Compile, Devise, Design, Explain, Generate, Organize, Plan, Rearrange, Reconstruct, Revise, Tell. Example: Upon successful completion of the course, students will be able to:"design methodologies of the natural and social sciences to formulate testable hypotheses concerning environmental problems and issues." F. To measure evaluation (judging and assessing), ask these kinds of questions: Appraise, Compare, Conclude, Contrast, Criticize, Describe, Discriminate, Explain, Justify, Interpret, Support. Example: Upon successful completion of the course, students will be able to:"judge and assess proposed solutions to environmental problems and make recommendations from the perspectives of both environmental science and resource management.” " 3 BLOOM'S TAXONOMY OF MEASURABLE VERBS These verbs are categorized according to the level of thinking (in Bloom's Taxonomy) required by each. The purpose of this list is to help generate ideas for learning outcomes, particularly those requiring higher levels of thought. The categories are presented in order from basic levels (literal thinking) to complex levels (critical thinking). KNOWLEDGE (literal-level thinking) cite label name define list quote pronounce reproduce identify match recite state tell recall remember repeat recognize describe memorize locate draw write select COMPREHENSION (literal-level thinking) alter discover manage relate summarize explain rephrase restate give convert substitute match represent change depict translate vary describe illustrate reword distinguish interpret paraphrase transform infer review generalize extend examples express predict compare APPLICATION (critical thinking) apply make manage relate classify employ predict show use evidence prepare solve demonstrate manifest present utilize direct practice compute report illustrate change choose interpret draw model modify sketch dramatize paint collect produce ANALYSIS (critical thinking) ascertain diagram outline diagnose reduce contrast differentiate survey associate conclude examine designate determine organize research investigate separate compare point out dissect categorize infer subdivide distinguish construct classify separate analyze divide find discriminate SYNTHESIS (critical thinking) combine devise integrate revise conceive propose generalize originate compose extend construct organize design add to rearrange synthesize create pose project produce expand rewrite develop role play modify plan imagine compile invent theorize formulate hypothesize EVALUATION (critical thinking) judge conclude appraise evaluate select defend weigh recommend decide criticize compare consider contrast deduce verify summarize relate solve critique assess justify debate 4 VERBS THAT ARE NOT MEASURABLE Certain verbs are unclear and subject to different interpretations in terms of what action they are specifying. Such verbs call for covert behaviour which cannot be observed or measured. In order for an objective to give maximum structure to instruction, it should be free of vague or ambiguous words or phrases. The following lists notoriously ambiguous words or phrases which should be avoided so that the intended outcome is concise and explicit. Words to Avoid Believe, Hear, Realize Capacity, Intelligence, Recognize Comprehend, Know, See Conceptualize, Listen, Self-Actualize Depth, Memorize, Think Experience, Perceive, Understand Feel Phrases To Avoid Appreciation for… Acquainted with… Anxiety Attitude of… Adjusted to… Immaturity Awareness of… Capable of… Insecurity Comprehension of…. Cognizant of… Enjoyment of… Conscious of… Feeling for… Familiar with… Interest in… Interested in…. Knowledge of… Knowledgeable about…. Understanding of… Self-Confi dent in. 5 6