Guide to Writing Outcomes ADC Guide to writing Learning Outcomes INTRODUCTION This guide has been prepared by the Academic Development Centre to assist staff involved in curriculum development. The Quality Assurance Agency for Higher Education requires that all course documents will be replaced by programme specifications and that these are to be written using an outcome-based approach. This guide attempts to explain the use of learning outcomes in the context of the credit system at Kingston University and to offer suggestions for the design of outcomes. WHAT ARE LEARNING OUTCOMES? Learning outcomes describe what a student should know, understand or do at the end of a period of learning. Learning outcomes are used as a tool for curriculum design. The writing of module outlines in terms of learning outcomes represents a shift in focus towards a more student-centred approach in higher education. The emphasis moves from the content (what staff will teach) to outcome (what a student will be able to do on successful completion of the module). Learning outcomes indicate to students what is expected of them, so that they know what to do to achieve a pass in the module. The experience of writing learning outcomes can also be helpful to staff, in that it provides an opportunity for them to decide what they want students to achieve when taking a module. Such achievement can include both knowledge and skills. LEARNING OUTCOMES AND CREDIT Kingston University’s Credit Accumulation and Transfer (CAT) Scheme is based on the award of credit to learners. The building blocks of a CAT scheme are credits which are awarded for the achievement of learning outcomes at a specified level. The achievement of learning outcomes is the basis for the award of academic credit. Each set of learning outcomes is allocated a credit value which indicates the volume and learning of the module. In the undergraduate modular scheme at Kingston the majority of modules are 15 credits at a specified level. 2 ADC Guide to writing Learning Outcomes One credit represents ten notional hours of successful learning time. This includes all the learning time a student will take to demonstrate achievement of the learning outcomes such as contact time, self-managed study time and time for preparation of assessments. As a general guide one 15-credit module assumes 150 hours of student learning. Since the achievement of the learning outcomes triggers the award of credit, the outcomes are usually written in a threshold model, ie specifying the learning outcomes which would enable a student to achieve the minimum acceptable standard to pass the module. Because this is the case it is important to avoid over specification. The learning outcomes specified are those which represent the essential learning for a module. There should therefore be a small group of learning outcomes which are of fundamental importance rather than a long list of superficial outcomes. It is probable that a module should have between four and eight learning outcomes. It is important that learning outcomes are written consistently to ensure clarity across modules. LEARNING OUTCOMES AND LEVEL In addition to credit value, each set of learning outcomes is related to a particular level of achievement. We generally expect a different level of performance from a Level 4 student than from a Level 6 student. A level is: “An indicator of the relative demand, complexity and depth of learning and of learner autonomy.” InCCA (Inter-Consortium Credit Agreement Project) 1998. Each credit level has a level descriptor. These are: “Statements which describe the types of learning demand which the learner will encounter at each level.” NICATS (Northern Ireland Credit Accumulation and Transfer System) 1999. Level descriptors give guidance on level of expectation and help to establish equivalence across disciplines. Kingston’s level descriptors are on page 4. 3 ADC Guide to writing Learning Outcomes LEARNING OUTCOMES AND ASSESSMENT By assessing students it is possible to see whether they have achieved the learning outcomes of a module. The assessment strategy for a module is therefore of fundamental importance. Ideally there will be a range of assessment methods and these should directly reflect the outcomes being assessed. It is helpful to ensure that learning and teaching strategies give students an opportunity to develop the required skills to cope with the assessment. For example, if assessing teamwork students should be given the opportunity to work in groups. Assessment criteria indicate what standard a student should meet to demonstrate that the learning outcomes of a module have been achieved. An assessment method could be ‘a 1500 word essay’, while the assessment criteria could be ‘a well structured essay containing an introduction and a conclusion with appropriate references and a bibliography’. In the threshold model, the assessment criteria indicate what a student needs to do to pass the module. They should therefore be distinguished from grading criteria, which indicate how well the student has met the assessment criteria. WRITING LEARNING OUTCOMES At Kingston University learning outcomes are prefaced by: “On successful completion of the module, students will be able to…” This ensures that the outcomes are expressed appropriately using verbs (see list on page 3) since this gives the opportunity to the student to demonstrate that they have learned / achieved the outcome. The use of some verbs in learning outcomes can make it difficult to design appropriate assessment. For example those related to knowledge outcomes, such as “know” and “understand” are rather general. It is more helpful when learning outcomes make clear how the students are expected to demonstrate that knowledge, such as “solve”, “discuss”, “analyse”. 4 ADC Guide to writing Learning Outcomes Examples Level 4 Accounting On successful completion of the module, students will be able to: • explain the role of accounting information in organisations. Level 5 Literature On successful completion of the module, students will be able to: • demonstrate skill in the critical evaluation of literary texts. Level 6 Business Strategy On successful completion of the module, students will be able to: • identify and critically evaluate the strategic options available to enterprises. The following list of verbs is designed to help you to write learning outcomes. This list is based on Bloom’s taxonomy (1956)* which identified six categories of learning: knowledge, comprehension, application of knowledge, analysis, synthesis and evaluation. These categories may be used at any academic level but it may be useful to use the more complex forms of learning behaviour at the higher academic levels. As undergraduates progress to Level 4 to Levels 5 and 6 the outcomes will reflect higher level intellectual activities, such as synthesis and evaluation. *Bloom B.S. (Ed) 1956 Taxonomy of Educational Objectives: The Classification of Educational Goals. New York, Toronto. Longmans. 5 ADC Guide to writing Learning Outcomes Verbs giving evidence of KNOWLEDGE Define, describe, identify, label, name, outline, reproduce, recall, select, state, present, extract, organise, recount, write, recognise, measure, underline, repeat, relate, match. Verbs giving evidence of COMPREHENSION Interpret, translate, estimate, justify, comprehend, convert, clarify, defend, distinguish, explain, extend, generalise, exemplify, give examples of, infer, paraphrase, predict, rewrite, summarise, discuss, perform, report, present, restate, illustrate, indicate, find, select, represent, name, formulate, judge, contrast, translate, classify, express, compare. Verbs giving evidence of APPLICATION OF KNOWLEDGE/UNDERSTANDING Apply, solve, construct, demonstrate, change, compute, discover, manipulate, modify, operate, predict, prepare, produce, relate, show, use, give examples of, exemplify, draw (up), select, explain how, find, choose, assess, practice, operate, illustrate, verify. 6 ADC Guide to writing Learning Outcomes Verbs giving examples of ANALYSIS Recognise, distinguish between, evaluate, analyse breakdown, differentiate, illustrate how, infer, point out, relate, select, separate, divide/subdivide, compare, contrast, justify, resolve, devote, conclude, criticise, question, diagnose, categorise, point out, elucidate. Verbs giving examples of SYNTHESIS Propose, present, structure, integrate, formulate, teach, develop, combine, compile, compose, create, devise, design, explain, generate, modify, organise, plan, rearrange, reconstruct, relate, re-organise, revise, write, summarise, tell, account for, report, alter, argue, order, select, manage, generalise, derive, conclude, build up, engender, synthesise, put together, suggest, enlarge. Verbs giving evidence of EVALUATION Judge, appraise, assess, conclude, compare, contrast, describe how, criticise, discriminate, justify, defend, evaluate, rate, determine, choose, value, question. 7 ADC Guide to writing Learning Outcomes LEVELS DESCRIPTOR - LEVEL M (ADAPTED FROM SEEC/HECIW LEVELS DESCRIPTORS) Operational Context The context of the learning will have the following characteristics: • complex with elements of unpredictability, demanding innovative work which may involve exploring the current limits of knowledge (either theoretical or applied) • containing a high level of responsibility for self and possibly for others • requiring an awareness of ethical dilemmas likely to arise in research/professional practice. Cognitive/Intellectual Skills (generic) The learner will demonstrate: • depth of knowledge in a complex and specialised area and/or a considerable breadth of knowledge across specialised and applied areas • an ability to deal with complexity, lacunae and contradictions in a complex knowledge base using appropriate methods • independent evaluation of alternative approaches (including their own) together with the ability to accurately report this • a confident response to problems by autonomously synthesising information/ideas to provide solutions. Key/Transferable Skills (generic)* The learner will demonstrate: • engagement with a critical community, including reflection on own and others’ practice • the ability to work with and within a group towards defined outcomes and can take the role of recognised leader or consultant • the ability to motivate others and to negotiate conflict effectively • the ability to engage in full professional and academic communication with peers. The demonstration of the above skills will normally be embedded in a specific subject base. The extent to which these qualities were expected would be indicated to the learner and not all aspects would be present in each element of work undertaken. 8 ADC Guide to writing Learning Outcomes LEVELS DESCRIPTOR - LEVEL 6 (ADAPTED FROM SEEC/HECIW LEVELS DESCRIPTORS) Operational Context The context of the learning will have the following characteristics: • complex with some unpredictability, response to which demands selection from a range of innovative or standard techniques • requiring autonomy in planning and managing resources and processes within broad guidelines • involving awareness of personal responsibility and professional codes of conduct and the ability to incorporate a critical ethical dimension into work where appropriate. Cognitive/Intellectual Skills (generic) The learner will demonstrate: • a comprehensive/detailed knowledge of (a) major disciplines(s) with areas of specialisation in depth and an awareness of the provisional nature of the state of knowledge • an ability to analyse new and/or abstract data and situations without guidance, using a wide range of techniques appropriate to the subject • having minimum guidance, the ability to transform abstract data and concepts towards a given purpose and can design novel solutions • the ability to critically review evidence supporting conclusions/recommendations including its reliability, validity and significance, and to investigate contradictory information. Key/Transferable Skills (generic)* The learner will demonstrate: • confidence in the application of their own criteria of judgement and can reflect on action to elucidate/evaluate chosen approach • confidence and flexibility in identifying and defining complex problems and the ability to apply appropriate knowledge and skills to their solution • the ability to engage effectively in debate in a professional manner and produce detailed and coherent project reports • the ability to interact effectively within a group, including the exercise of leadership skills on occasion and the ability to negotiate and manage conflict if necessary. The demonstration of the above skills will normally be embedded in a specific subject base.The extent to which these qualities were expected would be indicated to the learner and not all aspects would be present in each element of work undertaken. 9 ADC Guide to writing Learning Outcomes LEVELS DESCRIPTOR - LEVEL 5 (ADAPTED FROM SEEC/HECIW LEVELS DESCRIPTORS) Operational Context The context of the learning will have the following characteristics: • simple but unpredictable or complex but predictable contexts demanding application of a wide range of techniques • requiring the management of processes within broad guidelines for defined activities • requiring an awareness of the wider social and environmental implication of area(s) of study and the ability to debate issues in relation to more general ethical perspectives. Cognitive/Intellectual Skills (generic) The learner will demonstrate: • a detailed knowledge of (a) major discipline(s) and an awareness of a variety of ideas/contexts/frameworks which may be applied to this • the ability to analyse a range of information with minimum guidance, applying major theories of the discipline and comparing alternative methods/techniques for obtaining data • the ability to reformat a range of ideas/information towards a given purpose • the ability to select appropriate techniques of evaluation and use these to judge the significance of data collected. Key/Transferable Skills (generic) The learner will demonstrate: • an ability to develop own criteria and judgement, evaluating own strengths and weaknesses and challenging received opinion where appropriate • the ability to adopt a broad ranging and flexible approach to study with autonomous performance in straightforward tasks • the ability to identify key elements of problems and select appropriate methods for their solution, reporting effectively on work undertaken • the ability to interact effectively within a group, exchanging information and modifying response where appropriate. The demonstration of the above skills will normally be embedded in a specific subject base. The extent to which these qualities were expected would be indicated to the learner and not all aspects would be present in each element of work undertaken. 10 ADC Guide to writing Learning Outcomes LEVELS DESCRIPTOR - LEVEL 4 (ADAPTED FROM SEEC/HECIW LEVELS DESCRIPTORS) Operational Context The context of the learning will have the following characteristics: • a set of defined contexts demanding use of a specified range of standard techniques • containing limited autonomy, with work directed within defined guidelines • requiring an awareness of ethical issues in current area(s) of study and the ability to discuss these in relation to personal beliefs and values. Cognitive/Intellectual Skills (generic) The learner will demonstrate: • a given factual and/or conceptual knowledge base with emphasis on the nature of the field of study and appropriate terminology • the ability to analyse with guidance, using given classifications/principles • the ability to collect and categorise ideas and information in a predictable and standard format • the ability to evaluate the reliability of data using defined techniques and/or tutor guidance. Key/Transferable Skills (generic)* The learner will demonstrate: • the ability to work within a relevant ethos and to access and use a range of learning resources • the ability to apply given tools/methods accurately to a well defined problem and the awareness of the complexity of issues • the ability to work effectively in an appropriate format and to report in a clear and concise manner with all relevant information • the ability to meet obligation to others and to offer initiatives and support those of others. The demonstration of the above skills will normally be embedded in a specific subject base. The extent to which these qualities were expected would be indicated to the learner and not all aspects would be present in each element of work undertaken. *These are a generic guide to Key Skills. More details on Kingston University’s Key Skills are being developed. This guide was written by Anita Walsh and Marion Webb, September 2001. Updated 2009 11 Academic Development Centre, Millennium House, 21 Eden Street, Kingston upon Thames, Surrey KT1 1BL T: 020 8547 7770 www.kingston.ac.uk X(09.140)B