Assessment 102 Developing Program Goals and Intended Learning Outcomes Overview Topics for this Workshop Review of the Assessment Process Steps in the Process and Common Obstacles Assessment Goals and Objectives Rationale and Objectives Key Terms and Distinctions Sources & Origins: How and Where to Begin Types of Goals: Cognitive, Behavioral, Affective Writing Goals and Objectives Evaluating Your Goals and Objectives Sample Departmental Goals and Objectives Review: The Assessment Process State the broad educational purposes and goals of your program State your more specific objectives and intended outcomes Select appropriate assessment methods and measures; establish target criteria Gather data using assessment methods & measures chosen Analyze and interpret findings; report to stakeholders Identify, implement changes to your educational program Revise assessment objectives, methods, measures, criteria as appropriate Obstacles and Stumbling Blocks 3 Cardinal rules for those undertaking assessment and evaluation The tendency to focus on process rather than outcomes Assessment as an “add-on” Skepticism RE “new ideas” in higher ed. Disagreements among colleagues Goals and Objectives: Rationale “If you don’t know where you’re going, you’ll probably end up somewhere else” Campbell, 1974 Goals and Objectives: Rationale Goal-Setting and Goal-Directedness as Markers of Successful Individuals and Organizations The Importance of Modeling Goal-Setting and Goal-Directedness for our Students The Central Role of Faculty in Articulating Program Goals and Objectives Goals and Objectives: Rationale Why Goals and Objectives MUST Come First in the Assessment Process Why they Often Don’t Consequences Goals and Objectives: Defined Goals Statements regarding general aims or purposes of education Broad, long-range intended outcomes Used primarily in policy-making, planning Objectives Brief, more specific statements describing the intended learning outcomes of a program Focus is on features students expected to exhibit Objectives and Outcomes Objectives (continued) The intended consequences of instruction, curricula, programs, educational activities What students are expected to know, do, value Outcomes The actual or achieved results or consequences of instruction, curricula, programs, or activities Writing Goals and Objectives: How to Develop Them and Where to Begin Open Discussions--your ideal students? What can they do, what do they know, what do they value (at various points in your program)? What achievements do you expect of graduates (career, lifestyle, citizenship, aesthetic appreciation)? What skills, knowledge, & values are necessary for entry level work or graduate study? Writing Goals and Objectives: How to Develop Them and Where to Begin Collect & Review Documents Describing Your Program Brochures, catalogue descriptions, accreditation reports, general education reports, missions Review & React to External Input National organization, professional association, accrediting bodies, strong programs in field (especially any relevant assessment plans) Writing Goals and Objectives: How to Develop Them and Where to Begin Collect and Review Instructional Materials Syllabi, course outlines, assignments, tests, textbooks (especially tables of contents, introductions, summaries) Use Goal Inventories or Other Tools Types of Goals & Objectives Knowledge (Cognitive Outcomes) Skills (Behavioral Outcomes) Attitudes (Affective Outcomes) Cognitive Objectives: What we expect students to know Knowledge (e.g.,define,describe, identify, list, match,recall) Comprehension (e.g., explain,paraphrase,summarize) Application (e.g., apply, relate, use in new situations) Analysis (e.g., compare/contrast, differentiate, analyze) Synthesis (e.g., compose, create, design, produce) Evaluation (e.g., assess, critique, defend, grade, judge, prioritize Behavioral Objectives What we expect students to be able to do General Description Learned observable behaviors Contexts and Areas of Application Performing arts, professional programs, technical/trade programs, athletics, etc. Sample skills Oral & written communication; presentation; interpersonal problem-solving; managerial, laboratory, music/art performance skills Affective Objectives What we expect students to value or believe Categories (Krathwohl et al. 1964) Potential Problems attitudes, beliefs, values, goals, expectations, interests, appreciation of…(person, object, goal, place) Can’t be measured directly, often inconsistent w/ behavior, disagreement over conceptual, operational definitions Benefits Status, development of values and beliefs Writing Goals & Objectives NCTLA Model Assessable goals state… What is to be learned (knowledge,skills,values) What level of learning is expected In c.f., Bloom’s or Krathwohl et al. Taxonomies what context the learning is revealed Example Students will be able to apply logical and ethical principles to personal &social situations Writing Goals & Objectives (Source: Assessment Workbook, Ball State U) Essential Components Optional Components Behavior--actions indicating objective achieved Object--identify focus of learning (knowledge, skill,etc) Target Groups--subgroups (when objectives differ) Conditions--when, where behavior must be shown Performance Criteria--form of behavior, min. level Performance Stability--frequency, duration, stability Example After analyzing & interpreting info from public opinion polls, the graduating journalism major will be able to communicate results to at least 3 different audiences in written, oral, & graphic forms Guidelines for Evaluating Objectives Number of objectives limited (3-5)? Refers to student behaviors? Clear, simple language; action verbs? Is outcome reasonable, achievable? Objective assessable by multiple methods? Objectives validated by colleagues? Assessing important outcomes? Troublesome ones? Results of new initiatives? Sample Objectives and Intended Learning Outcomes Books (e.g., by Nichols, J.O. et al, Agathon Press) Internet A practitioner’s handbook for institutional effectiveness and student outcomes implementation Assessment case studies: Common issues in implementation w various campus approaches to resolution The departmental guide and record book for student outcomes assessment and institutional effectiveness http://www2.acs.ncsu.edu/UPA/survey/resource.htm Local g:\zurarm\assessment\sample.plans\ What’s Next: The Assessment Process Revisited State the broad educational purposes and goals of your program State your more specific objectives and intended outcomes Select appropriate assessment methods and measures; establish target criteria Gather data using assessment methods & measures chosen Analyze and interpret findings; report to stakeholders Identify, implement changes to your educational program Revise assessment objectives, methods, measures, criteria as appropriate What’s Next: A More Specific Look at the Next Step Select Assessment Methods & Measures Criteria by which to select measures Strengths & weaknesses of various measures Matching objectives to measures Scheduling of, Responsibility for Assessment Establishing Target Criteria Level of performance desired? When is objective met? Do target criteria vary over time (Entry? Graduation?) What conditions necessary before objective can be met? Concluding Remarks Goals & Objectives Needn’t Reflect Present can Not all Actual Outcomes are the Intended or Expected Ones side reflect hopes, future; serve as guide effects & surprises can be informative Any Progress is Better that Inertia or Regression don’t wait for “perfect” goals or objectives Need Help? Books and Periodicals see Internet Resources see references in this & previous workshop url’s in this & previous workshop Local Resources Upcoming Workshops Assessment Newsletter, Brochures Assessment Committee Members