Incorporating IDEA into Assessment of General Education Learning Outcomes Stephanie Oetting, PhD Director of Institutional Research University of Saint Francis Fort Wayne, IN General Aims Incorporating IDEA student ratings into general education assessment Using IDEA extra question process to collect data for general education assessment USF GE Assessment Model Mix of direct/indirect and internal/external measures Custom reports versus aggregate data files Using extra questions in many classes Background General education curriculum in place 20+ years Intense work for revision initiated in 2003 New GE Framework approved late Spring 2007 Assessment plan work successfully avoided Faculty Governance Structure Committee On Assessment Student Academic Achievement General Education Committee Academic Council General Education Goals 1. 2. 3. 4. 5. 6. 7. 8. Develop personal awareness of diverse/global society. Demonstrate leadership, service, social responsibility. Communicate effectively in personal/professional interactions. Demonstrate competence in applying technologies. Demonstrate personal and social awareness of importance of lifelong health/wellness. Think analytically, synthetically, critically, and creatively in the pursuit of knowledge. Develop awareness/understanding of artistic/expressive aspects of the human experience. Appreciate spiritual dimension of life, be conscious of own religious perspective within a community context. Layers of Complexity 8 General Education Goals operationalized with 31 Learning Outcomes grouped into 19 Distribution Requirements State of Assessment Challenges: Several significant administrative and structural changes New GE framework lacking assessment plan Exhaustion from GE framework revision No Clear leadership, accountability, authority for GE assessment and Committees searching for a responsible party! Strengths: Program assessment welldeveloped in some programs Emerging group of faculty knowledgeable about, invested in assessment Support, encouragement from University AQIP Systems Portfolio in progress Assessment “Opinion Leaders” vocalizing Getting Started… “You don’t have to see the whole staircase, just take the first step.” --Martin Luther King, Jr. Task Force Convened Composition Goals BEGIN draft of assessment plan Reduce fear and avoidance Identify critical resources for implementation Cultivate culture of assessment Compensation & reward Requirements of Assessment Plan! Simple, seamless, sustainable Produce meaningful data, quickly and efficiently Use what’s already in place, no extra time or work! Benefit students whenever possible Empower instructors/dept. to generate artifacts Need naturally occurring artifacts, not extra work Compensate faculty who give their time! No evaluation of faculty, course or program No policing or punitive approach IF ONLY… Comforting thoughts… “It doesn’t really matter whether you can quantify your results. What matters is that you rigorously assemble evidence – quantitative or qualitative – to track your progress.” “What matters is not finding the perfect indicator, but settling upon a consistent and intelligent method of assessing your output results, and then tracking your trajectory with rigor.” Collins, J. (2005). Good to Great and the Social Sectors: A Monograph…(p. 7, 8) GE Assessment Plan Direct measures Indirect measures Student perceptions Inferred measurements Internal measures Student demonstration of learning outcomes Evaluated by faculty Locally developed External measures Standardized assessments Results comparisons Indirect Measures External Student self-reported progress on IDEA objective that aligns with goal Noel Levitz SSI Internal Student self-reported progress on learning outcome using IDEA extra question format Other surveys and assessments Align Goals to IDEA Objectives Develop a personal awareness of our diverse and global society Developing a clearer understanding of, and commitment to, personal values (#10) Demonstrate leadership, service, and social responsibility Acquiring skills in working with others as a member of a team (#5) Direct Measure Internal Artifacts generated at course level Assessed by faculty committee Holistic rubric using learning outcome Allows for comparison of faculty and student ratings Goal 1 % students enrolled in foreign language LO3 Develop a personal awareness of our diverse and global society LO1 IDEA Extra Q IDEA Extra Q Student satisfaction Noel Levitz SSI #62: Commitment to racial harmony on campus MLK Survey Artifact: iConnect Progress on IDEA objective #10: Understanding of & commitment to personal values LO2 IDEA Extra Q = Direct Measure University of Saint Francis GE Assessment Model = Indirect Measure 4-year Assessment Cycle 2007-2008: Goals 1 and 2 2008-2009: Goals 3 and 4 2009-2010: Goals 5 and 6 2010-2011: Goals 7 and 8 2011-2012: . .. Goals 1 and 2 Preliminary Results IDEA Objectives Percent selecting as important/essential Progress on relevant objectives Learning Outcomes IDEA Extra Questions – student perception Artifact assessment – faculty assessment Results - Learning Outcomes Goal 1 Indirect Measure: Student Rating 3 2.5 2 1.5 1 0.5 0 Mean Mean Goal 1 Direct Measure: Artifact Rating LO 1 LO 2 LO 3 5 4.5 4 3.5 3 2.5 2 1.5 1 Fall 2007 Mean Spring 2008 Mean LO1 Learning Outcomes Goal 2 Indirect Measure: Student Rating Mean Mean 3 2.5 2 1.5 1 0.5 0 LO 5 LO 6 Learning Outcomes LO3 Learning Outcomes Goal 2 Direct Measure: Artifact Rating LO 4 LO2 LO 7 5 4.5 4 3.5 3 2.5 2 1.5 1 Fall 2007 Mean Spring 2008 Mean LO4 LO5 LO6 Learning Outcomes LO7 iConnect Course Summary Ratings Overall ratings for teacher and for course 5 4.4 4 3.2 iCON Course Institution IDEA 3 2 1 Excellence of Teacher Excellence of Course Student comments examined Faculty feedback sought Actionable Lessons New Results Learned Directions QUESTIONS? THOUGHTS? COMMENTS?