Assessment of Service-learning Courses in Higher Education: Understanding the Rationale and Techniques Kathy O’Byrne, Ph.D. UCLA Center for Community Learning May 5, 2010 Agenda Welcome and Introductions Definitions of assessment and service learning Learning outcomes Research options: unit of analysis Integration into your syllabus & examples Resources Questions and Discussion Your campus and your experience History of your service History with assessment? learning program? Institutional resources? Disciplines? Campus leadership? Service learning courses or others? Research and publications? Faculty development experience? Key components of assessment Assessment is linked to learning goals/outcomes. Meaningful work in service learning courses is designed to elicit higher order thinking Multiple forms of evidence Assessment activities should draw on the affective and cognitive domains of learning Self-assessment is often part of the overall design Student involvement and feedback is important data to guide future learning Consider an examination of processes as well as outcomes Pre-planning and integrated with instruction Expectations apparent to students Definitions and Common Language Assessment practices are a reflection of your values. Assessment is a means of understanding the learning process, and improving the quality of learning that occurs. Assessment is the systematic application of tools and techniques to better understand and improve the process of learning by everyone involved. Assessment is a mind set that asks questions– good questions, hard questions, about what and how students are learning. Assessment is an ongoing process aimed at understanding and improving student learning. It involves making our expectations explicit and public; setting appropriate criteria and high standards for learning quality; systematically gathering, analyzing and interpreting evidence to determine how well performance matches those expectations and standards; and using the resulting information to document, explain, and improve performance. Assessment…can help us focus our collective attention, examine our assumptions, and create a shared academic culture dedicated to…improving the quality of higher education. What is service learning? Equal parts “service” and “learning” Reflection is key Embedded in academic, credit-bearing courses Connection to graded assignments Reciprocal benefit Collaborative approach to projects Examples of strong service learning programs Tulane University University of San Diego Kapiolani Community College University of Washington UCLA Options for assessment: unit of analysis Students: learning outcomes, knowledge, skills, attitudes, behaviors Faculty: changes in teaching and instruction Community: impact of the project, qualities of the partnership Students Faculty Community Potential service-learning outcomes for students Communication Skills (written and oral) Interpersonal Skills Cultural awareness/ Appreciation of diversity Improved use of technology Logical reasoning Application of theory Self-understanding Critical thinking Investigation of Power relationships Analysis of social institutions Explication of values Identify and frame problems Active listening Negotiation skills Public dialogue on shared issues Creative conflict to produce growth Political imagination: reimaging futures Mentoring Commitment to service/service ethic Career Development Values clarification Character Education Civic Engagement Perspective Transformation Other? Student learning and assessment issues 1. 2. 3. Define student learning outcomes: knowledge, skills, attitudes or values? Systemic issues: what is the role of the student, faculty, sites? How will students demonstrate learning? What data will you use? Graded assignments? Pre-post instruments? Other? Service-learning outcomes for assessment 1. 2. 3. 4. 5. Understand and apply knowledge Develop identity and values Develop community awareness and practice community engagement Demonstrate a commitment to lifelong learning and action Awareness of social justice issues Summarize overall plan 1. 2. 3. 4. Expectations/assumptions about students Learning outcomes: what they should know and/or know how to do Teaching strategies in support of outcomes Assessment Strategies: who? when? how? where? what? Five Assessment techniques: Strategy #1: Critical Incidents Describe an incident that was “critical” to you. It can be surprising, frightening, enlightening or something that made you proud What made it significant? What did you learn or how did it change your worldview/assumptions? Strategy #2: Concept mapping One main concept from the course Additional cards or post-its that students arrange in a patter Related concepts connected or close to one another Policy, diversity, social justice, the role of government, economic factors, etc. Strategy #3: Problem based learning Based on training from medical schools Students write about an issue related to their service learning work. Have to provide a potential solution for change in the community, that includes classroom/curriculum learning and connections to the service learning site/staff/meaningful work they performed. Strategy #4: Focus Groups Small groups of students who all worked at the same site answer structured, pre-planned questions about the organization, the community, the work they accomplished, and the future of the organization. If change is needed, who would have to get involved to make that happen? Elected officials? Residents? Others? Strategy #5: Pre-post instruments Can focus on knowledge, attitudes, beliefs, values related to the work of the organization. Can have students complete the pre-test and return it after they complete the post-test to compare their answers. Can break down differences according to gender, majors, other characteristics including grades. Other options for assessing service learning: faculty or academic departments Departmental views for integrating service learning into a major or minor; Faculty transformation after teaching a service learning course Research and publication on discipline-based service learning Assessment with Community Partners Satisfaction Outcomes for community Impact residents or agency clients Connection of projects to public policy Research that informs decision-making Visibility Building the organization’s capacity Example: Sample questions for service learning community partners Rate student learning: cultural awareness, appreciation of diversity, communication, critical thinking, negotiation, career exploration, understanding of nonprofits and public policy issues, etc. General: understanding of their responsibilities, how well prepared were students? What was the meaningful work that was performed? Your syllabus Definition of service learning Learning outcomes Community partners and criteria Job descriptions for each site Connection to graded assignments Logistics Some learning outcomes Course Competencies and Learning Outcomes: Students in this course will learn: Definitions of civic engagement and service learning; Current and historical national trends in higher education to promote democratic action and civic engagement; The diverse traditions of service and the history of service movements; How civic engagement and participation in public life contribute to overall quality of life in the community; An overview of the issues raised by theories and empirical studies of links between individual motivation and success, access to resources and distribution of wealth; A description of the national movement for reform of undergraduate education; Trends in research and scholarship that are impacted by this movement; Definitions and research on sustainable environments, including (but not limited to) issues of water usage, endangered species and native habitats, transportation, and pollution. The role of community-based organizations that support a variety of sustainable environment programs, non-profits and governmental agencies; The potential range of personal choices that individuals can make to promote sustainable environments in Los Angeles. Principles of university/community partnerships (CCPH) Principles of good community-campus partnerships: Partners have agreed upon mission, values, goals, and measurable outcomes for the partnership. The relationship between partners is characterized by mutual trust, respect, genuineness, and commitment. The partnership builds upon identified strengths and assets, but also addresses areas that need improvement. The partnership balances power among partners and enables resources among partners to be shared. There is clear, open and accessible communication between partners, making it an ongoing priority to listen to each need, develop a common language, and validate/clarify the meaning of terms. Roles, norms, and processes for the partnership are established with the input and agreement of all partners. There is feedback to, among, and from all stakeholders in the partnership, with the goal of continuously improving the partnership and its outcomes. Partners share the credit for the partnership's accomplishments. Partnerships take time to develop and evolve over time. A “menu” of service learning options: Based on time available, resources, and experience An “Appetizer”: Pre-post research on knowledge, attitudes, etc. An “Entrée”: Ongoing collection of data throughout the academic term or a comparison of service learners to nonservice learners A “Dessert”: small scale study at the end of a course. Examples Attitude toward Korean immigrant neighborhoods before and after the service learning experience Compare the learning outcomes of two sections in a large general education course (service learning and non) In what way would you say you have changed your mind—if only a little bit—about one issue, as a result of the service learning work? Resources: Publications and Conferences Journal of Applied Learning in Higher Education): 2009 Ash and Clayton Generating, Deepening, and Documenting Learning: The Power of Critical Reflection in Applied Learning This article will consider the meaning of critical reflection and principles of good practice for designing it effectively and will present a research-grounded, flexible model for integrating critical reflection and assessment. http://www.missouriwestern.edu/appliedlearning/iss ues.asp 1999: Where’s the Learning in Service Learning? Janet Eyler and Dwight Giles. National Service Learning Clearinghouse Bonner Foundation Campus Compact Tenth International Research Conference on Service-learning and Community Engagement International Perspectives: Crossing Boundaries through Research October 28-30, 2010 Indianapolis, IN . The Indiana University-Purdue University Indianapolis (IUPUI) Center for Service and Learning, Indiana Campus Compact, and the International Association for Research on Service-learning and Community Engagement. October 28-30, 2010 at the Crowne Plaza at Historic Union Station in Indianapolis, IN. The featured theme for the 10th annual conference is “International Perspectives: Crossing Boundaries through Research.” Service-learning is valued as an active learning strategy across the globe; however, little is known about the ways that servicelearning is similar or different in varied contexts. Understanding service-learning and community engagement from diverse cultural perspectives will add insight necessary for comparative research and to improve practice. . Questions Reactions Discussion Relevance ? Need for further information Next steps for your service-learning course! Contact information Kathy O’Byrne, Ph.D. UCLA Center for Community Learning kobyrne@college.ucla.edu