Assessment and Problem Based Learning PBL 2004 Glen O’Grady Director, Center for Educational Development Overview of the Workshop More about you Assessment Assessment in PBL Grading How do you Assess in your PBL Programmes? Draw your response – – – How it is done? Why? The effect (intended or unintended) Why do we assess? To measure learning To foster learning What are we assessing? Learning…. but what is learning? Conceptions of Learning A. B. C. D. E. F. Increasing one’s knowledge Learning as memorizing and reproducing Learning as applying Learning as understanding Learning as an interpretive process aimed at understanding reality Learning as changing as a person Willis 1993 Instructionbased Syllabus Subject Z Subject X Subject X Subject X Teacher Dissemination of Knowledge Instruct, discipline, assess Students Assessment is testing how well students have retained what the teacher has disseminated. Assessment is about coming to know how students construct their understanding of knowledge by giving tasks that also helps to foster this construction of knowledge Problembased The Problem (Curriculum) Team members Student Construction of Knowledge Prior Knowledge Facilitator How can assessment be organized in PBL so that it both fosters & measures learning for understanding? Assessment needs to be… considered as an integral element in facilitation based on multiple methods. focused on both product and process. holistic Case Study on Assessment in PBL PBL at The Republic Polytechnic (RP-PBL) Each semester, 5 modules (subjects) each day students attend and work exclusively on one module Students follow a regular schedule each day RP-PBL: 1st meeting Class of 25, 5 teams of 5 students Presented a problem Students under the guidance of the facilitator work on defining the problem and identify issues they will do research on. Approximately 1 hour RP-PBL: 1st Breakout Student work individually and in their teams to: – – – Find and review resources Begin to develop tentative solutions for the problem Refine their definition of the problem RP-PBL: 2nd Meeting Meet with the facilitator who checks on their progress Focus on any difficulties students may be having Helps students to develop learning strategies RP-PBL: 2nd Breakout Student continue to work in their teams Review resources Develop a solution/ explanation based upon their shared understanding Produce a presentation 2-3 hours RP-PBL: 3rd Meeting Meet with the facilitator Students present their solutions/explanations Students observe how others have solved the problem Facilitators probes and critique these solutions giving additional information where necessary Students further check their understanding by doing a quiz focussed on the key issues RP-PBL: Assessment Performance & Presentations Self & Peer Evaluation Reflection journal Students get feedback everyday – – – Verbal feedback in class (interactions) Daily grade derived holistically Written feedback Students every month sit an understanding test Module grade is determined by a combination of daily grades and understanding tests performance. Exercise Method of Assessment How does it foster learning? How can it assist facilitators in their summative judgment (measure learning) Key challenges I. Team Oral/Written Presentations How can it foster learning? – – How can it measure learning? – Develops communication skills (Hay 1994) Verbal articulation (Rowntree 1987) It manifests the application of knowledge in the context of a problem Key challenges – – – Criteria Collusion Tip of the iceberg (Lakomski 2002) II. Peer and self assessment How can it foster learning? – – How can it measure learning? – Develops the capacity and ability to judge (Warren Piper et al. 1996, Boud 2001) Engenders normative values in the way students relate to one another (Shoop 2000) Manifests the way students think about themselves and others Key challenges – Reliability of these judgments III. Learning Journal How can it foster learning? – – – How can it measure learning? – Writing as a tool for organizing thoughts Reflection the integration of theory and experience (Schon 1983, Butler 1994) Focuses on the process of attributing meaning It allows the facilitators to have an insight into the experience of the learner (process of reasoning, their interaction with others, their emotive connection) Key challenges – Students not knowing how to reflect IV. Tests and Quizzes How can it foster learning? – How can it measure learning? – – Emphasizes specific skill sets and the need to be familiar with particular information. Demonstrates understanding that can be compared Gives insight into reasoning processes (Gibbs 1992b) Key challenges – – – Artificial and ritualized could encourage memorization & recall if questions are not carefully crafted Difficult to infer understanding (and its gradations) How to give feedback V. Observations of students’ discussions How can it foster learning? – How can it measure learning? – Allows the facilitator to interact with students in meaningful ways (Barrows 1980,1988, Huba 2000) Observe first hand elements of the construction of meaning Key challenges – Facilitators perspective (Pratt 1998) Other methods? How can it foster learning? How can it measure learning? Key challenges Marking & Giving Feedback Managing the task of marking – – Criteria Mark broadly, give specific feedback Making a difference with feedback – – – – Personal Regular Cumulative Holistic How do we derive a grade? How do we derive a grade? Deriving the daily grade in a manner the fosters learning for understanding. – – – – – Collect as much information that is possible and practical. Consider information collected in assessment as evidence that should be weighted in relation to the context. Have explicit criteria but be comfortable with the “subjectivism” in applying that criteria. Make it explicit (in feedback) the rationale for the judgment link it to the evidence. Be prepared to explain this judgement to peers (and others) Holistic Judgements Marks for student work should not be seen simply as individual bricks in a wall that represents a student’s final grade. Rather students’ work in its various forms are pieces of information or (evidence) that should be used collectively to form a picture of the quality of a students’ learning. Holistic Judgements The final grade of a student should represent our best professional judgment about the learning of a student both in terms of process and product. The learning and the judgement should enduring.