Peer observation, feedback and reflection for online practice Mark H. Jones, Faculty of Science & eSTEeM Overview • • • • • Motivation for the project Models of peer-review Implementation of a peer-review exercise Evaluation Comments on wider scale adoption Motivation • Level 3 physics and astronomy modules have moved completely to online tuition – with online rooms replacing face-to-face contact. • Staff Tutor has a responsibility to: – develop staff in their pedagogic practice – monitor the quality of what is provided • In an online environment, there are new possibilities to set up peer observation. • Can peer-observation be useful in developing practice for synchronous online tuition? Models of peer-review • Peer-observation is used throughout HE in several distinct contexts. • A useful classification was provided by Gosling (2002): - Evaluation model - Development model - Collaborative model • “Observation” is a loaded term. Peer observation models Characteristic Evaluation model Development model Collaborative model Who does it, and with whom? Senior staff, or chosen ‘evaluators’ other staff Educational developers observe practitioners Teachers/peers/ colleagues Purpose Quality assurance (e.g. identify underperformance); appraisal (e.g. promotion cases) Demonstrate or improve competencies in teaching; assessment. Improve teaching and learning through dialogue, reflection; stimulate innovation Outcome Report / judgement Report / action plan Analysis, discussion, wider experience of teaching methods Status of evidence Authority Expert diagnosis Peer shared perceptions Adapted from Gosling (2009) The collaborative ideal? • Purpose: improve teaching and student learning through dialogue, self and mutual reflection; stimulate innovation • Outcome: analysis, reflection, discussion, wider experience, improvement to teaching and learning. • Status of peer review: peer shared understandings and perceptions. Non-judgmental, constructive facilitated dialogue • Intended beneficiaries: mutual benefits for both peers, students • Conditions for success: a culture in which teaching is valued and discussed. • Risks: confirms existing practice, passive compliance, bureaucratic. Implementation • Voluntary participation • Reviewee is in control of the process: - makes initial contact with reviewer - reports when process is complete • Briefing for participants (online meeting – recorded) • Aimed for review of live sessions (but some review of recordings did take place) • A cohort of tutors on third level physical science modules • Tutor group tutorials (rather than module wide presentations) • Aimed to observe different modules (wasn’t always possible) SM358, SMT359, S382, S383 Practicalities: three stages 1. 2. 3. 4. Pre-observation contact Observation Feedback and reflection meeting (Notification that process is finished) Pre-observation discussion • The context of the tutorial (reviewers may not know the module). • Note-taking – are you both happy with the note-taking protocols (see below)? • Will whiteboards be shared beforehand? • Will notes be shared before or after discussion (see below) • How will the reviewer be introduced to the students? • Agree when feedback meeting will be held. From briefing session The observed session • • • • Turn up in good time. Tutor to introduce reviewer as agreed. Reviewer takes notes as agreed. Probably not a good idea to run on from the observed session straight into the feedback/reflection meeting. From briefing session Feedback / reflection meeting • Aim: to reflect on a teaching session in order to: - identify good practice - analyse situations which were less successful - feel confident about making changes to practice • NOT an aim to either identify everything that wasn’t perfect, or to solve every issue in online teaching. From briefing session Timeline • • • • • Number of participants: 12 (out of possible 20) Call for volunteers: April 2012 Online briefing: May 2012 Observations took place: June to September 2012 Feedback / reflection sessions after observations Practical problems • Timetabling (and use of recordings) • Permissions setting on VLE • Timescale for the process (too short) Evaluation Main tool for analysis is a structured conversation conducted at the end of the presentation (October 2012). Explores: •Practical problems. •Practical considerations. •Themes around peer observation. The extent to which the process meets the ideal of Gosling’s collaborative model (see also Martin & Double 2006) Evaluation: themes • Purpose: learning and reflection? Other peerobservation experience. • Apprehension (Bell, 2010) threatening nature (Cosh, 2006) • Nature of the feedback (Bell, 2010). Training observers to give constructive criticism (Cosh,2006) • Effectiveness (Bell, 2010) and uncritical positive feedback. • Benefits to observer, creative reflection (Cosh,2006) and Bell (2010) - 'modelling teaching skills'. • Ongoing professional development (Bell, 2010) • Identifying good online teaching (Swinglehurst, Russell, Greenhalgh, 2008 ) Evaluation: current status • Evaluative conversations were conducted in October / November. • Currently being transcribed to text. • Qualitative data to inform understanding of the themes. Wider adoption Some comments: • Findings of small scale qualitative survey should be a useful guide to issues for any larger scale implementation. • There is a specific need to develop discussion around practice in teaching using synchronous conferencing. • Important to create a model that is perceived to be collaborative. • Care needed to avoid the process being appropriated for other purposes (e.g. quality assurance).