Premier’s English Scholarship ICT-based peer and self-assessment Stacey Quince Campbelltown Performing Arts High School Sponsored by Aims of the study The increasing use of Information and Communication Technologies (ICT) as a learning tool is changing the face of pedagogy across the globe. ICT is transforming and redefining the way students engage in learning as a process, both within the traditional confines of the classroom and beyond. Emerging e-learning pedagogies must be embraced and adapted to suit the learning needs of students, whose learning will enable them to become valuable and significant contributors to local and global 21st century communities. The nature of e-learning pedagogies, combined with the exponentially expanding suite of software applications and web-based tools currently available, offer a timely opportunity to investigate ways to engage students more meaningfully in fundamentally important peer and self-assessment practices. In travelling through England, Wales, the Republic of Ireland, Northern Ireland and Scotland, it was my intention to investigate peer and self-assessment practices which had combined with e-learning pedagogies to improve student learning outcomes in the English classroom and beyond. E-learning pedagogies Professor Richard Andrews is undertaking significant research in emerging e-learning pedagogies which he discussed with me at the University of London. Together with Caroline Haythornthwaite, he is investigating the impact that e-learning is having within and beyond the school context. They argue that e-learning is a “re-conceptualisation of learning that makes use of not only instructor-led pedagogy but all the flexibility that asynchronous, multi-party contribution can bring” (Andrews and Haythornthwaite , 2007, p 19). Given the opportunity that “multi-party contribution” presents, it seems logical to empower students to contribute to the learning of their peers in a way that is more authentic and effective. Andrews argues that “e-learning is generated/made possible when the available resources for learning (existing knowledge, the communities in which we operate) are transformed by the learner (resulting in personal, social and/or political change) with the added dimension of peer as well as teacher discussion (in space, over a geographically much larger domain than in conventional learning) and asynchronously as well as synchronously (that is, in time)”. (Andrews, 2010, p. 4). This definition obviously applies most fittingly to online communities and virtual classrooms but has application in current contexts in a broad range of ways. He also writes, “rather than a hierarchical conception of knowledge, e-learning and its technologies promote a “flatter”, more democratic, more potentially dialogical relationship between the learner and knowledge” (Andrews, 2010, p. 7). Of particular interest is the concept of a more “democratic” approach to learning, which includes the essential components of “peer as well as teacher discussion". In a broad sense, one can argue that ‘good’ teachers have always fostered a democratic class culture and encouraged open class discussion. This model of learning, however, valorises peer assessment as a legitimate tool in providing feedback and highlights the importance of self-assessment inasmuch as the learner is an active contributor to the body of knowledge and therefore has a responsibility to ensure that their contribution is meaningful and accurate. Given that traditional models of assessment encourage competition rather than collaboration, it is also important for educators to investigate ways to harness the collaborative nature of e-learning and adopt it into assessment practices. Assessment for learning “Assessment for learning is any assessment for which the first priority in its design and practice is to serve the purpose of promoting pupils’ learning. It thus differs from assessment designed primarily to serve the purposes of accountability, or of ranking, or of certifying competence. An assessment activity can help learning if it provides information that teachers and their students can use as feedback in assessing themselves and each other and in modifying the teaching and learning activities in which they are engaged. Such assessment becomes ‘formative assessment’ when the evidence is actually used to adapt the teaching work to meet learning needs.” (Black, P. J., C. Harrison, C. Lee, B. Marshall, and D. Wiliam, 2004, p. 10). In my discussions with Dr Christine Harrison, Kings College, London, she indicated she had seen great success with teachers who engaged students in the learning process through discussing and negotiating learning intentions1 and success criteria2 with them. She emphasised that students are only able to achieve an identified outcome or learning goal if they understand that goal and are able to do what they need to do to reach it. Thus, she argues, self-assessment is essential to learning. (ibid, p. 14). In my interview with Paul Black, Emeritus Professor, University of London, he also discussed the importance of “training" students to engage in peer and self-assessment practices. He argued that students need to be taught the skills and habits of collaboration in peer assessment, “both because these are of intrinsic value and because peer assessment can help develop the objectivity required for effective self-assessment.” (ibid, p. 15). He highlighted the complementarity of peer and self-assessment, citing a number of instances where students were able to accept constructive criticism from their peers where they had not taken similar feedback seriously from a teacher and reasons that this may be because peer feedback is usually provided in language that the students themselves naturally use. Not only does the process of peer assessment assist students to achieve learning outcomes when their work is assessed and valuable feedback provided, but there is a further benefit in being the assessor. Black believes that peer assessment is also valuable because “students learn by taking the roles of teachers and examiners of others’. (ibid, p. 14). In every school I visited, classroom teachers who used peer assessment on a regular basis were able to substantiate that students benefited in both of these ways – as ‘assessor’ and ‘assessed’. ICT-based peer and self-assessment strategies Given the importance of peer and self-assessment in improving learning outcomes, and the opportunities provided through ICT, there is a range of ways in which these two crucial approaches to student learning can be married. Outlined herein are a number of suggested strategies which can be used in the English classroom and easily adapted for many other subjects. Learning intentions are statements which outline the expectations for the learning in an activity or a lesson or, sometimes, a series of lessons. (Harrison, C. and Howard, S. p. 23) 2 Success criteria are what students determine they need to achieve based on the learning intention. (ibid) 1 Audio and video strategies In addition to the importance of ongoing feedback from teachers, educators are aware of the importance of ongoing assessment by peers and learners themselves. There are several ways that audio and video can be used to provide detailed, ongoing feedback to students on their work. One such example is video to ‘screen capture’ to provide verbal feedback on a piece of ICTbased student work. This involves the teacher or peer assessor using the computer’s microphone, combined with standard computer tools such as the cursor and highlighter, to point to and provide detailed commentary on a student work sample. This might be something as simple as an essay, or as complex as a film, which the teacher or student can pause as required. There are many applications that can facilitate this type of feedback. At Queens University Belfast, lecturers are using a free Web 2.0 tool, Jing, for this purpose but Debut Video Capture can be used in a similar way. The feedback can be packaged as a flash file or saved on the internet which is also convenient when students are provided feedback in person, as the feedback can be recorded for the student who can then take it away and watch it as many times as required. Portable devices might also be used to capture audio or video feedback for peer and selfassessment purposes. Madeline Murray from the National Centre for Technology in Education (NCTE), Dublin, Republic of Ireland, discussed this increased uptake of cheap portable devices such as flip cameras and plug and play MP3 microphones in schools. In many schools, flip cameras capture feedback from students on their own or peers’ work, particularly in primary school or when students experience difficulty writing. Students are also using video cameras to capture debates, speeches and drama performances which they can then watch to peer and selfassess. Plug and play microphones allow students to provide feedback to their peers, or reflection statements on their own work for their teacher in audio form. In some classrooms, this is a precursor to a more formal written self-assessment. Alternatively, this technology can be used for viva voce, with students refining and reworking their response, as required, to ensure they submit the best possible product prior to it being summatively assessed. Professor Stephen Heppell, Bournemouth University, discussed with me the merits of such an approach in his EVIVA project some years ago. In this project, students had the opportunity to use their mobile phones for their viva voce on a major research project, “any time, anywhere”. In this instance, the university worked with a telecommunications company to harness the power of voice recognition software to ensure validity < http://rubble.heppell.net/archive/eviva/default.html>. These portable audio devices also offer great benefit in capturing discussions during group work, both for the students to keep as a record and for the teacher to be able to access the conversations of all students in any given class. Electronic documents A number of schools throughout the UK are using simple but effective software applications to engage students in peer and self-assessment on an ongoing basis. At Saltash.Net, students are engaged in cross-form peer assessment through shared Microsoft OneNote notebooks. Storing notebooks that are shared by a whole class on the school’s intranet allows students to view the work of, and provide feedback to, students in classes other than their own using the audio tool. With the success criteria clearly visible on the relevant notebook page, it is easy for peers to insert an audio file giving feedback on how effectively they believe a work sample has met the given criteria; what evidence they can find to support their assessment; and what the student needs to do to improve their work further. In ICT lessons, students were using the screen clipping tool in OneNote to provide evidence of having achieved each criterion as part of the self-assessment process. The Deputy Head, Dan Roberts, reported that teachers have worked hard with students over a long period of time to build a culture where peer and self-assessment is the usual and expected practice. At Sturminster Newton School, students are doing something quite similar using Microsoft Excel spreadsheets. I observed students developing advertising logos, with peer assessment occurring within the same class. After the class co-wrote the success criteria together, the teacher pasted these into a spreadsheet and emailed them to the class. Students then had to provide feedback to a peer on the logo they had developed according to the agreed criteria. The next step was for students to respond to the feedback they had been given, justifying the choices. The spreadsheet was colour-coded so that students understood which components needed to be filled in by themselves and which needed to be filled in by their peer assessors. Finally, students incorporated the relevant feedback into their refined logo. In other schools students were using track changes and comment boxes in Microsoft Word to provide commentary on their own work as well as feedback as a peer assessor. The success of these simple but highly effective methods was based on students’ deep understanding of the predetermined learning intentions or outcomes and often co-constructed success criteria. At a more technologically sophisticated level, students at St Ninian's High School in East Renfrewshire, Glasgow are participating in the e-scape Scotland project using ‘fizzbooks’ in a largely paperless classroom. In this project student work including photographs, audio files and written documents, is automatically collated into a portfolio on which the teacher provides ongoing written and verbal feedback . Students duplicate this process as peer assessors, editing and annotating each others’ work to refine and improve it. Students are highly engaged in the process, with teachers reporting a significant improvement in the quality of student work as a result. Web-based approaches Websites and social networking sites such as blogs, wikis and nings are also being used in many schools throughout the UK and the Republic of Ireland as tools for peer and self-assessment. Marilyn Murphy from NCTE showed me numerous examples of such work by students in both primary and secondary schools. This included students blogging to critically reflect on their own work; students co-constructing text on wikis and providing feedback to each other on their contributions; classes using Facebook to reflect on their collaborative learning; and schools working across contexts in the cloud to peer assess each others’ work. In many instances, at least in the early iterations, many of these student work samples were accompanied by explicit assessment criteria so that at every point the work could be assessed and refined to ensure maximum achievement. Many schools in Scotland are engaged in similar activities. In Cannich Bridge Primary School, a tiny two classroom primary school in the Scottish Highlands, students have developed culturally rich stories on the mythical Scottish ‘silky’ and are sharing them with students in schools in Russia and beyond. They then receive feedback from these students using it to inform their next writing process. Considering the remote location of the school, where students borrow books from the mobile library that visits the town once a week, this collaborative learning on a global scale is very powerful. In The Media School, Bournemouth University, Andrew Ireland and his team have developed CASPAR (Computer Assisted Self / Peer Assessment Ratings) <http://www.caspar.co.uk/>, a web-based peer and self-assessment tool that allows students to assess their own and other students’ contribution to the task as a whole, with each group negotiating and developing their own assessment criteria. The program allows students to input marks and comments on themselves and others at multiple points throughout the task. Students can view their cumulative mark as well as read the anonymous feedback from their peers. Students see real value in the process as the aggregated mark from CASPAR contributes to their overall grade. Online survey tools such as surveymonkey.com or zoomerang.com. could easily be adapted t be used in a similar way. e-portfolios The use of e-portfolios was a widely discussed topic across the UK and the Republic of Ireland. In England and Wales, portfolios of student work are replacing external examinations in some instances, and being used to report on achievement of National Curriculum standards in others. “To enhance consistency of judgements by subject teachers, schools are submitting sample portfolios of pupil work in every National Curriculum subject for scrutiny and feedback, backed up by school visits and verifier reports.” (Daugherty, 2009, p.64). Similarly, Scotland's new Curriculum For Excellence requires that, by 2012/13, students transitioning from primary to high school have an S3 Profile which records their progress and achievements in all subject areas. In some schools, these will include “learner statements" in which students reflect on the skills they have developed as well as how they have learned. Across the UK, an increasing number of schools have students developing these portfolios electronically (mostly due to ease of storage and sharing) with students reflecting on their own learning, both in terms of individual work pieces and on their progress over time. Professor Harry Torrance, Manchester Metropolitan University, suggests that electronic portfolios provide an outstanding opportunity to capture student learning as “expandable balloons". That is, they are an example of ‘divergent’ assessment, “oriented towards identifying what students can do in an open-ended and exploratory fashion.” (Torrance, 2007, p. 291). Facing the challenges There are a number of challenges which need to be addressed if teachers are to engage students in an ongoing basis in meaningful ICT-based peer and self-assessment. The first is to build their capacity to use a wide range of technology tools as well as a repertoire of meaningful ICT-based assessment practices. In those schools where students were using technology as a basis for quality peer and self-assessment which was embedded into school culture, extensive resources had been put into professional learning, including on school development days, through team teaching, in “twilight sessions” and as action learning projects. The second challenge relates to the design of assessment tasks and criteria. Whilst it is important to provide explicit assessment criteria, Torrance’s research in the area of post-secondary education and training has indicated that “transparency of objectives coupled with extensive use of coaching and practice to help learners meet them is in danger of removing the challenge of learning and reducing the quality and validity of outcomes achieved" (Torrance, 2007, p.292). Torrance and a number of other people I interviewed suggested that the emphasis on success criteria could become too strong, with teachers and students treating it as a “tick a box" exercise. He stressed the need for teachers to develop in students an understanding that not all criteria can be exemplified in their myriad forms and that, often, the most innovative and creative pieces of work, by virtue of their originality, deviate from a shared understanding of the set criteria. The third challenge revolves around access to computers and relevant software/web-based tools. Those schools in which e-learning, including technology-based assessment practices, had the greatest impact were those schools which were not only able to provide regular student access to computers , but which also developed policies and strategies allowing students and teachers to access and download the necessary software and tools. A number of teachers expressed their frustration at not being able to access appropriate sites or having limited access to the necessary hardware to implement these practices more widely and consistently. Without a doubt, however, the greatest frustration and challenge expressed by both teachers and academics was the pressure placed on schools to perform well in external examinations at the expense of what they believed constitutes quality teaching and learning. “The lack of synergy across the internal/external assessment interface is well known. The pressures exerted by current external testing and assessment requirements are not fully consistent with good formative practices, with the emphasis on good test results affecting the careers of teachers, and leading pupils to value their work only in relation to a perceived link with their successes or failures in tests.”(Black, P., C. Harrison, J. Hodgen, B. Marshall and N. Serret, 2010, p. 226). Time and again, teachers stated that the pressure to ensure that students performed well in external exams, combined with external inspections such as those conducted by OFSTED, meant that teaching and assessment practices were often directed towards “preparing students for the test” rather than engaging them in the innovative, creative, challenging, intellectual and collaborative practices underpinned by e-learning. “Resolution of such issues is an essential step towards replacing the damaging discordance between good teaching and learning practice and the pressures of summative testing.” (ibid, p. 227). E-learning which is supported by quality, ongoing and explicit peer and self-assessment practices provides a powerful educational platform for students. Not only has it been proven to improve learning outcomes for students in terms of curriculum achievement but it also prepares them as 21st century citizens who have the capacity to reflect deeply, collaborate effectively, think critically, and work creatively. Such an approach requires certain conditions however, namely - a broad and innovative curriculum, a systemic trust in teachers’ ability to assess student learning needs and meet these needs, appropriate resources and a balanced approach to assessment. In other words, a stark contrast to an inflexible curriculum and ‘high stakes’ external tests. References Andrews, R. and Haythornthwaite, C. (eds) (2007). The Handbok of E-learning Research. London: Sage. Andrews, R. (2010). “Does e-learning require a new theory of learning?”. Unpublished. Black, P., Harrison, C., Hodgen, J., Marshall, B. and Serret, N. (2010). “Validity in teachers’ summative assessments”, Assessment in Education: Principles, Policy & Practice, Vol. 17, No. 2, pp. 215–232. Black, P. J., Harrison, C., Lee, C., Marshall, B., and Wiliam, D. 2004, “Working Inside the Black Box: Assessment for Learning in the Classroom”, Phi Deltan Kappan, Sept. 2004, pp. 9-14. Daugherty, R. (2009). “Trusting the Judgement of Teachers: Changing Assessment Policies in Wales”, Education Review, Vol 22 No 1, pp. 61 – 68. Harrison, C., and Howard, S. (2009). Inside the Primary Black Box: Assessment for Learning in Primary and Early Years Classrooms. London: GL Assessment. Torrance, H. (2007). “Assessment as learning? How the use of explicit learning objectives, assessment criteria and feedback in post-secondary education and training can come to dominate learning”, Assessment in Education, Vol. 14, No. 3, pp. 281–294.