I CAN Secondary Talk An evaluation over three years Final report May 2015 Executive Summary Speech, language and communication underpin cognitive, emotional and social development – not only in early childhood, but throughout school, adolescence and into adulthood. Language and communication in secondary schools is often not seen as a priority, and recent curriculum and examination reform reflects this. Given the well evidenced link between good language skills and academic success this is of concern, particularly for the many students with poor language or speech, language and communication needs (SLCN). I CAN Secondary Talk is a whole school programme, which builds staff confidence in supporting language and communication in young people and in identifying those with difficulties. The programme aims to improve pupils’ behaviour and raise academic attainment, by focusing on language and communication. In 2011, Secondary Talk was evaluated as having a significant impact on aspects of practice in the classroom. There were also indications from schools that the programme had an impact on students themselves. These changes noted after one year piloting the programme were exciting. And yet the evaluation yielded more questions: ‐
‐
‐
Are the changes sustained over a longer period of time? Does Secondary Talk initiate other changes at a whole school level? Does Secondary Talk impact on the language/communication, attainment and behavioural outcomes of young people? At a time when changes to the curriculum mean a reduced focus on speaking and listening evidence of impact on learning for pupils is essential to ensure take up by schools. Equally, at a time of reduced spending for schools, demonstrating evidence of sustained impact is also important. This evaluation seeks to provide evidence to support both of these aspects. In collaboration with Dr Judy Clegg, University of Sheffield, Secondary Talk was evaluated over a three year period using a mixed methods, repeated measures approach. Following groups of schools engaged in Secondary Talk, data in the form of interviews with senior leaders, lesson observations and pupil focus groups was collected at the end of each academic year from 2012 to 2014. Running alongside this there was a small focused comparison study of groups of pupils in two schools: one implementing Secondary Talk and 1 one not. A range of standardised tests and non standardised tasks was carried out with groups of students with varying levels of language over a three year period. Progress was compared across the two schools. Headline findings Spread and reach ‐
Secondary Talk has continued to grow. Over the course of the three years, Secondary Talk has been rolled out into 52 schools, representing a total of 48,908 pupils and a total of 4487 school staff. ‐
Over half of these (52%) are academies. There is a trend for smaller than average schools and higher than average numbers of pupils eligible for Free School Meals and of those identified with SEN. ‐
Seven of the original eleven schools interviewed in 2012 year engaged in repeat interviews showing a continued engagement with Secondary Talk. Six out of the seven schools reported sustained or increased impact. ‐
While most schools started in discrete areas of the school – a year group or subject area ‐ the programme has grown to spread into more areas until in this third year it has become embedded as ‘custom and practice’ in what schools do on a day to day basis. At the end of year 3 all schools interviewed reported that communication was in their development plan. ‐
Through this evaluation there is now a good knowledge of how Secondary Talk develops in schools. Recognition at senior leadership level is essential in ensuring that the programme first spreads within schools, and then is sustained over time. Alongside this, it is important that staff have some simple, well defined, tangible approaches or resources to ‘hook’ Secondary Talk on to. ‐
The main barriers to its successful spread are identified as changes within school such as staff turnover, leadership change, or change in focus due to some external factor. Impact on pupils ‐
Three years after they started Secondary Talk, schools report that they see a continued impact on pupils’ learning, language/communication and behaviour ‐
This is supported by more in‐depth data collected from pupils in two schools in Derbyshire through a range of standardised assessments and tasks. Progress is more substantial in the first year after Secondary Talk, after which it is sustained but levels off. Progress in language and communication ‐
Assessment on standardised language tests shows that after Secondary Talk, students make statistically significant progress in aspects of understanding language and in expressing themselves. This progress is not seen in students who attend a school not engaged in Secondary Talk. ‐
The fact that there is no change to students’ underlying cognition (as measured by a test of working memory) suggests that progress in language may be related to engagement in a whole school programme. 2 ‐
This progress is most evident in students identified as having poor language skills compared to those with typical language development or those with identified SLCN. Although these students do still make progress. ‐
Despite this progress, student across language groups remain behind their peers with typical language ‐
After one year, 6 out of 9 pupils in the intervention school (66%) moved into the average range for understanding and explaining word meanings (dropping to 5 in the second year).In the same school 8 out of 9 pupils(88%) moved into the average range in their ability to listen to and answer questions about a spoken text (dropping to 6 in the second year). ‐
Students use significantly more word learning strategies following involvement in Secondary Talk. However, these do not in themselves necessarily help students to work out new word meanings. The study provides useful evidence for what might help them to use strategies more positively. Progress in learning ‐
Analysis of attainment data, shows that students make more progress after schools have implemented Secondary Talk, particularly in English as measured by school attainment data. This is as much as six times more than students in a school without Secondary Talk. ‐
Secondary Talk helps students to use more positive strategies for learning, and in particular for word learning. They use more word learning strategies after involvement in Secondary Talk, and this difference is statistically significant. This may help to boost progress at school if students are given help to use strategies positively. ‐
After Secondary Talk students are able to reflect on their performance more accurately. They have improved meta‐cognition, known to be important for learning. However, in areas of disadvantage they still tend to over‐report their competence. This may be the result of poor language skills. Secondary Talk may be most beneficial to this group of students with poor language in areas of deprivation. Progress in behaviour and social/emotional skills ‐
Schools see that understanding the link between communication and behaviour helps to ensure an appropriate approach to behaviour management. ‐
Schools’ environments vary and this can impact on the way pupils can make use of supportive strategies. ‐
During Key Stage 3 peer relationships can be challenging for all pupils. Impact on practice and whole school system change ‐
Two years after Secondary Talk there is continued impact of Secondary Talk on practice in schools and a high impact on changes to staff knowledge. This is reported by senior leaders, pupils and teachers. It is also observed in lessons 3 ‐
Teachers in Secondary Talk lessons are clearer about what they can do to support pupils’ speech, language and communication than teachers in non‐Secondary Talk lessons. Teachers in Secondary Talk lessons use strategies and approaches much more explicitly, they evaluate how effective they are, and encourage more active pupil involvement. ‐
After three years practice changes are sustained, senior leaders are able to identify approaches and strategies that are well‐embedded in school, but which originate from Secondary Talk ‐
Senior leaders report that Secondary Talk is increasingly embedded within the school, to the extent that it is not seen as a separate programme. There are changes at strategic level which ensure a focus on speech, language and communication. Some pupils can see these whole school changes. ‐
After three years, policy and practice change originating from Secondary Talk, has become ‘business as usual’, embedded in schools. Conclusions and implications for practice Implementing a whole school language programme, Secondary Talk, has a significant impact on students’ language, with resultant impacts on their learning in secondary school. Students with poor language make most progress, narrowing the gap between them and their peers. Significant impacts are seen one year after the programme starts, with progress sustained after this but levelling off. These impacts are accompanied by specific, structured changes to teaching practice in which strategies are introduced, students are taught how to benefit from them and progress is monitored. Furthermore, as a result of Secondary Talk school systems change so that a whole school focus on language and communication is identified as a priority, and embedded as ‘business as usual’. A qualitative difference is seen in the language levels of pupils, and in teaching practice in a disadvantaged area compared with a school in a more affluent area. Although both student performance and practice change in the disadvantaged school, they do not catch up with their peers or colleagues in a more affluent school. Secondary Talk may potentially have a unique role in supporting students with poor language in disadvantaged areas Change characterises secondary schools – at practice and policy levels. In order to sustain development for protracted periods, schools may need regular ‘refreshers’ or ‘prompts’. There appear to be associations between language level, engagement, school demographics, culture and students approach to learning which would benefit from further investigation. There are indications that engagement in learning is not a stable concept during adolescence. The Government’s ‘character education’ agenda will be helpful to see whether these traits can be changed. Evaluating whole school language initiatives can be methodologically challenging and this project presents some useful solutions to these challenges. Evaluating a programme that grows in schools which themselves also change, means that it is important to include a range of measures. Standardised assessments are useful when accompanied by more tailored tasks and qualitative interview data. The use of self‐report may be challenging for students who have poor language associated with deprivation. A skilled assessor, used to working with adolescents with language difficulties helps to gather accurate and comprehensive data.
4 1. Introduction and background The development of speech, language and communication continues through childhood, adolescence and into adulthood 1 . This development underpins cognitive, emotional and social development 2 , which are crucial for learning and later life chances including employment 3 . Secondary school gives pupils language and communication challenges – in both the academic and social curriculum. The sheer amount of information and new words, and the style of teaching being more focused on listening for longer periods of time can make lessons baffling. More complex social interactions and expectations can impact on successful relationships and friendships. And yet education staff report to be under confident in supporting pupils’ language and communication Many pupils in secondary schools have speech, language and communication needs (SLCN), particularly those in areas of social deprivation 4 . For these pupils, secondary school can be particularly challenging. Despite this, services to pupils with SLCN are under‐represented in secondary schools. At a time of significant cuts to public services, innovative models are needed to plug this gap in provision. A national programme of research into provision for children and young people’s speech, language and communication published its findings in 2012 5 . It described a three‐tier model of support for pupils, recognising that ‘All children need effective opportunities to develop their language skills in mainstream settings’ which was based on ‘effective classroom management and teaching’. It described this universal, quality first teaching as the ‘first phase’ in a strategic approach to supporting pupils’ language. The report states that if this is in place, then there can be more cost effective allocation of specialist resources at targeted and specialist levels. I CAN Secondary Talk is a whole school programme which aims to develop this quality first teaching. It builds staff confidence in supporting language and communication in young people and in identifying those with difficulties. The programme aims to improve pupils’ behaviour and raise academic attainment, by focusing on language and communication. Secondary Talk was designed specifically to speak the language of schools when talking about speech, language and communication. The programme’s standards and language indicators derive from systems that schools know and understand. Practical materials are arranged in chapters that relate to what teachers are doing in the classroom. Linking speech, language and communication skills firmly to the challenging school agendas of attainment and behaviour aims to help school leaders see the issue as relevant for all pupils and not just those with SEN or identified SLCN. Secondary Talk has been evaluated as having a significant impact on changes in staff knowledge and confidence in supporting students’ language and communication in secondary schools. It also found a significant impact on aspects of practice in the classroom. As well as this, there were indications from schools that the programme had an impact on students themselves. Teachers involved in the programme 1
Nippold M. (2007) Hartshorne M. (2012) 3
(UK Commission for Employment and Skills 2014) 4
Spencer et al 2012 5
BCRP 2
5 reported changes in students’ communication, learning and behaviour. 6 These changes noted after one year piloting the programme were exciting. And yet the evaluation yielded more questions: ‐
‐
‐
Are the changes sustained over a longer period of time? Does Secondary Talk initiate other changes at a whole school level? Does Secondary Talk impact on the language/communication, attainment and behavioural outcomes of young people? At a time when changes to the curriculum mean a reduced focus on speaking and listening 7 evidence of impact on learning for pupils is essential to ensure take up by schools. At a time of reduced spending for schools, demonstrating evidence of sustained impact is also important. This evaluation seeks to provide evidence to support both of these aspects. 2. Aims of project The current evaluation period covered three years, starting in April 2012 and ending in April 2015. Agreed outcomes against which this evaluation reports are: 1. Secondary Talk has spread successfully beyond the pilot phase to at least 35 schools and an initial understanding has been developed about how the programme develops within schools (to be evidenced by monitoring information collected from all schools about student and participating teacher numbers and spread of practice into year groups and subject areas) 2. Secondary Talk is achieving a demonstrably positive impact on pupils’ attainment, emotional and social skills and engagement with learning (to be evidenced by analysis of whole school data, and qualitative feedback from a representative sample of schools), as well as language and communication skills (evidenced by qualitative feedback from a representative sample of schools) 3. Secondary Talk is achieving a demonstrably positive impact on teaching practice, specifically teachers’ ability to effectively support young people’s communication to improve their learning outcomes, and on whole school policy and practice (to be evidenced by independent classroom observations and qualitative pupil perception data from a representative sample of schools, and interviews with school co‐ordinators and senior leaders) 3. Evaluation design. The Secondary Talk programme runs in each school for one calendar year, when an I CAN advisor supports the school. At the end of the year, I CAN support ends and successful changes to practice are validated. The aim is that changes are sustained and built on in the school over subsequent years. 6
7
Clegg et al (2011) DfE (2014)
6 March 2014. Second interim evaluation report March 2013. First interim evaluation report April 2015. Final evaluation report The evaluation and subsequent changes were designed in collaboration with Dr Judy Clegg at the University of Sheffield. The long‐term impact of Secondary Talk was evaluated over three years using a mixed methods, repeated measures approach. Running alongside this there was a small focused comparison study of two schools: one implementing Secondary Talk and one not. There were, in effect, four sample groups of schools: a) A sample of 8 schools starting Secondary Talk during year 1 of the evaluation. In these schools lesson observations were carried out in years 1 and 2 – in each school Secondary Talk lessons compared with non Secondary Talk lessons. In this sample, pupils and teachers from Secondary Talk lessons and non Secondary Talk lessons completed questionnaires, and pupils took part in a focus group. b) A sample of 11 schools which started Secondary Talk at the beginning of year 1 of the evaluation. In these schools interviews with senior members of staff took place at the end of each evaluation year. This reduced to nine schools in year 2 and seven in year 3. c) In a sub sample of these, using the National Pupil Database, whole school attainment data was collected and analysed d) Two schools which form a focused comparison study (see below). Data was collected at various points. ‐ Some baseline data was collected before the programme started, at the end of the first year when the schools are ‘in project’, and then again at the end of the following two years (observations, focus groups, questionnaires). 7 ‐
The first data collection point for some data was at the end of the ‘in‐project’ year, and then again at the end of each subsequent year (interviews with senior staff and whole school data collection).
Built into the evaluation was a small focussed comparison study of a group of pupils in two schools. The design of this study is a comparison of the potential impact of Secondary Talk on outcomes for young people. To investigate the impact of Secondary Talk, the progress of students in one school participating in Secondary Talk (the intervention school) were compared to students in another school that did not participate (the comparison school). This forms postgraduate study leading to a PhD by Mary Hartshorne, I CAN’s Director of Outcomes and Information. The spread of Secondary Talk was monitored over the three year evaluation period initially via feedback from I CAN advisors, and subsequently through interviews with senior members of staff each year. Appendix 4 shows the data collection schedule over the three years of the evaluation, together with main planned activities. Evaluation measures The spread of Secondary Talk was evidenced through: ‐
feedback by I CAN advisors ‐
telephone interviews with senior members of schools staff Impact on young people was evidenced using ‐
telephone interviews with senior members of school staff including some open questions and some where they were asked to rate impact on a scale of 1 to 10 (1 being no impact and 10 being impact across the whole school). ‐
impact data collected by schools as part of their involvement in Secondary Talk in year 1 of the evaluation(collecting evidence of change underpins the programme, and has been found to be a key factor in change within schools) ‐
a range of language measures administered to a sample of Year 8 pupils, and subsequently Year 9 and 10 in 2 schools. These include some standardised language assessments, tasks and a structured interview using rating scales ‐
attainment data collected via the National Pupil Database (NPD) from a representative sample of schools. This data set is end of Key Stage 3 data from before Secondary Talk and then after the programme implementation until 2013 when this data ceased to be collected nationally. In the two schools which formed the focused study, we collected end of Key Stage 2 and key stage 3 data from the group of pupils being studied (n=19), to compare progress made. Changes to teaching practice were evidenced using ‐
telephone interviews with senior members of schools staff (as above) ‐
Impact data collected by schools as part of their involvement in Secondary Talk in year 1 In years 1 and 2 8 ‐
a classroom observation schedule based on the tool designed by Sheffield University team as part of the initial evaluation of Secondary Talk, and on the ‘communication supporting classrooms observation tool’ developed as part of the Better Communication Research Programme for use in primary schools ‐
questionnaires for teachers and pupils, based on those used in the original evaluation ‐
pupil focus groups, structured around questions In year 3 observations, questionnaires and focus groups were planned in the two schools in the focused study. 4. Outcomes Secondary Talk has spread successfully beyond the pilot phase to 52 schools and an understanding has been developed about how the programme becomes embedded within schools Secondary Talk has rolled out into 52 schools since 2011, representing a total of 48,908 pupils and a total of 4487 school staff (teachers and support staff). We used each school’s Ofsted report (from the time they started Secondary Talk) to analyse the demographics of the 52 participating schools (see appendix 1 for the relevant charts). 52% (n= 27) of the schools are academies, but also represented are various other provisions ranging from small special schools (with 53 pupils) through to large community colleges (the largest having 1730 pupils). There is a greater proportion of ‘smaller than average’ schools (n=22, 42%) than ‘average’ (n=11, 21%) and ‘above average’ sizes (n=19, 37%). Data relating to Free School Meals (FSM) was available for 45 (out of 52 participating) schools. Of these 45 schools, over half (56% n=25) had a higher number of pupils eligible for Free School Meals, than the national average. This number has risen since 2012 when only 50% of the schools had higher than average numbers. Data relating to Special Education Needs (SEN) was available for 44 (out of 52 participating) schools. Of the 44 schools 57% (n=25) had above average numbers of pupils with SEN (this is not including the 3 special schools that have taken part in Secondary Talk since 2011). In 2012 the schools’ analysis showed a higher proportion (70% of schools n=28) having children with SEN. Data regarding number of students with English as an additional language (EAL) was available for 46 out of the 52 schools. Of this 46, 57% (n=26) had lower than average numbers of students with EAL. Three of the schools that have taken part in Secondary Talk since 2011 are special schools. Of the 49 schools that are not 27% (n=13) had on‐site specialist provision for pupils with some special educational need. Schools were located across England, the largest cluster being around London and the South East. 9 Secondary Talk Evaluation: Map of Secondary Talk Schools Growing Secondary Talk in Schools In order to chart the spread of Secondary Talk within schools, we have focused on a sample of schools (n=7) with whom we carried out interviews with senior leaders. Six of these schools have been interviewed twice before (in 2012 and 2013) and one has been interviewed once before (in 2013). In four of the schools the same person has been interviewed every year and in three at least two different people have participated in the interviews. The schools interviewed were largely representative of the sample as a whole. We used data from the schools’ most recent Ofsted reports to find they were mainly academies (71%, n= 5), with higher than average numbers of pupils with SEN (50%, n=3, as this information was only available for 6 of the schools) and higher than average numbers of pupils eligible for Free School Meals (57%, n=4). However most of the schools were larger than average (57% n=4) whereas in the whole data set the largest proportion are smaller. Half of the schools (n=3 as this data was not available for one of the schools) have higher than average numbers of pupils who speak English as an additional language (EAL) whereas the whole school set has on average fewer numbers of pupils who are EAL. Two of the schools (29%) have an onsite specialist resource which is in line with whole set of data. The chart below (‘2012: Where Secondary Talk started’) shows that within the sample group interviewed after one year of programme implementation, most schools (57% n=4) chose to start in a year group, 14% started in a particular subject (n=1), 14% used different strategies across different year groups (n=1) and 14% (n=1) started it as a whole school initiative from the outset. 10 However the interviews in 2013 showed a shift from specific year groups and subjects to a more whole school approach (see chart ‘2013: Secondary Talk after 2 Years’) within the sample group, the implementation has shifted, with 71% (n =5 ) of sample schools delivering Secondary Talk at whole school level; 14% (n=1) at year group level and 14% )n = 5) no longer using Secondary Talk as a discrete programme, but as part of their whole school approach. This shift reflects the growth of Secondary Talk in schools. 2013: Secondary Talk after 2 Years
1
1
Year Group
Whole school
No longer using ST as a discrete programme
5
This shift to a more whole school approach has continued into the third year. The participants interviewed at the end of year 3 described the approach as embedded and used across the whole school rather than with specific year groups or subjects. They describe whole school activities such as teacher professional development and using approaches such as ‘Talk Targets’ to demonstrate how Secondary Talk is now part of the school’s day to day activities 11 “I don't think if you came in to the school and asked anyone about Secondary Talk they would necessarily know what it was or that it has anything to do with it but across the school spoken language and the importance of spoken language is definitely a priority and I think the whole staff would agree with that and I don't think that's where we were three years ago at all so yeah I think the principles are there and I think they are deeply ingrained within the school practice now.” Intervention and Literacy co‐ordinator, Fazakerley High “I would say really it's become embedded in our whole school approach to RWCM (Reading, Writing, Communication and Maths) so we now have a director in post so she now leads on that whole agenda of improving reading, writing, communication and, maths across the academy. I guess that the introduction of pupil premium, the catch up money has made this a much bigger agenda within school so it's all just become one so I don't think we really think of it as ‘Secondary Talk’ anymore it's just all part of what we do.” Assistant Principal for Achievement and Standards, South Wolverhampton and Bilston Academy The seven senior leaders who were interviewed used the word “embedded” 15 times throughout the interviews to describe how Secondary Talk had been developed in the school. One school felt that they had tried a lot of new initiatives and were now in a period of taking what they had learnt and trying to making it part of the regular activities in the school. “We're in a position where we've done quite a lot, we’re not doing anything huge at the moment or things are settling and we need to think what’s the next step, how do we try to embed things.” Teacher in charge of the Speech and Language Base, Preston Manor Although the schools did not necessarily refer to Secondary Talk within their everyday practices the senior leaders could still identify strategies they were using that stemmed from the work they did as part of Secondary Talk. Examples from the interviews include focussing on tier 2 and 3 vocabulary, using tutor time to discuss world events, paired reading, ‘Talk Targets’, modelling good practice and checking understanding. They could also identify how Secondary Talk had impacted upon the strategic direction of the school. This included communication being in the school development plan, time in the staff meeting to discuss communication, systems for monitoring intervention and staff training. “Kat (the Director of Reading Writing, Communication and Maths) speaks to the staff every single Monday morning in our staff briefing, she's done one lot of whole staff training this term and she's got another lot of whole staff training next Thursday so it absolutely is part and parcel. We've got an objective in our school development plan” Assistant Principle for Achievement and Standards, South Wolverhampton and Bilston Academy Building on our analysis over the last 3 years, the pattern of development was analysed in more depth in one school. In this school, we charted the progression of Secondary Talk; this illustrates the process described in the majority of school interviewed. 12 The school journey over the first, second and third years for Fazakerley High School in Merseyside is plotted in the following chart: An example of successful approaches to developing Secondary Talk in schools Chart showing Fazakerley’s Secondary Talk journey over 3 years Year 1 Year 2 Year 3 • Focus on communication and ‘oracy’ embedded into school development plan •
•
•
Extended Talk Box activities from English department across whole school, as well as word of the week vocabulary competitions, voice box and student voice CPD and training programme developed so that whole inset days are focussed on developing communication across the curriculum Identification of children with SLCN now formalised • Focus on SLC(N) formed part of leadership observations 13 •
Importance of spoken language continues to be an identified priority in development plan •
‘deeply ingrained within school practice’ •
Staff regularly model good practice, pupils actively identify good and poor communication •
Lots of ‘No Pen Days’ and lessons based around discussion. •
Focus on speaking is timetabled in to the academic year, for CPD, in the school improvement plan, school evaluation and departmental evaluation. •
Detention is now ‘Resolution Time’ with more focus on talking and articulating what has happened. This will continue to be developed in the coming year. Embedding Secondary Talk in Schools: Facilitators and Barriers In order to find out more about how Secondary Talk develops in schools, thematic analysis of the interview data has been carried out for the last three years to identify the facilitators and barriers to firstly implementing Secondary Talk and then embedding it. This analysis has been used when supporting other schools to implement Secondary Talk, and across other programmes that work in secondary schools. Facilitators In the first two years, factors which supported growth of Secondary Talk were identified as being: 1) Maintaining speech, language and communication as high profile. Schools described making it an intrinsic part of everyday practice. For example, Park Hall Academy discussed how continued training and input focused on communication, helped maintain the profile as well as having specialist sessions on specific topics and interventions. 2) Hooking onto existing school initiatives 3) Cascading existing practice Ensuring there were ways to share practice was a crucial factor in growing Secondary Talk. The programme recommends that practical approaches to support can be trialled in small areas of the school. Once shown to be effective, the strategies are then introduced to wider parts using peer recommendation. Examples in schools were Teaching and Learning groups as established ways of sharing practice, with Peacehaven Community School discussed cascading elements of communication within the lesson observation format via Faculty Heads, which then became part of what was expected during lesson observations. 4) Senior leadership involvement. “it's pretty obvious but something like this won't work unless there's a route in to senior leadership and governor level ...it's about embedding it at the top and making sure it's driven through because it won't been driven through if it's not been discussed at an AIP level or it’s not at SLT level because they just won't have the clout.” Year 2 ‐ SENCO and Vice principal for inclusion, Park Hall Academy In the third year, the following five facilitators have been identified as key to sustaining Secondary Talk 1) Having identified staff responsible for Speech, Language and Communication; 2) support from senior leadership; 3) having communication identified in the school development plan; 4) hooking on to existing initiatives; 5) introducing tangible approaches that can be implemented without too much extra work. 1) Having identified staff responsible for Speech, Language and Communication Two of the schools mentioned that they now have specific staff that can spend time supporting communication. This enables communication to remain a priority and to ensure that new initiatives have a communication focus. The schools interviewed felt this was a very important factor in keeping Secondary Talk going. 14 “It absolutely is having somebody who’s specifically focused around language and communication, having a specialist teacher I think really” Assistant Head with responsibility for learning support, Peacehaven 2) Support from senior leadership The support from senior leadership means that communication can be supported on a whole school level. This support has been identified as crucial in interviews from all three years. The senior leadership have been instrumental in monitoring the impact of the programme. “We have focuses where they would look on learning walks and things to see because obviously I need to monitor how it is improving, how are these strategies showing themselves in everyday practice so senior leadership would feedback on that during learning walks and things as well” Intervention and Literacy co‐ordinator, Fazakerley High. 3) Having communication identified in the school development plan Having communication in the school development plan shows that the school has made a commitment to it. It also ensures that the school demonstrate that this is being followed up in the school’s everyday activities. “It’s part of our development plan so we will do what we've said we're going to do.” Assistant Principle for Achievement and Standards, South Wolverhampton and Bilston Academy “Yes it's on our school development plan so it's something every department is involved in that needs to be followed up by the SLT (senior leadership team) line management of course and some of our CPD is going to be on that as well.” Head of English and Senior Teacher, St Anne’s In 2012 33% of the schools interviewed (n=3) said that communication was in the school development plan and only one of those had it in there before the start of Secondary Talk. At the end of year 3 all 6 schools (100%) who completed the information said that communication was in their development plan. 4) Hooking on to existing initiatives One school has continued this year to roll out a campaign that has its roots in Secondary Talk. Cleeve School’s ‘Respect campaign’ has used the Secondary Talk principles and materials but has since been developed into a project specific to the needs of that school. The senior leader who was interviewed felt that the impact of Secondary Talk had been sustained even though they did not use the materials anymore because they are present under the guise of the Respect campaign. “The initial ideas came from Secondary Talk and then we ran with them.” Assistant Headteacher, Cleeve 5) Introducing tangible approaches that can be implementing without too much extra work 15 Looking back over the last three years schools can identify the most successful initiatives. These were ones where staff could quickly see the benefits and those that could be implemented without a lot of extra work or materials. These included activities such as such as introducing Talk Targets across the school, encouraging staff to use pause time when asking questions, experimenting with ‘word of the week’, a whole school approach to teaching tier 2 vocabulary. “Staff gave me a load of written feedback on it and pause time was the winner. Staff loved pause time; it was so easy to do.” Vice Principal for Inclusion, Park Hall Academy. This year schools were still working on new initiatives to maintain the focus on communication. “Because of the Talk Targets it (the impact) is maybe the same because they've kept the issue in the forefront” Teacher in charge of the Speech and Language Base, Preston Manor “Group work is still really key and the group work that we did as part of Secondary Talk so we need to revisit that but not necessarily the language bit but as a whole school movement towards improving outcomes generally.” Assistant Head with responsibility for learning support, Peacehaven “I think that we've identified for ourselves a lack of confidence in our young people through student voice work they've told us that they don't feel confident always to put their hand up and join in with the discussion. They have told us that even if they don't agree with what somebody's said they don't always feel they want to jump in and join the conversation so we've done a lot of work to try and build confidence.” Assistant Principle for Achievement and Standards, South Wolverhampton and Bilston Academy These future plans show that the schools still feel that speech, language and communication are important and the initiatives they have started could be developed further. The schools all had different aims for the coming year but were looking to use communication strategies to support them. For example one school wanted to look at improving behaviour, another at teaching academic language to improve exam results and another at offering more bespoke continual professional development to staff to ensure the training was appropriate rather than generic. Barriers In the first two years, factors which seemed to prevent or slow down the growth of Secondary Talk were identified as being: 1) Attempting too much change too quickly. Sometimes the simple ideas that worked well in confined parts of the schools took time to spread out. Preston Manor School gave the example of trying to implement ‘Talk Targets’ across the whole school as being challenging as it was difficult to ensure consistent application. 16 2) Staff turnover Keeping all staff at a similar level of knowledge and skill proved challenging in some schools, especially where there was year on year change. 3) Changes to the curriculum Even before its official implementation, curriculum reform and the reduced emphasis on speaking and listening was perceived as potentially having a negative impact on the school’s ability to take ownership of Secondary Talk, due to competing demands for teachers’ time. In the third year, schools identified two factors that reduced the impact of Secondary Talk 1) staff turnover and 2) sudden changes in the school such as going in to special measures. 1) Staff Turnover As with previous years staff turnover has continued to be a barrier to embedding Secondary Talk. This year budget cuts have been a factor, in one school for example they had lost the specialist communication support workers they had employed in previous years. We're in a slight hiatus now because of the changes of staff and things that have happened Teacher in charge of the Speech and Language Base, Preston Manor We had our specialist language person that we employed which I think I spoke to you about last time she's not here now and all that sort of stuff so as a school's vision of that had to shift Vice Principal for Inclusion, Park Hall Academy 2) Sudden changes in the school For one school in particular going in to special measures had really impacted upon their ability to focus on Secondary Talk. It had meant that although they felt that many of the approaches had been embedded they had been “diluted” while attention had been directed elsewhere. However the participant was hopeful that they would be able to regain the focus on Secondary Talk. They were also keen to point out that it was no fault of the programme. “I think there's enough of a legacy that it will recover and I think speech, language and communication will be back central. I mean it's already so central in so many high profile things it's just not sharpened and that’s the frustrating bit at the minute”. Vice Principal for Inclusion, Park Hall Academy Although none of the other schools identified such big changes some did refer to changes in inspections and the curriculum changes which make it difficult to measure the impact of an approach. “I think that it’s really difficult to make a sensible judgement when all of the indicators we use to 17 check progress the goalposts keep changing don't they? I think it is inescapable that it's getting more difficult to achieve the outcomes particularly if you're working with kids like we are from highly deprived areas so what we might assume is a small step in the right direction tends to become subsumed by the raising of the bar and the increased number of marks for spelling, punctuation, grammar.” Assistant Principal for Achievement and Standards, South Wolverhampton and Bilston Academy These changes were also highlighted as making it challenging to sustain the approach. “Sustainability is difficult and you still have to put in quite a lot of effort and in terms of we're going through a lot of changes at the moment with changes in legislation anyway so it's difficult to keep up the input and the effort” Teacher in charge of the Speech and Language Base, Preston Manor A full list of ‘facilitators’ and ‘barriers’ is in appendix 5 In summary, over the course of the three years, Secondary Talk has been rolled out into 52 schools, representing a total of 48,908 pupils and a total of 4487 school staff. Over half of these are academies. There is a trend for smaller than average schools and higher than average numbers of pupils eligible for Free School Meals and to be identified with SEN. While most schools started in discrete areas on the school – a year group or subject area ‐ the programme has grown to spread into more areas until in this third year it has become embedded as ‘custom and practice’ in what schools do on a day to day basis. We have developed a good knowledge of how Secondary Talk develops in schools, and are already applying this to our work with secondary schools. Recognition at senior leadership level is essential in ensuring that the programme first spreads within schools, and then is sustained over time. This can be through activities such as identifying it in the school development plan, allocating staff responsibility or building language and communication across initiatives rather than keeping them separate. Alongside this, it is important that staff have some simple, well defined, tangible approaches or resources to ‘hook’ Secondary Talk on to. The main barriers to its successful spread are identified as changes within school such as staff turnover, leadership change, or change in focus due to some external factor. 18 Secondary Talk is achieving a demonstrably positive impact on pupils’ attainment, emotional and social skills and engagement with learning, as well as language and communication skills Data to measure impact on pupils has been collected from three main sources: ‐
Qualitative data from interviews with senior members of school staff, including their ratings of impact during year 1 (n=11) and year2 (n=9) and in year 3 (n=7) their judgement about the continued impact. These interviews were carried out by a member of I CAN staff who was not involved in the project. ‐
Attainment data from the National Pupil Database. 13 schools who started implementing Secondary Talk at the beginning of Year 1 of this evaluation were identified within the national dataset; we looked at teacher assessment at KS2 and at KS3 in English, Maths and Science. The data for all 13 schools was amalgamated to compare attainment across three years: 2008 ‐ 2013. This represents before, during and after Secondary Talk. This enables us to see trends before the programme was implemented in progress students were making, compared to after Secondary Talk. The data does not look at the same students year on year, but it compares the attainment of students in the same year groups. They are controlled as far as possible regarding similar characteristics using the demographics of the schools. ‐
Results of a small comparison study of two schools in Derbyshire. Comparison of pupil outcome data in two schools Two schools were recruited, one implementing Secondary Talk and one who did not. Because of some difficulty recruiting schools to take part in the study, there was a delay to the start of data collection in the comparison school. This difficulty also means that although there are some similarities between the schools, they are not completely matched in their demographics. Both schools are smaller than average (with n=550 and n=673 pupils respectively) and have below average numbers of children from BME communities. However the intervention school has higher than average numbers of children eligible for pupil premium. It also has a higher than average proportion of students with disabilities and those who have special educational needs. The comparison school has below average percentage of pupils eligible for pupil premium and a below average proportion of pupils with disabilities or special educational need. We report, therefore on the progress made by pupils rather than actual levels which vary across the two schools (see below). A small sample of Year 8 boys was recruited in each school. Each school was asked to identify pupils with typical language, poor language and those with identified speech, language and communication needs (SLCN). An independent assessor – a speech and language therapist experienced in working with adolescents – carried out a range of assessments, tasks and interviews with each pupil group: ‐ Two subtests of the Clinical Evaluation of Language Fundamentals (CELF‐4), a standardised language assessment: Word Classes and Understanding Spoken Paragraphs 19 ‐ A standardised assessment of executive loaded working memory. This was included as a measure of cognitive ability that is not expected to change as a result of the intervention ‐ Tasks which investigated word learning skills, discourse and confidence in different communication situations ‐ Interview questions which asked students to reflect on their performance in tests and tasks, and to identify strategies which had helped them to carry them out In each school we also carried out interviews with staff, observations of practice and pupil focus groups. In the intervention school which implemented Secondary Talk (n=9) there is repeat data from 3 years Y8, Y9 and Y10. In the comparison school which has not implemented Secondary Talk (n=10) there is repeat data from 2 years Y8 and Y9. Quantitative data has been collated and analysed using SPSS. Aspects of the qualitative data have been transcribed and thematically analysed using NViVo software. October 2012 Intervention school December 2013 Secondary Talk in progress Baseline data Time 1 Outcome data Time 2 Comparison schools Not recruited Baseline data Time 1 December 2014 Outcome data Time 3 Outcome data Time 2 Further data collection, outside the scope of this current evaluation, is scheduled in a year’s time to monitor continued impact. ‐
An additional source of data was from pupil focus groups in 5 schools in year 1 and 2. Impact on pupils’ learning and attainment Over the three year evaluation, at the end of each year we interviewed a member of the management team who was not directly connected with the project in their school. Over this time, the number of school we spoke to reduced from 11 to 7, however in all of these schools staff spoke knowledgeably about Secondary Talk and its impact, even after three years. A copy of the telephone interview schedule used can be found in Appendix 6. In the first two years, participants in the interviews were asked to rate the impact on a likert scale of 1 to 10 (1 being no impact and 10 being impact across the whole school). After two years, asked to rate the impact of Secondary Talk on pupils’ learning, those who felt they could put a numerical rating on it (n=6) rated it as having a high impact, with 3 rating it as significantly high (8/9 out of 10). The average rating for the impact on learning and attainment was 8: Secondary Talk having a high impact on attainment and learning across most of the school. This was the same average rating as first year; schools felt that Secondary Talk had continued to have an impact on learning. 20 ‘it's just something that we've done because we've had to because we wouldn't be able to make progress if we didn't address these communication issues’ Assistant Principal, South Wolverhampton and Bilston Academy Seniour leaders were highlighting Secondary Talk as a major contributory factor in increasing attainment within specific subject areas and /or year groups, with some senior leaders making clear links between the Secondary Talk strategies and improved student learning. Although some students reported they found Secondary Talk strategies limiting, for example having to use set phrases in their presentations, mostly students identified they helped them to learn: ‘It really does pay off because to be able to hear what the fellow students are doing it helps affirm your ideas also allows you to develop them if you’re not quite there’ Three years on, interviewees rated whether progress has been sustained, increased or decreased since last year. In most schools, three years after initial implementation there continued to be a reported impact on pupils’ learning, communication and behaviour. Reflecting on the impact on student’s learning, four schools thought the impact had maintained, one felt it had increased and one that it had declined. I think being engaged and active in their learning has increased in three years. Head of English and Senior Teacher, St Annes 21 One school felt they could see impacts more clearly for pupils with SLCN or those with poor language ‘it continues to improve year on year for the specific groups of children’ , less so for students more generally but they could identify that the school had more of a focus on the links between language and learning: “the thing I am conscious of I think is the spreading out amongst staff is understanding of the importance of vocabulary, tier 2 vocabulary and also the language for thinking a lot a lot of work has gone in to helping students to understand what words like analyse, evaluate, describe, explain staff are much more aware that students can fall down in exams because they haven't understood the question because they haven't understood the thinking words that are in the question. I think that's been good and also "what does this word mean" not assuming that students will have a slightly more academic vocabulary and only the tier 3 words need to be taught. That's definitely been a very positive result.” Teacher in charge of the speech and language base, Preston Manor School Likewise, teachers in another school identify language skills in relation to engagement with learning “they've [students] told us that they don't feel confident always to put their hand up and join in with the discussion. They have told us that even if they don't agree with what somebody's said they don't always feel they want to jump in and join the conversation so we've done a lot of work to try and build confidence.” Assistant Principle for Achievement and Standards, South Wolverhampton and Bilston Academy Across the schools, as indicated by the ratings above, although there is acknowledgement that the impact has been sustained, a ‘tailing off’ is described with fewer new initiatives being introduced in lessons – rather that approaches have been integrated: “I'd say it's certainly been sustained. I know that in the first year of the project one of the geography teachers she felt that, she was looking at group interaction, and when she gave the project over to the class to do and allowed them a lot more freedom than they'd had previously she was surprised by how their attitude to learning was transformed and that would have been included in the first report. Now nothing specific as that came out last year but that doesn't mean to say that it's not going on. But that became more of an accepted way of working.” Head of English and senior teacher, St Annes In the focused study of 2 schools, there is some good evidence emerging from the baseline and follow up assessments carried out in the intervention school. In this third year, these are compared with the comparison school. Over the three year period, we have a wealth of data – both quantitative and qualitative. To explore the impact of Secondary Talk on learning, five aspects of the data were analysed in more detail. 22 1. Learning strategies It has been interesting to explore ways of analysing changes to the strategies used by students in approaching the tasks. Use of self‐regulation strategies is strongly associated with high achievement – the more strategies used, the higher achieving (Zimmerman and Pons 1986). We know from research that pupils with language difficulties find these key learning strategies more difficult (Pena et al 2007). In the second year of the evaluation, initial descriptive analysis of assessment transcription showed a qualitative change in approaches to learning for this group of boys in the intervention school over the two years. There are fewer reactive strategies e.g. ‘I just try and get on with my work’, ‘I just try harder’ and a move towards more planned, supportive strategies such as directing mental processes (e.g. visualising, ‘slowing it down in my head’), organising and planning (e.g. ‘I think about the task and where to start’, ‘I use a mind map’). Use of more interaction‐based strategies (asking a peer, teacher or other adult) is evidenced as having particular association with achievement (Zimmerman and Pons 1986) and this is a strategy that has increased in the intervention school over the first two years. Example ‐ Adam: pupil with language difficulties Strategies Awareness Reflection October 2012 at baseline October 2013 “I just get on with my work” Visualising ‘pictured it’ Needed prompting to ask for repetition “Ask my teacher what it means, look it up on the internet” Limited awareness of how difficult he found tasks. Self rated most things as ’easy’ Most rating went from ‘quite easy’ to ‘quite difficult’ Tended to say “I don’t know” “‘Word classes’ were a bit hard and the ‘understanding paragraphs’ were kind of a bit easy” “It’s kind of difficult as it didn’t tell you what other stuff’s happened” “it’s really struggling for me, I’m more used to ...” 2. Word learning strategies This initial analysis of data widened in the third year to look more closely at strategies pupils use for word learning. Knowledge of words is an important factor in learning. A large vocabulary is strongly associated with both reading proficiency and with school achievement in general in younger as well as older pupils (Snow et al 2009). In secondary aged students, measures of vocabulary have been shown to be the most effective and efficient predictors of performance in both reading comprehension as well as learning and understanding information in lessons (Espin and Foegen 1996). In both the intervention and comparison schools, students carried out a word learning task (full details of this in appendix 7). In this task, they had to make an attempt at working out the meaning of a complex 23 unfamiliar word such as ‘homogeneous’ or ‘prestidigitation’. The assessor noted the strategies they used, and then asked students to reflect on how they had done, and how they found understanding words in class. Over the three years, students in the intervention school increased in the number of strategies they used, over‐taking the students in the comparison school who decreased in the number of strategies used: Statistical analysis using a paired samples t‐test showed the difference between the number of word learning strategies used at time 1 compared to time 2 to be statistically significant 8 . This was not the case for pupils in the comparison school who used fewer word learning strategies. More detailed analysis of word learning strategies used showed that the intervention group used an increasingly wider range of strategies which involved not only looking at the word structure (e.g. breaking the word down, looking at the beginning and end) but looking at the word meaning (e.g. making links with other words) or seeking support (e.g. from a peer, adult or the internet). 8
A paired‐samples t‐test was conducted to evaluate the impact of the intervention on the number of word learning strategies used. There was a statistically significant increase in mean scores from Time 1 (M = 5.5, SD = 1.66) to Time 2 (M = 7.0, SD = 1.73), t (8) = ‐2.7 p .026 (two tailed), and from Time 2 (M =7.0 , SD = 1.73 to Time 3 (M = 8.0, SD = 1.87), t (8) = ‐2.26 p = 0.53.. 24 This shows a development in the way students look at word meanings which reflects what is known about how these skills change 9 . As part of the word learning task, students make a decision about the meaning of a word. Despite the increased number and range of strategies used, students in the intervention group make fewer correct decisions than the comparison group. Although, by the third year this is beginning to change: Qualitative analysis of interview data gives more information about this development. Despite not making correct decisions, over the course of the evaluation students in the intervention school make increasingly informed decisions about what words mean based on logic rather than ‘guesses’. So when asked to explain a decision, instead of responses such as ‘normally people always go for A’, ‘I don't know I just don't think the other ones are like it.’ or 9
Lubliner (2005) 25 ‘cos it just doesn't seem like it would be b or a’ Responses are more considered, for example: “I've kind of remembered what homo means not like homo just on its own but like do you know homosexual like when 2 people of the same gender like each other maybe that's something to do with 2 like the same things or the same type of thing so I'll guess the same type of thing” Students in the intervention school more frequently talked about specific cognitive strategies for word learning, such as mind maps, word of the week, keeping a record of new words – but these don’t necessarily help them to work out word meanings. This suggests there are processes involved for applying the strategies they are increasingly using. Qualitative analysis gives some indication of what may help this transfer process. For example, students in the comparison group more often use strategies such as talking around the word to help work out the word meaning: Interviewer: OK. why have you got rid of those Student: because homo means the same. I: yeah S: (talking to himself) so..to like the same things. the same type of things. (whisper gene) think it might be. the same type of thing The students in the comparison group also more frequently talked about words, whether this was about the structure of the word: “ er the same type of things, to like the same things and milk that has been boiled to a high temperature would really have a really complicated word” About the sound of the word “Erm.. because it‐ it doesn’t really sound like a talkative word . talkative you would use like ... conversation or something” Or about the meaning of the word “cos if you're to like the same type of thing you're just you wouldn't turn it into a scientific word” 3. Student’s view of their learning The information in the table on page 23 shows how Adam’s ability to talk about his learning and communication has changed. This ‘meta‐cognition’ is strongly associated with achievement (Frugt and Oort 2008), and is also a skill, which we know pupils with language difficulties find challenging (Pena et al 2007). In the interview included in the assessment session, students are asked ‘How easy is it to understand lessons?’. There has been limited shift in how pupils responded to this, as shown in the following graph. As lessons become more difficult it is likely that students will find it harder to understand, and likewise as they become more aware of this difficulty they are more likely to judge lessons as ‘difficult’. 26 A similar ‘limited shift’ patterns are seen across two years in the comparison school: In a further assessment task, students rated how confident they were in different communication situations. These were categorised into ‘social/emotional situations’ such as ‘talking about how you feel’, and ‘starting a conversation with someone you’ve just met’, and ‘learning situations’ such as ‘’keeping your attention going in lessons’ and ‘remembering ideas and words in class’. This provided a rich source of qualitative data, and also demonstrated that over the first two years of the evaluation, students in the intervention school became more confident in using language in learning situations (chart below). For example, a third more intervention pupils reported they found understanding the teacher in lessons easy a year after Secondary Talk, however, statistical analysis of the ratings showed these changes were not significant. 27 * students used a three point rating scale (easy, Ok, difficult) meaning that average changes are small By correlating the pupils’ language scores with their reported confidence in communicating, it was possible to see whether ‐ over the course of the study – students became more accurate in how they are able to reflect on their abilities. That is, did they way the rated their confidence in communicating reflect their actual communication ability. In the intervention school, at base line there was a strong negative correlation between language skills and confidence in communicating r= ‐.68, n=9, p= 0.43. That is, the weaker the language skills, the greater their confidence in communicating – based on self‐report. This could be described as poor meta‐
cognition. Two years later the correlation remained negative but was less strong (r=‐.55, n=9, p=0.12), with poor confidence levels typifying the school population. In the comparison school, at baseline there was a strong positive correlation between language skills and confidence in communicating r=.62, n=10, p= 0.05. The better the language skills, the more confident they reported themselves to be. After one year, correlation continued but with medium strength (r=0.04, n=10, p=0.19). Scatter graphs illustrate this in appendix 3. As this confidence rating is based on self‐report, this could indicate that it is meta‐cognition rather than confidence which is being measured i.e. the pupils’ ability to accurately judge their own skills. This suggests that a) School‐based factors other than language contribute to confidence and/or meta‐cognition. Language differences associated with deprivation may be a contributory factor. b) Strengthening language skills can enhance meta‐cognition and/or confidence communicating c) Even in more able students, this is not a stable concept – changes happen during adolescence 4. Standardised assessment of executive loaded working memory 28 As part of the assessment battery, in both schools a standardised Listening Recall test from the Working Memory Test Battery for Children 10 was carried out, designed to assess students working memory. Working memory has been shown to correlate with academic achievement in younger and older children, particularly in science and maths with adolescents 11 . Patterns of change were similar across each group, and these were varied. In both the intervention and comparison groups, some students’ scores increased, some decreased and some stayed the same. The difference between the change scores across two groups was not significant. 5. Attainment data analysis From each school for the pupils in each of the groups, we obtained attainment data from the end of primary school (this was Key Stage 2 SATS data), and at the end of Key Stage 3 (teacher assessment). Because the two schools are very different, we looked at progress made between these two time points. This represents progress from the beginning of Year 7 (age 11) to the end of Year 9 (age 13). We currently have data from the intervention schools showing progress between end of Key Stage 2 and end of Key Stage 3. We will have this data from the comparison school in June 2015 when pupils reached the end of Year 9. We also looked at the trends across the whole school, given that Secondary Talk is a whole school programme. From the National Pupil Database, we looked at the number of pupils reaching expected levels at the end of Key Stage 3 (Level 5) in English, maths and science. Data was analysed four years before Secondary Talk in the intervention school, and two years after it started (between 2008 and 2013). End of Key Stage 3 data ceased to be collected nationally after 2013. Collecting data across these years gives a picture of trends before and after Secondary Talk. In both schools, more pupils reach the expected level in 2013, but comparison of the scores from 2008 and 2013 shows the intervention school to have the biggest difference in numbers of pupils. A comparison of scores from 2012 (when Secondary Talk started in the intervention school) and 2013, showed that in the intervention school 29% more children reached expected level in English in 2013 compared to only 4.5% increase in the comparison school – nearly six times as much progress. This difference is pleasing, but the fact a very high percentage of pupils in the comparison schools are already reaching expected levels means they have less room for increase, so these findings are reported with caution. 10
Pickering and Gathercole 2001 11
Gathecole et al (2004) Working Memory Skills and Educational Attainment: Evidence from National Curriculum Assessments at 7 and 14 Years of Age Applied Cognitive Psychology 18 29 Secondary Talk starts here Data analysis on a wider sample of Secondary Talk schools was also carried out. 30 Data analysis from the National Pupil Database The data from 13 schools was amalgamated to compare attainment across six years: 2008‐2013. This represents before, during and after Secondary Talk. This allows us to look at trends for the whole school, which will include pupils who have been involved with Secondary Talk. In the amalgamated data, in all three subject areas before Secondary Talk (2008‐10) the percentage of pupils reaching expected levels decreased. After Secondary Talk (2010‐13)the trend was for it to increase and then for this progress to ‘tail off’, with some decrease. More detailed analysis on progress made in one curriculum area across this groups has been interesting: 31 Across the schools, the trend following Secondary Talk is for more pupils to exceed expected levels in the years, compared to decreasing numbers before. Analysis also looked at the progress made in different core subjects before and after Secondary Talk. The following bar chart shows trends in English across the group of schools. The trend is for pupils to make more progress after Secondary Talk is implemented. It is not possible to say that these trends are a direct result of Secondary Talk, but in collecting data from a range of sources we aim to triangulate and therefore strengthen evidence of this relationship. To summarise, there are indications that in the year after Secondary Talk there are impacts on the progress students make in school. There are also indications that this progress tails off after a year. The fact that there is no impact on students’ underlying cognition (working memory) suggests that progress in other aspects of learning may be related to engaging in a whole school language programme. Students make more progress after schools have implemented Secondary Talk, particularly in English as measured by school attainment data. This is as much as six times more than students in a school without Secondary Talk. Students with poor language in disadvantaged areas make progress, but are still behind their peers in more advantaged areas. Secondary Talk helps them to use more positive strategies for learning, and in particular for word learning. They use more word learning strategies after involvement in Secondary Talk, and this difference is statistically significant. This may help to boost progress at school if students are given help to use strategies positively. After Secondary Talk students are able to reflect on their performance more accurately (increased meta‐
cognition), but in areas of disadvantage they still tend to over‐report their competence. This may be the result of poor language skills. 32 Impact on speech, language and communication In the first two years of the evaluation, schools who took part in the Secondary Talk interviews were asked to rate impact on a scale of 1 to 10 (1 being no impact and 10 being impact across the whole school). Over the first two years, the average rating for the impact on language and communication rose to 7.6: Secondary Talk having an impact across most of the school. Many improvements were noted for all pupils, with some schools noting that changes had taken a while to bed in: ‘we were starting to see dividends in terms of in the classroom the enhancement of children's language skills and that's children speaking as individuals, children working in a group, children working in pairs being able to demonstrate a much higher level of vocabulary being able to extend that vocabulary in different forms, language and communication’ SENCO and student support manager, Fazakerley High School In this third year, interviewees rated whether progress has been sustained, increased or decreased since last year. All six senior leaders interviewed this year who responded to this question felt that the impact on language and communication had either improved or been sustained. Schools report both staff and pupils to be aware of the importance of language “I think that there’s, amongst pupils, a willingness to acknowledge a difficulty in language, there's no shame in not knowing. It's that kind of culture. So there's an openness about it and from the teacher's perspective there's a greater understanding of the need to reshape and rephrase in order to make it accessible. “ Head of English and senior teacher, St Annes Pupil focus groups carried out in four schools one and two years post‐implementation of Secondary Talk illustrated the difference between students who been involved in Secondary Talk lessons, and their peers. Common themes came out across both groups, but overall the pupils in Secondary Talk groups had more to say and were more aware of the importance of communication. The average length of a focus group was 13.31 minutes, but for Secondary Talk focus groups this average was 17.13 minutes and for non Secondary Talk groups 7.21 minutes. For example: From the same school, asked what bad communication is.... Secondary talk group: ‘Bad communication is mis‐communication when an idea isn't transferred effectively and can therefore be misconstrued in the process. If I was to tell you something, have my meaning behind it but you take a different meaning from that I would be a bad communicator’ Non Secondary Talk group ‘Like if you've got your back to someone’ 33 In both year one and year two, students were aware they changed their communication style to suit different occasions and different people. Compared to non‐Secondary Talk groups, students in Secondary Talk groups were more likely to identify a purpose or reason for doing this: ‘I sometimes alter the way I communicate depending on the person I'm communicating with because I feel it really helps to try and get in tune with the way the other person's doing it to be able to transfer your ideas.’ Over the three years of the evaluation, there is increasing evidence about impact on language and communication from pupils in the focused study. From the outset, there have been noticeable differences between the language skills in the two schools in Derbyshire. Staff selected students in three different groups: 3 students with identified SLCN, 3 with poor language and 3 with typically developing language and communication. There was agreement between their judgements and assessment results: those who were identified with most severe SLCN scored lowest on assessments. These two schools are located in very different demographic areas. The intervention school is in an inner city area, an area of significant deprivation whereas the comparison is in a rural market town with much less deprivation. There were differences at baseline between the two schools: This pattern reflects what is known about the association between poor language and social disadvantage. In this third year, there is now data from three years in the intervention school, and two years for the pupil group in the comparison school. This allows some comparison between the two. Because of the demographic differences between the schools, and the differing levels of language (above), the progress of pupils rather than actual levels is compared. 34 The chart above shows average standard scores on subtests ‘Word Classes’ and ‘Understanding Paragraphs’ subtests of the Clinical Evaluation of Language Fundamentals (CELF‐4) 12 assessment for both the intervention and the comparison group. Time 1 scores are in pastel colour, the heavier colours show average scores for time 2. This shows more progress for the intervention group than for the comparison group. The gap between them narrows. This progress is shown in more detail below for each pupil group: 1. Change in average standard score for the intervention group. 12
Semel E. and Wiig E. (2006) Clinical Evaluation of Language Fundamentals – Fourth Edition Pearson Assessment 35 2. Change in average standard score for the comparison group There is more progress for pupils in the intervention school than for the comparison school. Statistical analysis showed there to be a significant difference in progress made over the first year of Secondary Talk for the intervention school, with a large effect size 13 i.e. the difference in scores in unlikely to have occurred by chance. Similar analysis with data from the comparison school did not show significance. The fact that this progress is not seen in more cognitive‐based tests (Listening Recall from the Working Memory Test Battery 14 ) suggests the progress may be specific to language and communication. These CELF subtests measure competence in: ‐
processing information ‐
listening to complex information and answering literal and inferential questions ‐
being able to identify words that are related ‐
being able to explain reasoning – put thoughts into words. All of these tasks relate to classroom demands. A number of students in the intervention moved from out of average range into the average range, as shown on these charts: 13
A paired‐samples t‐test was conducted to evaluate the impact of the intervention on students’ scores on the Word Classes subtest of the CELF (Total score). There was a statistically significant increase in mean scores from Time 1 (M = 5.4, SD = 2.29) to Time 3 (M = 7.3, SD = 3.0), t (8) = ‐4.857 p .001 (two tailed). The mean increase in Word Classes Total score was 1.88 with a 95% confidence interval ranging from .99211 to 2.78567. The eta squared statistic (.74) indicated a large effect size. 14
WMTB-C, Pickering & Gathercole, 2001 36 8 out of 9 pupils (88%) moved into the average range in their ability to listen to and answer questions about a spoken text (dropping to 6 in the second year). After one year, 6 out of 9 pupils (66%) moved into the average range for understanding and explaining word meanings (dropping to 5 in the second year). 37 The following charts separate out the pupils into three distinct groups: from left to right a) those with typical language skills, b) those with poor language and c) those with identified language difficulties. This shows clearly the amount of change across groups, but also that in the school with Secondary Talk, the most progress is made for pupils with poor language. Finally, looking at the progress made in the intervention school over three years, a pattern is seen where initial progress is sustained, but tails off in the third year: 38 Although the information gained from standardised language assesments shows clear developments. The range of more qualitative data from bespoke tasks, together with feedback from staff and pupils makes these findings more robust. As previously noted, following involvement in Secondary Talk students use more word learning startegies, and this difference is statistically significant. In summary, assessment on standardised language tests shows that after Secondary Talk, students make statistically significant progress in their ability to process information, listen and respond to complex information, identify when words are related and in being able to put their thoughts into words to explain their own reasoning. This progress is not seen in students who attend a school not engaged in Secondary Talk. The fact that there is no change to students’ underlying cognition (as measured by a test of working memory) suggests that progress in language may be related to engagement in a whole school programme. This progress is greatest in students identified as having poor language skills compared to those with typical language development or those with identified SLCN. Although these students do still make progress. After one year, 6 out of 9 pupils(66%) moved into the average range for understanding and explaining word meanings (dropping to 5 in the second year). 8 out of 9 pupils(88%) moved into the average range in their ability to listen to and answer questions about a spoken text (dropping to 6 in the second year). Progress is more substantial in the first year after Secondary Talk, after which it is sustained but levels off. Students use significantly more word learning strategies following involvement in Secondary Talk. However, these do not in themselves necessarily help students to work out new word meanings. The study provides useful evidence for what might help them to use strategies more positively. 39 Impact on social, emotional skills and behaviour In the first two years of the evaluation, schools who took part in the Secondary Talk interviews were asked to rate impact on a scale of 1 to 10 (1 being no impact and 10 being impact across the whole school). In the second year the average rating for the impact on behaviour was 7.6: Secondary Talk having an impact across most of the school. This represented a slight increase of 0.1 from the first year. In most of the schools (n=7) senior leaders specifically mentioned the link between language, communication and behaviour – and were able to identify examples of changes to students’ behaviour. They were clear that due to improved communication skills, students were less likely to act up and more likely to sort things out verbally. ‘They're allowed to express in a way that they're not throughout the rest of the school and they're given the tools in order to express. So whether that's a verbal way and using language that is more appropriate than they might otherwise use.’ Instructor, Blackfen School for Girls In some schools, robust monitoring systems showed a decrease in behaviour incidents since engaging in Secondary Talk, and these are linked to the strategies and approaches adopted through the programme. ‘Absolutely and in pupil's behaviour we've still seen a continuation in drop in serious negative behaviour referrals and we've seen that year on year in the last 2 years we've seen that continue to drop.’ ‘Yeah, our pupil behaviour is still, I mean it has been dropping for the last 4 years, exclusion has dropped, our permanent exclusion has dropped, those students with SLCN because we're getting them diagnosed quicker because we're able to recognise it.’ Assistant Principal, Director of Inclusion Park hall Academy In student focus groups, while students could see the benefits of talking in groups and the way it supported their learning they also often linked talking in lessons with ‘messing around’ or getting into trouble. Some schools talked about a ‘change in culture’ and about taking time for changes and expectations to embed. This is true for both staff and students. In the third year interviews, six out of the seven senior leaders interviewed felt that the impact on behaviour had been sustained or increased. Overall, interviewees found it harder to identify specific impacts, but they could see that they had adopted more talk‐based approaches to behaviour management. “we've got work to do but it will involve talk so things like detentions are becoming Resolution Time and there's more debate, well discussion and reflection of "what did you do, what could you do" and making them more meaningful “ Intervention and Literacy Co‐ordinator, Fazakerley 40 In one school, the senior leader identified that for groups of students, behaviour had improved and they were able to link this to the way that behaviour incidents were managed: “I think it has had a positive impact because something like teacher talk for example if teachers are not talking so long then children tend to be less lacking focus in class and that sort of thing. And certainly for children with significant difficulties we've got better and better at managing the things that cause problems for them and using different ways of responding to issues when issues occur” Assistant Head with responsibility for learning support, Peacehaven Similarly, in another school, impacts for students with language difficulties could be seen: “The talk targets we used last year were very much to do with how students relate to each other. Whether they're letting other people finish what they've got to say or making positive comments so all of those are important in students understand what good social communication skills look like and sound like I think that that has had a very positive impact” Teacher in charge of the speech and language base, Preston Manor However, for the wider group across the school there were challenges identified with recognition that there was more work to be done in helping staff see the link between communication and behaviour “we wanted staff to understand the students when they're reporting something that's happened don't always have the narrative skills”. In the focused study, it has been more difficult to identify specific impacts on social and emotional skills. Students were asked to rate their confidence in different communication situations. There are slight shifts in the rating to show that students rate social and emotional situations as easier after a year of Secondary Talk; a shift not seen in the comparison school. However these changes are not significant. 41 Interviews with the students in both groups reveal a lot about the nature of learning in each school. In the comparison school, all students were well engaged with school. When questioned they see the link between learning and outcomes. They relate well to both pupils and staff. During their interview, they equate difficult lessons with the subject matter ‘things that are new’ or with specific teachers, rather than behavioural issues. They feel confident at asking for help; this is not seen as a weakness. Pupils with SLCN feel supported by their peers, but can find lessons noisy or distracting. In the intervention school, all students were less well engaged with school, shown in the above chart. Although they see the link between learning and outcomes, they are less positive about relationships with staff, and can find peer relationships challenging. It is interesting that even in the comparison school it is peer rather than staff relationships that take a dip in Year 9. During their interview, they describe their experiences of speaking out in lessons. 7/9 of the students rated this as ‘difficult’, describing it as embarrassing – four students describe a fear of being laughed at. Only 3/10 of the students in the comparison school find speaking out in lessons difficult. Most pupils find the class environment distracting, describing it as noisy. Some pupils talk about trying to block the noise out in order to concentrate.
42 In summary, schools do continue to see that understanding the link between communication and behaviour helps to ensure an appropriate approach to behaviour management. Schools environments vary and this can impact on the way pupils can make use of supportive strategies. During Key Stage 3 peer relationships can be challenging. There appears to be less of an impact on behaviour and social/emotional skills than other areas. Secondary Talk is achieving a demonstrably positive impact on teaching practice, specifically teachers’ ability to effectively support young people’s communication to improve their learning outcomes, and on whole school policy and practice Over the first two years of the evaluation observations, pupil and teacher questionnaires and pupil focus groups were carried out in Secondary Talk and non Secondary Talk lessons (2012 n= 8, 2‐13, n=5). This data was collected before the start of Secondary Talk and then again after one year. In addition to these observations, pupil focus groups and interviews have been carried out in the two schools in the focused study, allowing comparison between a Secondary Talk and non Secondary Talk school. As well as this data, senior leaders in their interviews were asked questions relating to changes to practice. What pupils say Pupils were asked to rate how strongly they agreed with various statements about how their teachers supported their communication using a 5 point scale: strongly disagree through to strongly agree. There were no substantial differences between 2012 and 2013, nor between Secondary Talk and non Secondary Talk lessons. Most students agreed strongly with most of the statements. In focus groups, pupils were asked to consider good and poor communication, whether they had noticed any changes, and to reflect on what helped them to learn in lessons – in particular learning new words. This produced some useful evidence of changes to teaching style in comparison with the previous year: Four key themes were identified: a)
More interaction in lessons Some pupils who had been involved in Secondary Talk lessons commented on the changes to the types of activities they were involved in. They saw more opportunities to interact, and were able to describe activities where they had been supported to develop communication: ‘Our teacher nominated one person in each group to be a leader and then we were rated at the end by the other team members. It was good’ Some pupils were able to link this to how it helped them learn: Pupil 2: Because you knew who to turn to to ask the questions in your group whereas the others are like "what are we doing" "no idea" "what are we doing" no idea”! Pupil 1: Yeah and then at the end the leader knew what to do to be a better leader.” 43 ‘if you know everyone else is [thinking the same thing] through discussion it makes you feel more confident and then you'll be able to learn better and achieve more’ b)
A wider range of approaches to encourage communication Pupils noticed teachers using a wider range of approaches, which encouraged communication. They talked about teachers explaining things in different ways to ensure everyone understood, using colour coded ‘log books’ to flag when they didn’t understand thus allowing time for thinking. Not surprisingly, some approaches were considered more effective than others, for example, students talking about the ‘no hands up’ rule: ‘If I was to be totally honest sometimes I feel whilst it’s really important to get people on the same page it can drag the lesson down a bit if someone really obviously isn't going to answer this question. They just spend 5 minutes going "ahhhhhh" and it depletes the rest of our time.’ c) Approaches for word learning Students talked about a range of ways to help with learning new words in lessons. They did this last year, but then the focus was on using a dictionary, teacher repeating or writing down words. This year pupils talked about a much wider range of strategies: they talked about key words, about writing down the explanations, referring back to what they already knew, the importance of hearing words in a sentence and using words in context. Students were able to explicitly identify these strategies as being helpful. ‘I would say the best way to teach words to people is not through copying it down it's through using the words in context which allows me to absorb the meaning on a deeper more significant level’ There was clearly a focus on word meaning, though students did not always view this positively (Some teachers) "go on forever (about a new word). It does get boring but it kind of helps us remember” d) Talking and doing helps learning Most pupils in focus groups reflected on ‘good’ and ‘bad’ lessons – ones that they found it harder to learn in. Key features of more positive lessons for learning were those where teachers made time for questioning, and supported students with this. Students talked about not being put on the spot in front of the whole class: ‘So it’s good to have teachers who don’t just stay still, who walk around the classroom and look at your work to see if you’ve done it right and if you haven’t they just say “ooh just think about this”’ ‘Sometimes they make it more enjoyable and then you like focus more. But sometimes the teacher just stands there talking all the time’ Although these reflections are encouraging, we should be cautious about the way this information is used. The groups were different pupils to last year, and from different year groups. 44 Focus groups were also carried out in the Derbyshire schools in year 1 and 2 of the study. In the second year pupils from both schools were asked to reflect on the last year and say whether they thought their lessons had changed. In the Intervention schools the pupils identified Circle Time as something new that had been introduced. The pupil’s felt that this was preferable to lessons with a lot of writing and that they enjoyed the opportunity to share their views. Like it's better to discuss things in the classroom Coz everyone gets their point across. In the comparison school the pupils struggled to identify any changes to the teaching. Some of the group felt that they had more time for discussion in lessons and some felt that they had less. One pupil suggested that they had recently been working on projects that leant themselves to discussion and this might be why some of the group felt there had been an increase in discussion based lessons. We’ve had a lot more discussion based topics in philosophy and PSE like in human rights. In both of the schools pupils were able to list different language‐based activities and strategies. There were more examples in the comparison school: “Some lessons are really good so like last year in humanities we studied a big project and then picked which side we were on and had a big class debate and things like parliament like one side the other side and then we had to have the communication skills to put your side across.” What teachers say In the second year of the evaluation, repeat teacher observations were carried out in five of eight schools. There was no increase in the use of communication‐supporting strategies used in lessons, and no observed differences between Secondary Talk and non Secondary Talk lessons. Methodological limitations may have influenced this : Very often different teachers were observed each year, teaching a different year group. As the programme develops in schools it is increasingly difficult to find comparison non‐Secondary Talk lessons. However, observations have also been carried out in the focused study in the two Derbyshire schools. The same teachers were observed teaching one year apart. One school had been taking part in Secondary Talk in that year and the other had not. The teacher in the Intervention school was observed to be using 13 more strategies in the second year than they were in the first (a 41% increase). In the comparison school the teacher was observed to be using fewer strategies than the previous year (a decrease of 17%). In the intervention school the teacher used many more strategies to support pupil interaction in the second year for example structuring the interactions and explicitly referring to interaction skills. Strategies specific to Secondary Talk were observed: structuring pupil interaction, interaction skills being made explicit, extending student responses and use of visual support. There were seven incidents of the teacher expanding the pupil’s response to a question (compared to only one the year before) and four examples of the teacher using visually supportive strategies (compared to 1 the previous year). The teacher planned in more time for pupil interaction in the second lesson and also introduced the lesson in a much clearer way. The chart below shows that despite this increase, there were still qualitative differences between lessons in the two schools. Practice in the comparison school is still stronger in supporting pupils’ communication. 45 Teacher questionnaires were completed at the beginning of Secondary Talk and then one year later. These were completed by 5 teachers (4 teachers involved in Secondary Talk and 1 teacher who was not). Staff were asked to rate on a 5 point scale how strongly they agreed or how often they used certain strategies (strongly agreed/always to strongly disagree/never). They were also asked to list the strategies they used. Although only a small sample returned the questionnaire, results are interesting. In the follow up questionnaire, teachers from Secondary Talk lessons agreed more strongly than the previous year with all except one of the statements. The reverse was true for non ‐Secondary Talk lesson, where agreement decreased. More interesting is the analysis of the strategies they reported using. The differences seemed to be in the amount of structure and level of explicitness when talking about language and communication, the focus on evaluation, and in the involvement of students: Before Secondary Talk, baseline observation Many strategies were reported to be in use, reflecting good quality first teaching, for example: One year after Secondary Talk started Again, many strategies reported: ‐
A continued focus on key words ‐
Highlighting key words, and word learning: a focus on explaining etymology and spellings ‐
Clear links to national curriculum requirements ‐
Opportunities for discussion and group work, debate ‐
Mentions of interactive activities ‐
Use of visuals such as role play, displays, ‐
Many visual approaches identified 46 modelling Some specific, focused approaches used e.g. use of signposts and explicit phrases for students to use in lessons. However, many more general approaches e.g. ‐
Reading out aloud ‐
An open and safe environment ‐
Encourage pupils to repeat ‐
Insisting on speaking in full sentences, on full and clear answers Limited focus on supporting pupils in how to develop communication skills, or in evaluating how effective their approaches were e.g. ‘I think I do but am not sure it is effective’ Differences were shown in: ‐
Increased structure given to students so they know how to improve communication skills: ‘explicitly teach the differences between formal and informal English’, ‘describe key skills, showing them videos’ ‐
A focus on evaluation – both from the teacher but also between students ‘feedback areas that need improving’ , evaluating each others’ performance’, ‘reflect and compare to key skills/talk targets’ ‐
Increase involvement of students ‘choose a pupil to explain the task in their own words’, ‘encourage students to act as a dictionary investigator’ It has not been possible to collect observation or focus group data in the third year of the evaluation. Where we were comparing Secondary Talk lessons and non Secondary Talk lessons in the same school, because the programme develops to become embedded within school after two years this comparison became impossible. In our focused study in Derbyshire, staff absence prevented us from observing the same teacher for three consecutive years. This is planned for later in 2015. An interview with a senior leader in the intervention school was carried out. She reported that three years on from when the programme started, the school had stayed focused on enhancing the talk in lessons, with teachers very skilled at questioning, using questioning techniques to encourage thinking and more detailed answers in students. What senior leaders in school say In a different dataset, collected at the end of the project year, senior school staff were interviewed. In the first two years, they were asked to rate impact on a scale of 1 to 10 (1 being no impact and 10 being impact across the whole school). After two years, senior staff reported impact on staff knowledge to be sustained, rating it slightly higher than the previous year (2012 mean = 8.1, 2013 mean = 8.3). Similarly, they also felt that Secondary Talk had had a sustained impact on teaching practice across most of the school (2012 mean = 7.3, 2013 mean = 7.6). In all but one school, these ratings have either sustained or increased. What was clear in many of the interview responses was the move to explicitly teaching or prompting aspects of language and communication that would not have happened before – the difference between knowing about an approach and putting it into practice. 47 I think the staff being aware of how they can support students’ language development through explicitly trying to teach vocabulary, through explicitly trying to encourage students to use more academic type language is something that is increasing in terms of staff awareness of how important a role that has to play in children’s learning and their understanding as well as what they hear and what they read. Teacher in charge of speech and language resource, Preston Manor School This also came through in observation and questionnaire data with teachers. Aspects showing most change were those associated with labelling or praising good communication, making clear links between language, communication and learning, pointing out which communication related behaviours are needed for a task. In the third year of the evaluation, senior leaders were asked whether the impact had been sustained, increased or decreased. In the interviews with seven senior leaders all seven could identify some impact on teaching practice or teacher knowledge. During the interviews participants were asked to rate in the impact of Secondary Talk on whether teaching knowledge and teaching practice had increased, decreased or stayed the same since the previous year. 80% (n=4) felt it had stayed the same or increased. Nearly all of the schools could identify examples of continued changes to teaching practice as a result of taking part in Secondary Talk, for example teaching tier two vocabulary, modelling good practice, rephrasing instructions and building on techniques such as pause time. “I was going to say one other thing, the thing I am conscious of I think is the spreading out amongst staff is understanding of the importance of vocabulary, tier 2 vocabulary and also the language for thinking a lot a lot of work has gone in to helping students to understand what words like analyse, evaluate, describe, explain staff are much more aware that students can fall down in exams because they haven't understood the question” Teacher in charge of the speech and language base, Preston Manor 48 “I think it's still there that staff are modelling that good practice” Intervention and literacy co‐ordinator, Fazakerley “Rephrasing the way that you use talk to make instructions simpler with less contortion and just being aware of the way that the teacher uses language.” Head of English and senior teacher, St Anne’s It is encouraging that schools are also revisiting key strategies and building them into future plans: “I had a teacher asking the other day "I really need the talk targets in my year 8 class because they would still really benefit from having them" so I'm wondering whether just doing something in Year 7 would be enough for instance or whether you need to think how would you then follow that up in terms of what the students are asked to think about or what teachers require of them in the next year up.” Teacher in charge of the speech and language base, Preston Manor In summary, we have comprehensive data from the first two years following Secondary Talk which shows that changes to staff knowledge and practice are sustained for at least a year after the programme starts. Senior leaders report a high impact on staff knowledge and practice, and both pupils and teachers identify practice changes. In Secondary Talk lessons, teachers are clearer about what they can do to support pupils’ speech, language and communication than teachers in non‐Secondary Talk lessons. Teachers in Secondary Talk lessons use strategies and approaches much more explicitly, they evaluate how effective they are, and encourage more active pupil involvement. We have less data from the third year of the evaluation, but four out of five senior leaders continue to report an impact on knowledge and practice. They are able to identify approaches and strategies that are well‐embedded in school, but which originate from Secondary Talk. School systems change The senior leaders were also asked to reflect upon changes to whole school strategy since taking part in Secondary Talk. After two years, they talked about Secondary Talk as a catalyst to embedding a focus on communication at school policy level but it was difficult for some to isolate the programme from the way they differentiate and from what is considered good teaching practice in school. Interviewees were able to recognise the approaches they used to support communication, but some did not label them as ‘Secondary Talk’. ‘I think the techniques have been incorporated. The wait time and things part of the whole ‘no hands up’. So I think for pupil's learning if anything even though it's not called ‘Secondary Talk’’ Assistant Principal, Uckfield Community Technical College Many language‐focused activities and systems were initiated as a result of Secondary Talk, notably: language and communication in leadership observations, in discussion at staff meetings and communication built into 49 ways of supporting behaviour. This analysis reflects how a focus on language and communication is increasingly embedded at policy as well as practice level. There were examples across the schools interviewed of whole school initiatives which focused on language and communication ‘.....for example, word of the week. I'll give you an example last week we've got all this flashed up on screens across the whole school on Monday the constant subliminal reminders there actually had to recap "practice" is it a noun or a verb.’ SENCO and student support manager, Fazakerly High School Really encouraging was the fact that students noticed changes in whole school systems. ‘The objectives themselves have changed slightly. They are usually now do include being able to discuss a particular topic in things such as geography. Most subjects really. We’ve also noticed that we’re given more time to freely discuss things in the teacher circulates the tables and tried to take out ideas into account’ In the third year, senior leaders found it even harder to isolate the influence of Secondary Talk. As part of interviews, senior leaders were asked to complete a grid to say which school systems had been in place before Secondary Talk and which were implemented as a result of Secondary Talk. The chart below shows which communication supportive strategies six of the schools identified that they are currently using (one school interviewed did not complete this information). Speech, language and communcation practices employed by Secondary Talk schools 2014
6
1
1
5
1
2
1
1
2
2
4
4
4
5
3
Number of schools
6
5
5
2
4
6
6
6
6
6
6
6
5
5
5
4
4
1
2
2
1
0
No
Yes
50 After three years, speech, language and communication is a feature of many policies and practices across schools. In all six schools, speech, language and communication is in the school development plan – an identified driver for change. Interviewees could think of several examples such as a communication focus in the school development plan and in leadership observations, changes to pupil tracking, to staffing models (with some schools employing specialist staff) and changes to training. “Yes it's on our school development plan so it's something every department is involved in that needs to be followed up by the SLT (senior leadership team) line management of course and some of our CPD is going to be on that as well.” Head of English and senior teacher, St Annes “Communication still a priority in the school development plan; we've increased the time our specialist teacher is employed” Assistant Head with responsibility for learning support, Peacehaven “We probably do lesson observations and pupil tracking with a speech, language and communication angle” Vice Principal for Inclusion, Park Hall Academy “It’s there on the school improvement plan, school evaluation, departmental evaluation, it's timetabled in to the academic year” Intervention and literacy co‐ordinator, Fazakerley However, many senior leaders felt that these aspects of policy and practice had always been in place and found it difficult to identify Secondary Talk as the catalyst. As we have similar grids and interviews from year one and two of the evaluation, it is possible to track the development and see the extent of the programme impact (appendix 2). The fact that many interviewees in the third year see this as ‘custom and practice’ in school reflects how embedded Secondary Talk principles have become. In summary, in all of the schools we followed for three years, Secondary Talk principles and practice have become embedded at a strategic level. So much so that it is difficult for senior leaders to identify a time when this has not been so ‐ despite evidence showing that Secondary Talk has been the catalyst for change. 51 5. Discussion How sure can we be that Secondary Talk has a significant impact on students’ learning and their language and communication? The significant changes to aspects of students’ language were seen only in the group of students where Secondary Talk had been implemented, not in the non‐Secondary Talk comparison school. While this is a pleasing result, the relationship is not straightforward; it is compounded by a range of factors. The evaluation design was not controlled, so the two schools who were compared were not matched demographically, with very different pupil populations. Schools themselves selected students in each of the three ‘language‐type’ groups and while they were given guidance this was not based on stringent criteria. There was also no control for other interventions the schools may have been involved in during the evaluation period. We were also very aware that this evaluation happened in a context of change: the effective ‘demoting’ of speaking and listening through changes to the curriculum and to the assessment system, rising floor standards for schools, an evolving Ofsted evaluation framework. These external pressures can very easily alter priorities for schools. However, we were keen to keep this a functional, real life evaluation. This is what the education system and schools are like, and so these methodological issues were expected. To mitigate these, we looked at the progress students made rather than absolute scores and also collected a range of different measures from pupils to triangulate findings. This gave a picture of students who made more progress in language and learning, both in terms of changes to scores or levels, but also in the strategies they used to communicate and learn. The fact that measures of cognition did not change in either groups helped to support the specific changes to language and learning. Our focused assessment on students was with a relatively small group (n=19), but our wider data collection from different groups of schools corroborated the impacts we found in this group. Senior leaders reported impacts on language and communication, on learning and on behaviour. Collecting this wide range of quantitative and qualitative data means that we can confidently report that students make statistically significant gains in their language and communication following Secondary Talk. This impacts on their learning in the classroom, and contributes to progress in curriculum subjects. Evaluating whole school programmes – what other issues? In addition to the methodological issues highlighted above, we have built on our understanding of evaluation design for whole school programmes for secondary aged young people. The study has confirmed and further emphasised the importance of triangulating data sources. We comment above that this adds rigour, but also due to the nature of secondary schools it can be important to have more than one data source. For example: a) in the second year our lesson observations took place in the summer term, when some lessons consisted of watching a DVD rendering the session useless. b) while it is good to have access to the National Pupil Database, the data sets are not complete and are also reliant on teacher assessment. In both of these instances, corroborating findings from a range of sources has been important. With a programme that is designed to grow in schools, and influence staff and pupils across the school, there also come issues in evaluation design. The original design has become challenged and no longer fit for purpose. We relied on an in‐school comparison group, and it has been difficult to find ‘non Secondary Talk’ lessons to act as this comparison. With secondary schools being large, dynamic organisations, the changes to 52 staff – and in particular key contact people – has meant a challenge to achieving consistent, replicable observations and interviews. We have suggested changes as a result of this. Again, our range of evidence sources has mitigated well against this. How long does Secondary Talk continue to make an impact? This evaluation shows that in the year after Secondary Talk starts, pupils make significant progress, but the rate of this change slows after a year. This is also reported by senior leaders in school who continue to report an impact two and three years after the programme started, but are less willing to put a numerical value on this after two years. There are a number of possible reasons for this. Staff in the intervention school felt the slowing of assessed progress could in part be due to increased language demands of Key Stage 4 compared to Key Stage 3 – a different style of learning, increased teacher talk, more independent or peer study. This came through in interviews with senior leaders as well who cited the heavy demands of GCSEs, and the fact that increasingly they taught to language needed for exams. It could also be that schools find it harder to isolate the impact of a programme which, by their own report, becomes embedded into the ‘business as usual’ of school. It is interesting to note that informal analysis of the language used by senior leaders in their Year 3 interviews shows them to be confidently using terminology such as “talk as a tool for learning” ( Head of English and Senior Teacher, St Annes) “Unmet communication needs” and “avoidance behaviours” (SENCO and Vice principal for inclusion, Park Hall Academy) and “shared, paired reading” (Assistant Principal for Achievement and Standards, South Wolverhampton and Bilston Academy) ‐ terms that they would not have used before the programme onset. They also find it hard to recall a time when language and communication was not in their school development plan, or built into planning. And yet our evidence from earlier interviews clearly shows the lack of these systems pre‐Secondary Talk. The nature of change in schools provides a potential barrier for effective evaluation if self report is the only measure used. This project has provided the opportunity to look at longer term impact of a whole school commitment to speech, language and communication. Many whole school initiatives are time limited and short lived. This evaluation is showing that there is a considerable initial impact on students and school systems. Without further input the pupil impact risks levelling off, although changes to school systems are sustained. This implies that ‘refresher’ activities, prompts and input may be needed to keep specific approaches and strategies alive in schools. What have we found out about language and communication in secondary schools? Investigating two very different schools over a period of three years has provided the opportunity to both chart development, and also to compare. Language levels in the two schools varied enormously. School staff were given the same guidance and although each school selected students with a spectrum of language abilities, the students in the disadvantaged area had lower language levels than their peers in the more affluent area. This reflects what is known about language and deprivation, and also what has been reported about difficulties identifying language levels in schools. This year, speaking and listening was taken out of contributing towards English GCSE qualification on the basis that they are difficult to accurately assess and benchmark. This evaluation supports this; staff find it hard to distil what is meant by ‘typical language’ and ‘poor language’. Discussion with the SENCO in the school situated in a more disadvantaged area 53 revealed that some of their students identified by the school as having poor language would be judged at having SEN in a neighbouring school. There are implications for secondary school staff knowing more about differentiating language levels. Secondary Talk works best for students with poor language making it even more imperative that schools are able to identify these young people. Aspects of language continue to develop in adolescence Analysis of the rich data we have gathered has only just started. This will continue as part of post graduate study. Already it is possible to see that as well as language competence in areas such as narrative and vocabulary, students also develop in the way they use language to support their learning. We have discovered more about how students are able to reflect on their own performance – and how they need good language skills in order to be able to do this. Analysis of the learning strategies used by our pupils has also shown that these become more complex and sophisticated. Language and approach to learning The levels of engagement, shown by an engagement questionnaire, were higher in the comparison school than the intervention school. As well as very different demographics (deprivation index, numbers of children with SEN, rural/urban) the schools had very different cultures. Descriptions on page 42 illustrate how very different the two school environments are. In the comparison school, high value was placed on creating a nurturing environment ‘We believe that school is a place where students should feel safe and happy and should therefore develop their potential to the full’ (school website). Despite having no school uniform, the students are proud of their school and have a strong sense of ownership. The intervention school places value on maximising potential but less so on the developing the strong community spirit. Interviews with senior leaders in each school confirm this. Our analysis of student responses to school engagement, and of how confident they are in different situations shows that these are not stable concepts for any young person in adolescence. This suggests that there is room and flexibility for change. How teachers support language and communication Our observations in schools over the first two years of the evaluation have enabled more in‐depth analysis of effective approaches. Teachers’ use of skilful questioning, scaffolding learning through talk and giving students time to process information are all features of high quality teaching, valued by Ofsted. What Secondary Talk is bringing, is additional to this, enhancing the quality of interaction in lessons. Teachers in Secondary Talk lessons don’t just use supportive approaches, they explicitly teach skills, explicitly praise students’ use of language and will overtly measure the impact of this, feeding back their reflections to students so that students are more involved. Analysis of word learning strategies used by students shows that although students in the intervention school use more strategies, they need additional support to use the strategies to successfully learn words. These are all aspects which can be formed into practical strategies helping us to explain what we mean by ‘enhancing the communication environment of lessons’. What are effective ways of measuring language in secondary aged students? Seeing progress on standardised measures of language is pleasing, but these give up snap shots of performance rather than a full ‘video’ of how students communicate. They have been criticised as limited in showing progress in adolescents. The range of tests and tasks used in this evaluation has provided rich data 54 not only about language levels, but also about the strategies young people use, and how they are able to reflect on their performance – meta‐linguistic skills. Students using self‐rating scales The evaluation shows that students with poor language find it hard to accurately reflect on their performance. So is self‐report a useful tool to use? The experience in this evaluation is that there are different ways of gathering information from students, some more effective than others. Within this study, self rating using a 5 point scale was used, but also a sorting activity whereby students placed cards depicting communication situations into three different categories: easy, OK, difficult. This latter activity yielded more interaction and a more considered approach than merely asking students to rate performance. Developing these skills in young people, however, will enable them to more accurately be involved in giving feedback and contributing to decisions made about them – something central to current SEN practice. The assessor in this evaluation was an experienced, skilled speech and language therapist, which may also have been a contributory factor. Distilling what exactly it is that she does to support student feedback would form useful guidance for others. Bespoke tasks Several tasks were designed specifically for this evaluation, notably one which investigates students’ word learning skills. There is not enough time to teach students every single word they need to know in secondary school, they depend on word learning skills. Seeing progress in students’ strategies for word learning can be just as meaningful as an absolute vocabulary score. Carrying out repetitions of the same tasks over a number of years has given plenty of information so that these can be refined into accurate, functional school‐based tasks which will help staff to demonstrate impact in practical ways. 6. Conclusions Implementing a whole school language programme can have a significant impact on students’ language, with resultant impacts on their learning in secondary school. Students with poor language make most progress, narrowing the gap between them and their peers. Significant impacts are seen one year after the programme starts, with progress sustained after this but levelling off. These impacts are accompanied by specific, structured changes to teaching practice where strategies are introduced, students are taught how to benefit from them and progress is monitored. As well as this, school systems change so that a whole school focus on language and communication is identified as a priority, and embedded as ‘business as usual’. A qualitative difference is seen in the language levels of pupils, and in teaching practice in a disadvantaged area compared with a school in a more affluent area. Although both student performance and practice change in the disadvantaged school, they do not catch up with their peers or colleagues in a more affluent school. Change characterises secondary schools – at practice and policy levels. In order to sustain development for protracted periods, schools may need regular ‘refreshers’ or ‘prompts’. 55 There appear to be associations between language level, engagement, school demographics, culture and students approach to learning which would benefit from further investigation. There are indications that engagement in learning is not a stable concept during adolescence. The Government’s ‘character education’ agenda will be helpful to see whether these traits can be changed. Evaluating whole school language initiatives can be methodologically challenging and this project presents some useful solutions to these challenges. Evaluating a programme that grows in schools which themselves also change, means that it is important to include a range of measures. Standardised assessments are useful when accompanied by more tailored tasks and qualitative interview data. The use of self‐report may be challenging for students who have poor language. A skilled assessor, used to working with adolescents with language difficulties helps to gather accurate and comprehensive data. Mary Hartshorne, I CAN Director of Outcomes and Information Rachael Black, I CAN Monitoring and Evaluation Co‐ordinator In collaboration with Dr Judy Clegg, The University of Sheffield May 2015
56 Appendix 1. Demographic information Secondary Talk Evaluation 2015: Type of School
4
1
3
17
Community School
Academy
Special School
Voluntary Aided school
Foundation School
27
Chart 1: Showing the types of settings that have taken part in Secondary Talk since 2011 Secondary Talk Evaluation 2015: Size of School
19
22
Above Average Average
Below Average
11
Chart 2: Showing the size of settings that have taken part in Secondary Talk since 2011 57 Secondary Talk Evaluation 2015: Percentage of pupils in receipt of Free school meals (FSM) 12
Above Average Average
Below Average
25
8
Chart 3: Showing the percentage of pupils in receipt of Free School Meals Secondary Talk Evaluation 2015: Percentage of pupils who speak English as an additional language (EAL) Above Average 19
Average
Below Average
26
1
Chart 4: Showing the percentage of pupils who speak English as an Additional Language 58 Secondary Talk Evaluation 2015: Percentage of pupils with Special Education Needs (SEN) or disabilities
11
Above Average Average
Below Average
25
8
Chart 5: Showing the percentage of pupils with Special Education Needs Secondary Talk Evaluation 2015: Schools with specialist provisions
13
Yes
No
36
Chart 6: Showing the number of settings with specialist provisions 59 Appendix 2: Changes to school systems 2012 2012: School Systems 8
Number of schools
7
6
5
4
3
2
1
0
Not in place
In place before Secondary Talk
Implemented as a result of Secondary Talk
2013 compared with 2012 School Systems Introduced as a Result of Secondary Talk 100
90
80
70
60
% of schools
50
40
30
2012
2013
20
10
0
60 Appendix 3: Correlation between language levels and self‐report of confidence Intervention group at baseline. Using Spearmans rho correlation coefficient, a strong negative correlation between language skills and confidence in communicating was found r= ‐.68, n=9, p= 0.43 Intervention group after two years. Using Spearmans rho correlation coefficient, a medium negative correlation between language skills and confidence in communicating was found r=‐.55, n=9, p=0.12 61 Comparison group at baseline. Using Spearmans rho correlation coefficient, a strong positive correlation between language skills and confidence in communicating was found r=.62, n=10, p= 0.05 62 Appendix 4: Data Collection schedule and evaluation activities. Completed activities are shaded: proposed activities are shaded: September 2011 Cohort 1 ( x schools) Sample for whole school data Cohort 2 (x schools) Sample for classroom observations July 2012 September ‐ December 2012 Interviews Secondary Talk project taking place with senior staff. Whole school data collection Secondary Talk project taking place Data analysis Reporting January – May 2012 B/line Lesson observations, focus grps and questionnaires January – May 2013 Lesson observations, pupil focus groups and questionnaire Informal, descriptive analysis March 2013 interim report September – December 2013 Interviews with senior staff. Whole school data collection January – May 2014 Lesson observations, pupil focus groups and questionnaire Descriptive analysis March 2014 interim report September – December 2014 Interviews with senior staff. Whole school data collection January – May 2015 March 2016 Descriptive and inferential analysis March 2015 Final full evaluation evaluation report report Monitoring data of reach and spread collected throughout Focused comparison project in 2 schools Intervention school Comparison school October 2012 Baseline data collected January 2013 March 2013 December 2013 Secondary Talk in progress Follow up data Delay in recruiting comparison school Baseline data December 2014 Follow up data December 2015 Follow up data Follow up data Follow up data March 2016 Final evaluation report (delayed til April 2016) 63 Appendix 5 Facilitators and barriers in implementing Secondary Talk Facilitators Barriers Active endorsement of Head teacher and wider Leadership team for implementation of Secondary Talk. An absence of facilitating factors. School’s self evaluation process has highlighted the need for support in SLC skills and the ST programme is recorded as addressing this need within the school development plan. Enthusiasm of school coordinator for supporting students’ SLC/N substitutes for thorough planning of how Secondary Talk can be most effective. ST training explains the link between SLC skills and literacy, behaviour, social disadvantage, and achievement. Lack of resources (e.g. time for school coordinators to coax and coach colleagues to use the materials etc). The school introduces an ST approach at the outset which is likely to lead to ‘quick wins’ for teachers – benefits of using the approach are quickly evident. First ST approaches to be introduced are challenging for teachers to adopt. The school is clear at the outset about how the programme is structured and milestones occurring during the ST year. Provision of ST training or other inputs is seen by coordinator / school as ends in themselves. The school coordinator has accessible versions of selected approaches to communicate expectations succinctly to colleagues. The school coordinator of ST is the SENCo, leading to the programme being seen as ‘only for SEN students’. School staff are clear about the importance of taking baseline measures prior to introducing strategies, so that progress can be judged reliably. Staff are not adequately supported to translate training about how to support students’ SLC/N into classroom practice. The school uses pre‐existing communities of interest (e.g. Literacy group, Inclusion champions) to support the introduction of ST. Changes to the curriculum which de‐emphasize the importance of Speaking and Listening skills. Aims of ST are communicated to students and their opinions are sought and used to develop good practice. Staff turnover – an enthusiastic advocate of using ST approaches leaves the school. The school nurtures a classroom culture in which an acknowledgement of language difficulties is seen as a normal part of learning and not something to evoke embarrassment in pupils asking for help. A poor Ofsted report leads to the school focusing on ‘quick fixes’ to address highlighted weaknesses. Generally this detracts from efforts to develop students’ language skills (seen as a longer term aim). The expertise of specialist staff (i.e. specialist language teachers or HLTAs) is used to reinforce good practice and demonstrate its benefits to others. Attempting too much change within a short period leading to staff resistance. Over time, the importance of spoken language becomes Difficulty arising from competing demands from a range of initiatives within school – particularly relating to central to the school’s culture and intrinsic to professional dialogue, even if particular ST approaches scheduling training at whole school level. are no longer in use. 64 Appendix 6 ‐ I CAN Secondary Talk HT/senior leader/ST co‐ordinator interviews ‐ September 2014 Interviewer: First of all thank you very much for taking part in this interview. We appreciated that it is now 3 years since Secondary Talk first started in your school but what we’d really like to find out what, if anything, has been the lasting impact of the approach. Feedback comments from [name of school]: Headteacher/senior leader: Co‐ordinator: ST consultant: Date: Interviewed by: Permission given to use comments / quotes given: YES / NO BACKGROUND INFORMATION What is your current role? (if ST co‐ordinator) Why did your school decide to commission ST? 1. GROWING ST IN SCHOOL Interviewer: So going back to when you first started ST‐ 1.1 How was ST started in the school‐did you start with particular year group/subject? 1.2 What about this last year? Has ST continued to grow and develop in the school? What does it now look like? What supported this to happen? (E.g. sharing good practice in team meetings, putting it on the website, having whole school training courses)? [Fazakerley, Cleeve used as case studies last year] 1.3 Do you feel the principles of ST (Reminder: ST aims to develop secondary schools so they are more supportive of students’ language and communication, principles on reverse) are still operating in the school? If yes, please give examples What supported this happening? (refer to 1.2, anything different?) If no, what has stopped this from happening? 2. IMPACT OF ST Interviewer: We are really keen to find out what the long term impact of ST is, so this year we are looking at the continuing impact – whether the impact has been sustained, has increased or decreased compared to last year. Over the next 6 questions I am going to remind you of the score you/x gave last year on a scale of 1 to 10 (where 1=no change and 10 = substantial impact) about the impact of various elements of the programme. We are trying to find out what, if any, impact ST continues to have in your school. So you need to think – has that score gone up, stayed the same or decreased. What impact, if any, does the ST program continue to have in your school… 2.1 On staff knowledge and understanding about SLC/N? 1 = no impact at all and 10 = substantial change to their knowledge and understanding 1 2 3 4 5 6 7 8 9 10 Guidance: if made a big impact for some staff but not all, then 7/8 – if impact for all then 8/9/10 Please give examples of the type of impact you have observed or that staff/pupils have reported (please note specific examples to follow up later) What differences have you noticed? 2.2 On teaching practice? 1 = no impact at all and 10 = changed practice substantially 1 2 3 4 5 6 7 8 9 10 Guidance: if made a big impact for some staff but not all, then 7/8 – if impact for all then 8/9/10 Please give examples of the type of impact you have observed or that staff/pupils have reported (please note specific examples to follow up later) What differences have you noticed? Prompt: quality of teaching, differentiation 66 2.3 On pupils’ learning? We predicted that the impact on pupils’ learning, behavior, language skills would take a while, so we are interested in gathering your thoughts 3 years down the line. Same as before, last year you/x said [score] – is this any different? We know it won’t be the only factor, but do you feel that ST/focus on communication has contributed to the impact. 1 = no impact and 10 = substantial impact. 1 2 3 4 5 6 7 8 9 10 Prompt engagement, behaviour for learning. Please give examples of the type of impact you have observed or that staff/pupils have reported (please note specific examples to follow up later) What differences have you noticed? Prompt: helped them to access curriculum, understand in lessons? 2.4 On pupils’ behaviour? 1 = no impact and 10 = substantial impact. 1 2 3 4 5 6 7 8 9 10 Please give examples of the type of impact you have observed or that staff/pupils have reported (please note specific examples to follow up later) What differences have you noticed? 2.5 Pupils’ language and communication? 1 = no impact and 10 = substantial impact. 1 2 3 4 5 6 7 8 9 10 Please give examples of the type of impact you have observed or that staff/pupils have reported (please note specific examples to follow up later) What differences have you noticed? Comments about students with poor language? Comments about pupils with SLCN? 67 2.6 On School systems? 2.6.1
Interviewer: What changes can you see in the way the school works to support students’ language and communication? This could be curriculum, teaching and learning, specific approaches…. At this stage refer to appendix 1. Which of these are in place now, that weren’t before? 2.6.2 Interviewer: What impact has ST had on the strategic direction of the school (for example language and communication in the school development plan, other literacy/language initiatives)? 1 = no impact and 10 = substantial impact. 1 2 3 4 5 6 7 8 9 10 Please give examples of the type of impact you have observed or that staff/pupils have reported (please note specific examples to follow up later) 3
OVERALL SATISFACTION WITH SECONDARY TALK 3.1 Overall, how satisfied are you with the impact of ST in your school 23years after you first started it? 1
1 = not at all satisfied and 10 = totally satisfied 2 3 4 5 6 7 8 9 10 3.2 Do you think that ST has been worth the effort and money that your school put into it? 1 = not at all and 10 = excellent value for money 1 2 3 4 5 6 7 8 9 10 8 9 10 4
SUSTAINABILITY 4.1 Do you think the ST approaches have been enduring in your school? 1
1 = not at all, 10 = yes, totally 2 3 4 5 6 7 4.2 Do you think they will continue to endure over the next 12 months? 4.2.1
If yes, how will you support this to happen? 4.2.2
If no, why not? 68 Appendix ‐ Which of these were in place before ST? Please asterix* Which have happened since starting ST? Please add ST in box. Please add ST* if it happened this year. Provision maps mentioning language Language and communication in and communication leadership observations Resources to support students’ communication Language and communication in school development plan Systems for identifying students with Ways of monitoring progress in SLCN students language and communication Activities to support students’ communication in class Whole school language initiatives e.g. word of the week, vocabulary competition etc Lesson plans with a focus on language and communication Training on language and communication built into INSET programmes Displays focusing on students’ communication Reporting to parents about students’ communication SLC on agenda at Governor’s meetings Language and communication discussed in staff meetings Other initiatives that focus on language and communication A policy for supporting communication A focus on language and communication in homework Communication friendly ways of supporting students’ behavior A member of staff with responsibility for coordinating support for students’ language and communication development 69 Appendix 7 word learning task Non‐standardised word learning task I am interested in how you go about learning new words that you hear in lessons. I’m going to ask you to do 2 very short tasks. 1. I’m going to show you a long, technical word that you don’t know – you won’t be expected to know the word already as it is very unusual – even to many adults. PRESTIGITATION How would try to work out what the word meant. Pupils will be scored on the number of useful strategies they talk about or use. Break up the word Look for words inside the word Say it aloud Ask to hear it again Write it down Look at the beginning and end Use term ‘prefix’, ‘suffix’ Ask a peer Ask an adult Put it in a sentence Ask to hear it in a sentence Look for clues in the sentence Think of something that sounds a bit the same Look it up in a dictionary Think what type of word it is – noun, verb etc 70 2. Now I’m going to give you a simple, familiar word to talk about. FOOT Tell me everything you can about the word. Pupils will be scored on the number of different aspects relating to sound structure (phonology) and meaning (semantics). Number of syllables Initial sound Final sound Rhymes with Long/short word Sounds within Sounds like.... Describe the ‘sound’ of the word e.g. ‘a sharp sounding word’ Category: a kind of..... Physical attribute Function: use it for...... Location: find it.... Another word for it is..... Put in sentence Something that goes with it ‘foot and hand/sock’ Grammatical category: noun, verb, adjective Person association: a x would use it... 71