Evaluation of Excellence in Education through Business Links (EEBL) Final report to Skills Development Scotland 25 February 2009 Evaluation of Excellence in Education through Business Links (EEBL) Final report to Skills Development Scotland Contents Executive Summary ................................................................................................................. 1 1: Introduction .......................................................................................................................... 5 2: Programme overview........................................................................................................... 7 3: Planning and preparing placements ................................................................................ 12 4: Placement experience ....................................................................................................... 19 5: Benefits and impact of EEBL ............................................................................................ 26 6: Challenges .......................................................................................................................... 39 7: Conclusions and recommendations ................................................................................ 44 Annex A: Participation by local authority area ................................................................. A-1 Annex B: List of consultees ................................................................................................ B-1 Annex C: Questionnaire used in on-line survey of teachers........................................... C-1 Contact: Sheila Sim Tel: Approved by: Graham Thom Date: 0131 225 4007 email: ssim@sqw.co.uk 25 February 2009 Associate Director www.sqw.co.uk Evaluation of Excellence in Education through Business Links (EEBL) Final report to Skills Development Scotland Executive Summary Evaluation aims 1. The purpose of this short evaluation was to review the EEBL programme in preparation for the handover of responsibility from Careers Scotland. Its aim was to assess the impact of the programme on the teachers concerned, as well as the effectiveness of processes and relationships with host organisations and local authority partners. Careers Scotland was also interested to identify the effect of an EEBL placement on the pupils concerned. 2. The evaluators were tasked with covering the following points: How does EEBL affect the individual perception of education / basic philosophy by teachers? How does EEBL develop partnerships, and what do these partnerships look like? How do employers / host organisations benefit from EEBL (e.g. staff development)? What works, what doesn’t work, and why? What triggers interest in a teacher participating in EEBL? How much influence on participation comes from the individual and how much is prompted by the local authority? How effective are individual placements versus national / regional themed placements? How does EEBL impact in the classroom, both in terms of attainment and achievement? How does EEBL deliver a Curriculum for Excellence? Recommendations for enhancing the effectiveness of EEBL. Methodology 3. Our research methodology has comprised several strands: desk based review of Careers Scotland monitoring data and internal reports (including teacher satisfaction surveys, teacher impact assessment forms, etc) telephone interviews with EEBL co-ordinators and their local authority counterparts across the five Careers Scotland regions five paired interviews with individual teachers and their corresponding host organisations (a total of ten interviews), plus an additional five interviews with teachers 1 Evaluation of Excellence in Education through Business Links (EEBL) Final report to Skills Development Scotland three case study visits an electronic survey of 894 teachers who have participated in EEBL to date, selected on the basis of their email address being available. The survey generated a response rate of just over 23%, which has allowed us to produce a robust set of findings. Main findings 4. The vast majority of teachers and host organisations rate their EEBL experience very highly. Amongst host organisations, 81% report a ‘good’ or ‘excellent’ experience; and teachers report even higher satisfaction levels. The outcomes that teachers expect from their placement are generally being achieved: over two thirds of teachers report that their desired outcomes are met in full. 5. Take-up of EEBL across local authority areas has varied, and in some areas the programme has been adversely affected by the lack of funding for supply cover. 6. Effective planning and preparing before a placement is crucial to success, and this process can be very time-consuming. The most effective placements are those where aims and objectives have been clearly defined at the outset by both teachers and host organisations. The brokerage role played by Careers Scotland has been important in helping participants to articulate their aims. 7. The majority of teachers and host organisations report that they intend to maintain their links after the placement has ended. The willingness of employers to continue, and even broaden, their links with teachers offers great potential for local authorities to cross-promote other initiatives, particularly regarding work experience for pupils. 8. The best EEBL placements are those where teachers are involved in active learning and work shadowing, rather than attending presentations (i.e. ‘doing’ is better than ‘listening’). Themed and national placements have no less impact than individually tailored placements. Longer placements have greater impact on teachers and pupils than shorter placements. Teachers feel that EEBL is a very valuable contribution to continuing professional development, and many suggest that every teacher should be encouraged to undertake a placement. 9. The impact of EEBL on pupils is hard to quantify, but it is clear that the programme has helped teachers to deliver the four capacities of Curriculum for Excellence. 10. Although the vast majority of teachers report that they have disseminated the learning and experience from their EEBL placement, this is not done consistently, and many teachers are unaware that placement opportunities exist. Better awareness raising and dissemination are required. Recommendations 11. Here we present a number of key learning points from the evaluation, which should be of use to local authorities if they take on responsibility for delivering EEBL. 2 Evaluation of Excellence in Education through Business Links (EEBL) Final report to Skills Development Scotland Awareness of EEBL is generally low amongst teachers. This is partly due to ‘gatekeeping’ issues amongst local authorities and individual headteachers: some local authority officers do not encourage Careers Scotland to market EEBL to schools, and some headteachers do not promote EEBL within school. On the other hand, dissemination by teachers who have taken part in EEBL is high: the vast majority have recommended EEBL to their peers and their headteachers. The transfer of responsibility for EEBL to local authorities would remove a layer (Careers Scotland) from the communication hierarchy, thus in principle making it easier to market the programme. Recommendation: EEBL should be marketed effectively to schools, so that all teachers know that placements are available as part of the CPD offer. The pre-placement stage of planning and preparation is crucial to the success of EEBL. In particular, it is vital that teachers and their host organisations are both able to identify and articulate their respective aims for the placement. In order to achieve this, face to face meetings prior to the placement should take place if possible; when this does not happen, participants may have a less successful experience during their placement. Recommendation: sufficient resource should be given to preplacement planning and preparation. Themed and national placements are just as valuable as individual placements. This is potentially good news to local authorities if they take over responsibility for EEBL, as such placements are less resource-intensive to organise. However, it has other implications: in particular, the ending of Careers Scotland involvement means that national placements will no longer continue to be organised, unless whoever takes over responsibility for EEBL is able to co-ordinate such placements through a national network. Likewise, themed placements that have been operating across local authority boundaries (e.g. BBC Learning) may also be lost, or run the risk of duplication of contact as individual local authorities pursue their own employer leads. In this environment, if responsibility for EEBL is handed to local authorities it would make sense for groups of local authorities to employ a joint co-ordinator to work across boundaries. Recommendation: resource should continue to be given to coordinating placements across local authority boundaries, if necessary by sharing resources to employ joint co-ordinators. The points above indicate a need for significant management and co-ordination resource if EEBL is to operate effectively and at the same scale as in previous years. It is important that whoever takes over responsibility for EEBL recognises the role played by Careers Scotland staff and identify how this gap will be filled if responsibilities are transferred. Recommendation: the new providers should learn as much as possible about the role of Careers Scotland in managing the programme up till now, and make arrangements to resource a version of that role within the new delivery structures. Placements should be designed to be as hands-on as possible. Teachers benefit most where they are able to (sometimes literally) get their hands dirty. Work shadowing is much more useful than listening to presentations, and has more impact. Agreeing 3 Evaluation of Excellence in Education through Business Links (EEBL) Final report to Skills Development Scotland exactly what will happen during a placement is part of the planning process. Recommendation: the new providers should encourage host organisations to make sure that placements allow as much active learning / work shadowing as possible. Placements should have equal benefits to both parties. We are aware of a few instances where host organisations use teachers to produce materials that benefit the organisation, and where the benefit to teachers is not as clear-cut. Again, the benefits to both sides should be clearly articulated at the pre-placement stage. Recommendation: the new providers should liaise closely with teachers and host organisations to make sure that placements are mutually beneficial, with aims and objectives being clearly defined on both sides during the pre-placement phase. Longer placements (5 days) have more impact on teachers and pupils than short placements (2 days). This has obvious resource implications, and it is likely that a balance will need to be struck. Recommendation: where possible, and providing clear aims and objectives have been set by both parties, resourcing of 4- and 5day placements should continue alongside shorter placements. Some stakeholders assume that block placements (taking place over consecutive days) are more effective than non-consecutive placements (whereby a teacher attends for half-days over a period of time). This is not necessarily the case: there is some evidence that teachers and hosts benefit from non-consecutive placements, which can give more time for reflection. Recommendation: the benefits of non-consecutive placements should be investigated in more detail, with a view to promoting them if it becomes clear that they bring benefits. EEBL contributes directly to all four elements of Curriculum for Excellence and also to a wider range of outcomes. It has significant potential to develop and consolidate the links between schools and employers. Recommendation: EEBL should be regarded as integral to, not separate from, school-employer links; in particular, it has an important role in generating work experience opportunities for pupils. Dissemination of learning from EEBL has not been consistent, and could be improved. Recommendation: all teachers benefiting from EEBL should be expected to disseminate key learning points to their colleagues at school level, and Glow should be used to promote EEBL internally and externally. 4 Evaluation of Excellence in Education through Business Links (EEBL) Final report to Skills Development Scotland 1: Introduction Background and context 1.1 Various forms of teacher placements have been a part of the Scottish educational landscape for many years, as a professional and personal development opportunity for teachers. Excellence in Education through Business Links (EEBL) has been an integral element of the Scottish Government’s strategy for enterprise in education Determined to Succeed (DtS), 2003, which recommended that ‘teachers, as part of their CPD, must have the opportunity to participate in training and development in enterprise in education, including enterprise experiences, at least once every two years’. EEBL offers teachers an experience of the world of work through placements into businesses and other public and voluntary organisations. 1.2 EEBL placements are available to all primary and secondary teachers who work across all areas of the curriculum in local authority mainstream and special schools. Placements last for a minimum of two days, with no upper limit – although in practice five days has generally been the maximum, as this is the upper limit for local authorities to access funding for supply cover. Placements can be tailored to suit an individual teacher, or designed at regional or national level to focus on a specific theme or business sector and offered to teachers with a shared interest. 1.3 Careers Scotland (now part of Skills Development Scotland) has managed and delivered the programme for the last three years through its network of local co-ordinators on behalf of the Scottish Government. This is now changing, however, as responsibility for EEBL is being discussed with COSLA, with the intention being to hand it over to the 32 individual local authorities. A transition period is currently in operation, with the full, formal handover due to take place by 31 August 2009. Evaluation aims 1.4 The purpose of this short evaluation was to review the EEBL programme in preparation for the handover of responsibility. Its aim was to assess the impact of the programme on the teachers concerned, as well as the effectiveness of processes and relationships with host organisations and local authority partners. Careers Scotland was also interested to identify the effect of an EEBL placement on the pupils concerned. 1.5 The evaluators were tasked with covering the following points: How does EEBL affect the individual perception of education / basic philosophy by teachers? How does EEBL develop partnerships, and what do these partnerships look like? How do employers / host organisations benefit from EEBL (e.g. staff development)? What works, what doesn’t work, and why? 5 Evaluation of Excellence in Education through Business Links (EEBL) Final report to Skills Development Scotland 1.6 What triggers interest in a teacher participating in EEBL? How much influence on participation comes from the individual and how much is prompted by the local authority? How effective are individual placements versus national / regional themed placements? How does EEBL impact in the classroom, both in terms of attainment and achievement? How does EEBL deliver a Curriculum for Excellence? Recommendations for enhancing the effectiveness of EEBL. In Section 7 we attempt to address these questions, drawing on the evidence that we present in the next sections of the report, which in turn cover: an overview of the programme (Section 2) a discussion of pre-placement planning (Section 3) teachers’ and employers’ experience of EEBL placements (Section 4) benefits and impacts of EEBL (Section 5) challenges (Section 6). Methodology 1.7 Our research methodology has comprised several strands: desk based review of Careers Scotland monitoring data and internal reports (including teacher satisfaction surveys, teacher impact assessment forms, etc) telephone interviews with EEBL co-ordinators and their local authority counterparts across the five Careers Scotland regions five paired interviews with individual teachers and their corresponding host organisations (a total of ten interviews), plus an additional five interviews with teachers three case study visits an electronic survey of 894 teachers who have participated in EEBL to date, selected on the basis of their email address being available. The survey generated a response rate of just over 23%, which has allowed us to produce a robust set of findings. (The response rate was hampered by problems in Fife local authority, where recipients were unable to access the survey weblink due to an electronic firewall.) 6 Evaluation of Excellence in Education through Business Links (EEBL) Final report to Skills Development Scotland 2: Programme overview Key points In the two years and nine months covered by this evaluation (April 2006 to December 2008) a total of 1,432 teachers have participated in EEBL. Participation rates vary across regions and local authority areas, with take-up against target lowest in Edinburgh, Midlothian and South Lanarkshire. There has been a trend away from individual placements towards themed / national placements. The overall ratio is one third individual: two thirds ‘group’ placements. 2.1 In this section we provide a brief overview of EEBL, summarising the key features of take-up during the period April 2006 to December 2008. Participation 2.2 In total, over the period April 2006 to December 2008 (two years and nine months), 1,432 teachers have participated in the EEBL programme. In this time a quarter (358) of all participants have come from the West region, a fifth (283) of participants have come from the South East region, while the South West, North East and Highlands & Islands regions have provided a smaller amount of participants at approximately 18% each (this breakdown is broadly in line with population, with most schools being concentrated in the central belt of Scotland). 2.3 Careers Scotland sets a target of 100 placements per year in each of its five regions, and in turn each local authority is allocated its own target. Achievement against target is shown in Table 2-1. 2.4 The 2006-07 phase of the programme attracted the largest number of participants, with 42% of total participation in the programme happening during this time. (It should be noted that the different time periods illustrated in Figure 2-1 are not directly comparable, as the April to December 2008 period only covers nine months, unlike the other phases which cover twelve months.) 2.5 Participation rates across regions and council areas very much reflect population data, with larger urban areas having higher participation rates in EEBL. However, it is apparent that performance against target varies considerably across local authorities, with significant variations from year to year in some areas. Nineteen local authorities reached their target in each of the two years for which we have full data. Authorities where take-up appears low in relation to target include Edinburgh, Midlothian and South Lanarkshire. 7 Evaluation of Excellence in Education through Business Links (EEBL) Final report to Skills Development Scotland Figure 2-1 Total participation April 06-December 08 180 160 358 Total participants 140 283 278 120 258 255 South West Highlands & Islands 100 80 60 40 20 0 West South East North East Region Apr 06 - Mar 07 Apr 07 - Mar 08 Apr - Dec 08 Source: SQW analysis of Careers Scotland EEBL data 2.6 The local authorities that we spoke to have differing views about the appropriateness of target setting if they take over responsibility for the programme: some are strongly against them, whilst others fear that without targets, EEBL will disappear. Table 2-1 % target placements commenced Local authority / Careers Scotland region 2006-07 (12 months) 2007-08 (12 months) 2008-09 (9 months) 131% 25% 8% 88% 33% 0% Highland Council 109% 143% 52% Moray Council 114% 100% 40% Orkney Islands Council 140% 140% 80% Shetland Islands Council 113% 100% 27% Aberdeen City Council 59% 100% 14% Aberdeenshire Council 165% 100% 35% Angus Council 118% 77% 31% 64% 129% 43% Fife Council 100% 124% 94% Perth and Kinross Council 164% 169% 46% Highlands & Islands Argyll and Bute Council Comhairle nan Eilean Siar North East Dundee City Council South East City of Edinburgh Council 103% 72% 56% Clackmannanshire Council 125% 100% 0% East Lothian Council 200% 120% 80% 92% 100% 58% Falkirk Council Midlothian Council Scottish Borders Council Stirling Council West Lothian Council 63% 75% 38% 210% 230% 60% 88% 88% 50% 125% 56% 119% South West 8 Evaluation of Excellence in Education through Business Links (EEBL) Final report to Skills Development Scotland Local authority / Careers Scotland region 2006-07 (12 months) 2007-08 (12 months) 2008-09 (9 months) Dumfries & Galloway Council 121% 136% 164% East Ayrshire Council 100% 100% 8% North Ayrshire Council 100% 117% 17% North Lanarkshire Council 111% 100% 40% South Ayrshire Council 117% 133% 17% 86% 73% 52% East Dunbartonshire Council 250% 180% 40% East Renfrewshire Council 100% 183% 75% Glasgow City Council 150% 108% 88% Inverclyde Council 160% 58% 67% 80% 106% 44% West Dunbartonshire Council 280% 280% 40% TOTAL ACROSS ALL AREAS Source: Careers Scotland 120% 112% 55% South Lanarkshire Council West Renfrewshire Council 2.7 Although the current year 2008-09 is not yet finished, it is clear that there has been significant difficulty generating take-up this year. Sourcing and paying for supply cover has been problematic in some local authority areas in 2007-08, due to more general budget issues. The 2007 concordat between the Scottish Government and COSLA gives local authorities more say over how they spend the funding available to them, and some of our consultees have suggested that programmes such as EEBL may be disadvantaged as a result of this. 2.8 We also know from our consultations that, regardless of funding issues, there remain wider difficulties in making teachers aware of EEBL. (We discuss this later in the report.) Similarly, some EEBL co-ordinators report that employers are also facing problems: in the current economic climate, we have heard anecdotally that some are hesitant to invite external guests into their organisation at a time when they may be considering redundancy measures. Length of placement 2.9 Careers Scotland’s survey of teachers asks respondents to report how many days their placement lasted. These returns are not completed with consistency, but we have been able to put together an idea of how long the average placement lasted in years 2006-07 and 2007-08. As we can see in Table 2-2, this varies across regions. It is also clear that the average placement length has decreased over time in three of the five regions. Table 2-2 Average placement length in days Region 2006/07 2007/08 Highlands & Islands 4 2 North East 5 3 South East 4 4 South West 2 2 West 3 2 Source: Careers Scotland EEBL data, April 2006 to December 2008 9 Evaluation of Excellence in Education through Business Links (EEBL) Final report to Skills Development Scotland 2.10 In our e-survey of teachers we asked respondents to confirm how long they spent on their placement (see Figure 2-2). We found that the average length of placement is similar for primary and secondary teachers. The most common length of placement is two days for both primary and secondary teachers, with nearly half of all secondary placements and 37% of primary placements being two days in length. Indeed, if anything the data would suggest a polarisation to two or five day placements. We return later in the report to the potential impact of this in terms of soft benefits generated. 2.11 Nearly three quarters (72%) of respondents indicated that their placement was undertaken over consecutive days, while 28% had attended non-consecutively (i.e. on separate days over a period of time). Figure 2-2 Placement length Secondary Other Primary 5 days 4 days 3 days 2 day 1 day 0% 10% 20% 30% 40% 50% 60% Source: SQW Consulting (base: 206) NB: Other = more than 5 days 2.12 Given that the minimum duration of an EEBL placement is two days, it is not clear why twelve respondents indicated that their placement lasted only one day. Given that the majority of them took place in 2006-07 or 2007-08, it seems unlikely that this was the first day of a non-consecutive placement; it is perhaps more likely that respondents simply have a flawed recollection. Individual, national and themed placements 2.13 There are several ways in which placements can come about: an individual teacher recognises a need for professional development and asks their EEBL co-ordinator to help tailor a placement for them local EEBL co-ordinators identify host organisations who can deliver group placements on a particular theme 10 Evaluation of Excellence in Education through Business Links (EEBL) Final report to Skills Development Scotland the National EEBL Co-ordinator identifies national organisations that can deliver group placements to teachers from anywhere in the country. 2.14 When EEBL was conceived it was envisaged that individually-generated placements would form the basis of the programme. In practice, however, there has been a trend away from individual placements as a result of the difficulty in generating demand amongst teachers. We have been advised that the national management team strives to maintain a good balance between individual and themed placements – which implies a belief on the part of Careers Scotland that individual placements have particular value and should continue to be encouraged. 2.15 We used our e-survey to gauge the overall ratio of individual placements to national or themed ones. Feedback from the survey to date shows that the overall ratio is one third individual (tailored) to two thirds group (national/themed) placements. In Chapter 5 we discuss the relative merits of the different types of placement and the impact that they have. 11 Evaluation of Excellence in Education through Business Links (EEBL) Final report to Skills Development Scotland 3: Planning and preparing placements Key points The pre-placement planning stage is of vital importance to the success of EEBL. In particular, it is crucial that teachers and host organisations are absolutely clear about their aims and objectives. Pre-placement preparation is very time-consuming. The great majority of teachers and host organisations feel that pre-placement information and materials are ‘excellent’ or ‘good’. Although EEBL co-ordinators try to arrange for teachers and host organisations to meet prior to the placement, in practice this can be difficult to achieve (for teachers in particular). However, pre-placement meetings are to be encouraged, as they provide the opportunity for both sides to reach a clear and common understanding of their objectives. Introduction 3.1 This chapter outlines the planning and preparation that goes in to placements. This task involves considerable input from Careers Scotland staff, teachers and employers. Typical planning process 3.2 In the case of individual requests for placements, a suitable programme is developed for the teacher in discussion with the EEBL co-ordinator, who then identifies a suitable organisation to host the placement. 3.3 Increasingly, however, themed and national placements have become more common. A typical planning and preparation process for a themed placement, involving a host organisation experienced in offering placements, has been described to us by one EEBL coordinator thus: host organisation advises EEBL co-ordinator that it would like to offer a placement the following year co-ordinator starts the preparation several months in advance, to allow enough time for planning. The first step is to negotiate the best time for the placement to happen, bearing in mind that certain times of year are not ideal for teachers co-ordinator visits the host organisation to talk through the timescale, length of placement, and what outcomes the organisation hopes to achieve the organisation sends a brief to the co-ordinator co-ordinator makes any appropriate changes and sends the brief to the local authority for approval 12 Evaluation of Excellence in Education through Business Links (EEBL) Final report to Skills Development Scotland on approval, co-ordinator posts a letter to Headteachers along with the brief (direct communication by email is not permissible), inviting teachers to apply. This stage is crucial, as what happens to the brief at this point is entirely in the hands of the Headteacher co-ordinator also sends the same information to other EEBL co-ordinators within the same region, so that they can also market the placement within their local authorities co-ordinator receives applications from teachers and forwards them to the host organisation for approval (where placements are over-subscribed the host will select the best applicants) ideally a pre-placement briefing is held with the host organisation and the teachers. In practice it can be difficult for teachers to find time for such out-of-school meetings, so the briefing is often done by telephone and/or email. The briefing includes a review of the Careers Scotland paperwork, e.g. to stress the importance of teacher evaluation forms, which have to be submitted in order for local authority funding to be released. 3.4 The teacher arranges suitable supply cover and resource through their Headteacher or school CPD co-ordinator, and the placement goes ahead. 3.5 In the words of one EEBL co-ordinator: “Planning and preparation is everything: in particular, the host organisation must be very clear about its aim and objectives for the placement, and the benefit to teachers must be clear.” 3.6 During our consultations with EEBL co-ordinators a common message has emerged: this process is very time consuming. In this section we discuss some of the key features of the planning stage. Quality of pre-placement briefing and materials: teachers’ views 3.7 Careers Scotland asks teachers and host organisations to rate their level of satisfaction with the briefings and materials provided during the planning and preparation stage, using one of four indicators: excellent; good; fair; or poor. (Careers Scotland’s questionnaire does not require respondents to give more detailed feedback than a tick-box response.) 3.8 Overall approval of the pre-placement briefing and the pre-placement materials was high, with 78% of teachers categorising the pre-placement briefings as excellent or good (Figure 31) and 77% categorising the pre-placement materials as excellent or good (Figure 3-2). 3.9 Satisfaction with the pre-placement briefing and the pre-placement materials was similar across all regions apart from the Highlands and Islands region, where the majority of participants claimed that the quality of the pre-placement briefing and materials was ‘good’, compared to the majority in other regions who thought the quality was ‘excellent’. 13 Evaluation of Excellence in Education through Business Links (EEBL) Final report to Skills Development Scotland Figure 3-1 Teachers’ views on quality of pre-placement briefing 70 % of Respondents 60 50 40 30 20 10 0 SW SE NE HIE W Region Excellent Good Fair Poor Source: SQW analysis of Careers Scotland EEBL data, April 06 to December 08 (base: 715) Figure 3-2 Teachers’ views on quality of pre-placement materials 70 % of Respondents 60 50 40 30 20 10 0 SW SE NE HIE W Region Excellent Good Fair Poor Source: SQW analysis of Careers Scotland EEBL data, April 06 to December 08 (base: 715) 3.10 A very small number of teachers in Careers Scotland’s survey rated the pre-placement experience only as ‘fair’ or even ‘poor’. Not all of these respondents provided feedback to indicate why they were dissatisfied - indeed, in some cases their overall feedback about the placement experience was overwhelmingly positive, apparently outweighing any dissatisfaction over the pre-placement issues. However, a small number of negative comments were forthcoming: I think that the placement was a great opportunity but more consideration about our role whilst on placement was needed Enjoyed placement but felt that time could have been better used Objectives could have been clearer [same comment from two respondents] 14 Evaluation of Excellence in Education through Business Links (EEBL) Final report to Skills Development Scotland Quality of pre-placement organisation and information: host organisations’ views 3.11 Planning and preparation concern host organisations just as much as teachers, and it is important that both parties receive appropriate assistance. 3.12 Careers Scotland’s survey of host organisations found that 83% of organisations believed the quality of the pre-placement organisation to be ‘excellent’ or ‘good’ (Figure 3-3), while 80% found the standard of pre-placement information to be ‘excellent’ or ‘good’ (Figure 3-4). It is worth noting that these satisfaction levels are even higher than the corresponding findings from teachers (see paragraph 3.8). Host organisations in the North East are significantly more likely to find the pre-placement experience to be ‘good’ (as opposed to excellent). It is not clear why this should be the case. Figure 3-3 Employers’ views on quality of pre-placement organisation 70 % of Respondents 60 50 40 30 20 10 0 SW SE NE HIE W Region Excellent Good Fair Poor Source: SQW analysis of Careers Scotland EEBL data, April 06 to December 08 (base: 210) Figure 3-4 Employers’ views on quality of pre-placement information 70 % of Respondents 60 50 40 30 20 10 0 SW SE NE HIE W Region Excellent Good Fair Poor Source: SQW analysis of Careers Scotland EEBL data, April 06 to December 08 (base: 210) 15 Evaluation of Excellence in Education through Business Links (EEBL) Final report to Skills Development Scotland 3.13 Examples of reflective feedback from host organisations, as recorded in Careers Scotland’s survey, are highlighted below. As hosts of the placement there are things we could possibly improve e.g. sending a briefing pack to teachers in advance. Might have been useful to have a pre-placement meeting so that the content of the presentation could be tailored to suit their requirements. With so many applications it would have been useful to have more of a recruitment process. Would like to arrange more meetings/opportunities for teachers to engage with more staff at the centre. Teacher motivation 3.14 We were interested to explore why teachers choose to take part in an EEBL placement. In our e-survey of teachers we asked them where the idea came from: did they instigate the placement themselves, or was the idea suggested to them? 3.15 Primary and secondary teacher responses were very similar; 36% and 37% respectively said that the placement was their own idea, while 29% and 30% said that the idea came from Careers Scotland. Of those teachers who responded ‘other’ to this question, the most common explanation was that they had enrolled on the EEBL placement as part of the Scottish Qualifications for Headship Programme. This suggests the important role for those overseeing EEBL in promoting the opportunities directly to teachers and through more general CPD channels. “I was frustrated that there was no public gallery in this area or somewhere the pupils could go to experience the public gallery environment. In this rural area, there is not much to do in terms of creative/ arts activities. I grew up in the local area and strongly felt not being able to experience art in a gallery, somewhere local. I wanted to address this problem and see what could be done to provide pupils with the experience of developing their understanding of art and specifically the perspective that galleries can bring. In the past we had made visits to galleries out-with the region and the pupils were not able to fully take in the experience because it was all new and very different environment for them. It became apparent to me that they needed to be educated and introduced to the galleries in a different way which tied in with their classroom activities. In essence, I did the placement for creative interaction and to broaden horizons. Working in a school can only give you certain view of art and the creative industries. I wanted to expand my horizons as well as introducing the pupils to new art forms through galleries.” Teacher, Lossiemouth High School Teachers’ aims 3.16 One of the important issues that has been highlighted in our consultations is the importance of teachers – and host organisations – having a very clear idea of why they want to do a placement and what they hope to achieve from it. This is part of the overall planning and preparation process. 3.17 Our consultations suggest that teachers find it difficult to identify and articulate their aims. From some EEBL co-ordinators we have heard that teachers struggle to express exactly what 16 Evaluation of Excellence in Education through Business Links (EEBL) Final report to Skills Development Scotland it is that they want to achieve from their placement and require a lot of advice and assistance from Careers Scotland in order to complete a satisfactory application form. (Indeed, the feedback from our e-survey of teachers confirms that some had clearer ideas than others about why they wanted to do a placement.) The fact that EEBL co-ordinators have been able to spend time helping teachers to sharpen their objectives and to articulate them effectively in their application form has been a strength of the programme – but this process can be highly resource-intensive. 3.18 Below we provide an overview of teachers’ aims, summarising some of the main themes arising from the e-survey feedback about why teachers want to do an EEBL placement. To see and understand at first hand the job opportunities available to young people and understand what skills are needed, e.g: To gather information that would help pupils to gain employment with local engineering companies [to gain] an insight into business and the skills/attributes businesses are looking for in prospective employees. This in turn would help influence my teaching methods to ensure I was giving pupils the best possible chance of being employed in the private sector I wanted to gain knowledge of the workplace and how young people could get jobs when they leave school, what skills are required, etc To improve knowledge of a particular subject area (e.g. architecture, space) in order to deliver better lessons, e.g: To be able to talk authoritatively about real-life issues, e.g: I was teaching a P6 class, one of their topics was Space - an area of personal development need. I was therefore keen to develop my own knowledge, and learn new ways to make the subject relevant and interesting for the children I teach I wanted to be able to discuss with my students the current issues involving IT, and I didn't want it to just be what I had read about in a book - I wanted to be able to talk from a real-life perspective To understand what services are available to young people outside school, e.g: I wanted to get an understanding of the different providers/facilities in West Lothian and to find out exactly where these things were to let pupils know Increased knowledge & understanding of the appropriateness agencies/supports. Personal contact with workers, agency reps of To update professional knowledge and experience, e.g: 17 Evaluation of Excellence in Education through Business Links (EEBL) Final report to Skills Development Scotland Having a background in engineering before teacher training I was interested to see how engineering had moved on and to see if I could pick up some pointers etc for classroom use It was a long time since I'd had computer experience in the workplace and I wished to address that Interest in helping their school to gain a particular status (e.g. Eco-School or Forest School) by improving knowledge and understanding of particular issues, e.g: I undertook Forest Schools level 3 leadership training as a way to gain first hand experience of outdoor experiences which I could them use in school How to get Fair Trade status for the school I'm running an eco-project at school and wanted to further my green credentials To gain new management or leadership skills (notably in the context of the Scottish Qualification for Headship) To look at different aspects of management within an industrial setting i.e. quality assurance, succession management To compare the way people are managed, evaluated and developed as individuals. 3.19 All of these aims reflect the various ‘benefits to teachers and education’, as set out in the EEBL Policy and Practices Framework. 3.20 In Section 4 we explore whether teachers’ aims and expectations are being met. 18 Evaluation of Excellence in Education through Business Links (EEBL) Final report to Skills Development Scotland 4: Placement experience Key points The vast majority of teachers and host organisations rate their EEBL experience very highly. Teachers on themed or national placements have broader expectations than those on individual placements, particularly about achieving additional motivation, developing new lesson plans and teaching resources and making new contacts. Those on individual placements are more likely to have expectations about gaining insight into how organisations work and learning about management and leadership. Primary teachers have higher expectations than secondary teachers about gaining new confidence as a teacher and producing new lesson plans or teaching resources. The outcomes that teachers expect from their placement are generally being achieved: over two thirds of teachers believe that their desired outcomes are met in full. 4.1 In this section we provide an overview of teachers’ and host organisations’ general satisfaction with their placement experience. We then look at the extent to which teachers’ aims and expectations have been met. Teacher review 4.2 In total, 715 teachers in the period April 2006 to December 2008 completed a placement quality review. The review is carried out by Careers Scotland after placements have been completed to assess teachers’ views on the quality of their placement. Placement programme content 4.3 Respondents were asked to rate the content of their placement. Very few respondents categorised the programme as fair or poor. Of the 715 teachers surveyed between April 2006 and December 2008 only three teachers viewed the placement content as ‘poor’ in contrast to the 439 people who thought it ‘excellent’. The key message from these responses regarding the quality of the placement is that the substance of the placement fulfils the requirements of those who take part to a very high degree, resulting in high satisfaction ratings from the majority of participants in EEBL. 4.4 A high proportion of teachers in all regions excluding the North East rated the EEBL programme as ‘excellent’ ranging from 66% in the West to 58% in the South East. In the North East, 29% rated the programme as excellent. However, this may not be a reflection of the content of the programme in the North East as a high number of respondents in that region (36%) did not answer this question, thus perhaps skewing the results. 19 Evaluation of Excellence in Education through Business Links (EEBL) Final report to Skills Development Scotland Figure 4-1 Teachers’ views on placement content 70 % of respondents 60 50 40 30 20 10 0 SW SE NE HIE W Region Excellent Good Fair Poor Source: SQW analysis of Careers Scotland EEBL data, April 06 to December 08 (base: 715) Host review 4.5 In total, 210 host organisations completed a placement quality review covering the period April 2006 to December 2008. This captured host organisations’ views on a number of aspects of the placement programme such as pre-placement organisation, their views on the teacher experience and their intentions regarding maintaining future links with teachers. 4.6 The vast majority of host organisations were happy with the overall teacher placement experience, with 81% of all host organisations reporting that the placement was either ‘excellent’ or ‘good’. Encouragingly, no host organisation believed that the teacher placement experience was ‘poor’ and of the 210 organisations only two reported their experience of the placement as ‘fair’. Figure 4-2 Hosts’ experience of teacher placements 70 % of Respondents 60 50 40 30 20 10 0 SW SE NE HIE W Region Excellent Good Fair Poor Source: SQW analysis of Careers Scotland EEBL data, April 06 to December 08 (base: 210) 20 Evaluation of Excellence in Education through Business Links (EEBL) Final report to Skills Development Scotland 4.7 Across the regions the perceptions of host organisations regarding the teacher placement experience differed, with large majorities in both the North East (67%) and West (61%) believing the experience to be excellent, while in the South West the largest proportion of respondents (48%) believed the experience to be ‘good’. This was of great benefit to the training dept. Rod's skills as a teacher allowed us to have our training packages/methods reviewed. This, coupled with Rod's involvement with the specials [constables], provided a very worthwhile experience. There are no areas of the placement programme that require improvement. The organisation will liaise with Maureen to further support the teaching of waste education at her school. Her enthusiasm gave the education team a lift. Educational, inspiring, an eye-opener, really made me understand the requirements of teachers and the connection of the Scottish Curriculum to active learning and engagement of young people. Margaret's participation made such a difference, the experience she was able to bring helped focus the provision we had put together. Margaret has agreed to bring her class along on the 21st of March to pilot the workshop. We aim to get evaluation and feedback from both Margaret and her pupils. Margaret also helped with putting together proprietary and follow up resources linked to objects from the Museums collections that can be loaned out to schools. Again she has taken this material in draft form and is piloting it with her pupils in class. Once her project is complete Margaret has agreed to revisit us and evaluate these resources. I really think that this experience was very well organised and the two projects I have been involved in were both productive and instructive... The single thing that would help … to make the placements even better is TIME. Five days is not long enough for the person to become used to the house and - I would say - to feel confident in what they can do / achieve. Otherwise - excellent! We would like to use teacher placement to help us improve the way in which we communicate with teachers and also support the development of our new schools programming as part of our redevelopment project 20082011. Are anticipated outcomes being achieved? What outcomes did teachers expect to achieve 4.8 In Section 3 we looked at teachers’ aims for their placement, which they defined during the pre-placement planning phase (i.e. why did they want to do the placement). In this section we explore the particular outcomes they expected to achieve (i.e. what did they want to get out of it), and whether these expectations were fulfilled in practice. 4.9 Figure 4-3 indicates that teachers’ aims and expectations are similar across primary and secondary sectors. The most commonly cited expectations relate to: gaining new insight into how organisations work; gaining additional motivation as a teacher; and making useful new contacts. Primary teachers were more likely to express expectations about gaining new 21 Evaluation of Excellence in Education through Business Links (EEBL) Final report to Skills Development Scotland confidence as a teacher, producing new lesson plans and new teaching resources. More primary teachers expected to be able to organise class visits to the organisation after their placement, whereas more secondary teachers expected to be able to organise work experience for pupils (Figure 4-3). Figure 4-3 Did you expect to achieve any of the following? (proportion saying yes) 100% Primary Secondary 80% 60% 40% 20% Learn more about management/leadership Pupils invited to do work experience at the organisation Class able to visit the organisation after my placement New school-business link established for the longer term Make useful new contacts Produce new teaching resources Produce new lesson plan/s Freshen approach to teaching Gain new confidence as a teacher Gain additional motivation as a teacher Gain new insights into how an organisation works 0% Source: SQW Consulting (base: 206) 4.10 As shown in Figure 4-4, expectations were more broadly held amongst those teachers who were on a national or themed placement (i.e. as part of a larger group of teachers): on average 63% of teachers taking part in a group placement reported that they expected results in each category shown in Figure 4-3 and Figure 4-4, compared to 58% of teachers who were on an individually tailored placement. This might reflect a more specific and self determined need amongst those on individual placements . 4.11 Expectations were particularly high amongst those on national/themed placements with regard to achieving additional motivation as a teacher, developing new lesson plans and teaching resources, making new contacts and being able to organise class visits. Those on individual placements were more likely to have expectations about gaining new insights into how an organisation works and learning about management and leadership (Figure 4-4). 22 Evaluation of Excellence in Education through Business Links (EEBL) Final report to Skills Development Scotland Figure 4-4 Did you expect to achieve any of the following? (proportion saying yes) 100% Only participant Part of a larger group 80% 60% 40% 20% Learn more about management/leadership Pupils subsequently invited to do work experience at the organisation Class able to visit the organisation after my placement New school-business link established for the longer term Make useful new contacts Produce new teaching resources Produce new lesson plan/s Freshen my approach to teaching Gain new confidence as a teacher Gain additional motivation as a teacher Gain new insights into how an organisation works 0% Source: SQW Consulting (base: 135) 4.12 There was little difference in expectations depending on whose decision the placement had been. Expectations were highest among those who had made the decision to do the placement themselves but those on placements for other reasons also had high expectations. 4.13 Finally, teachers’ expectations of the placement were analysed according to the length of the placement. There were significant differences in expectations surrounding gaining new confidence, with 40% of two-day participants expecting to achieve new confidence as a teacher compared to over 60% of those teachers on a three or four day placement. This pattern is also strong in teachers’ expectations surrounding the production of new lesson plans and teaching materials, with expectations rising as the length of the placement increases (Figure 4-5). Teachers on shorter placements generally had fewer objectives, perhaps reflecting realism about what could be achieved in the time available. Figure 4-5 Did you expect to achieve any of the following? (proportion saying yes) 100% 80% 60% 40% 20% 0% 1day 2 days 3 days 4 days 5 days Other Gain additional motivation as a teacher Gain new confidence as a teacher Freshen my approach to teaching Produce new lesson plan/s Produce new teaching resources Source: SQW e-survey of teachers (base: 208) 23 Evaluation of Excellence in Education through Business Links (EEBL) Final report to Skills Development Scotland Were expectations met? 4.14 Careers Scotland data across all five regions shows that satisfaction with placements is high amongst teachers (Figure 4-6): just over two thirds of respondents (68%) believe that their desired outcomes from the placement were met in full. A further 23% of respondents felt their desired outcomes were met ‘somewhat’, while only a very small minority reported that they were not met at all. Figure 4-6 Were teachers’ desired outcomes met? 80 70 % of Respondents 60 50 40 30 20 10 0 SW SE NE HIE W Region Yes totally Yes somewhat No, not really Source: SQW analysis of Careers Scotland EEBL data, April 06 to December 08 (base: 715) 4.15 We have compared Careers Scotland’s data with the findings from our own survey, and there is a strong correlation in terms of aims being met. In our own survey, a very high proportion of respondents reported that they had achieved their aims of going on the placement (73% and 79% of primary and secondary teachers respectively) – rather higher than indicated by Careers Scotland’s data. A further 24% and 21% felt that their aims had been achieved in part (similar to Careers Scotland data). 4.16 Our survey differs from that of Careers Scotland in that we also asked teachers to define what outcomes they expected to achieve from the placement, using a list of prompts. The highest proportion of teachers reported that their expectations had been met in terms of gaining additional motivation as a teacher, gaining new insights into how an organisation works and making useful contacts (answering 4 or 5 on a scale of 1 to 5, where 1 is 'expectations not met at all', and 5 is 'expectations met in full'). These are important issues that underpin the rationale for the programme. 24 Evaluation of Excellence in Education through Business Links (EEBL) Final report to Skills Development Scotland Figure 4-7 Proportion of respondents who answered 4 or 5 to the question “how far were your expectations met, on a scale of 1 to 5 (where 1 is 'not at all', and 5 is 'met in full')” 100% Primary Secondary 80% 60% 40% 20% Learn more about management/leadership Pupils subsequently invited to do work experience at the organisation Class able to visit the organisation after my placement New school-business link establihsed for the longer term Make useful new contacts Produce new teaching resources Produce new lesson plan/s freshen my approach to teaching Gain new confidence as a teacher Gain additional motivation as a teacher Gain new insights into how an organisation works 0% Source: SQW Consulting (base: 208) 4.17 On average, our survey shows that secondary teachers’ expectations were met to a higher degree than those of primary teachers, with an average of 58% of secondary teachers responding that their expectations in each category were met, compared to 52% of primary teachers. Placement structure 4.18 Teacher surveys (our own and that of Careers Scotland) have not specifically sought to investigate the benefits of consecutive and non-consecutive placement periods. However, from the small amount of qualitative feedback available, the non-consecutive approach has its merits: I think I should have split up the days over 2 weeks to allow me time to reflect on what was being done [teacher] 4.19 As we noted earlier, teachers on non-consecutive placements were slightly more likely to have come away with tangible outputs such as new teaching materials. 4.20 One host organisation commented on the structure of the placement thus: It might be useful to spread placements over a period of time, e.g. one day per week rather than doing them in a block of three days at a time. This could be very convenient for the teacher and the host site. 4.21 We do not have sufficient evidence on which to make a robust conclusion about the relative merits of consecutive and non-consecutive placement periods, but clearly the latter approach should not be dismissed out of hand. 25 Evaluation of Excellence in Education through Business Links (EEBL) Final report to Skills Development Scotland 5: Benefits and impact of EEBL Key points Nearly half of all teachers feel that their placement reinforced the relevance of their taught subject, and nearly half feel that a key benefit of EEBL is in allowing them to share ideas and thoughts with others. More than one third of teachers report that the placement helped them with some or all of the four capacities in Curriculum for Excellence. The vast majority of teachers (with only two exceptions) have disseminated their learning and experience following their EEBL placement. Almost one third of teachers recommend EEBL to their headteacher, and 56% encourage their peers to apply for a placement. Teachers feel that EEBL is a very valuable contribution to CPD, and many suggest that every teacher should be encouraged to undertake a placement. The primary impacts on pupils are in helping them to make links and connections and to see the relevance of their school work to employment, and in broadening their horizons (e.g. increasing their social and environmental awareness). Teachers on themed or national placements report slightly higher impacts on pupils than teachers on individual placements. 5.1 The main purpose of this evaluation is to assess the impact of EEBL on the participating teachers – and, if possible, on their pupils. In this section we consider the tangible and intangible benefits of placements, and explore what the overall impact of the programme has been. Tangible outputs arising from placements 5.2 In our e-survey of teachers we asked them to identify any tangible outputs, such as new lesson plans or teaching materials, arising from their placement. Of the 130 teachers who reported tangible outputs, exactly half (65) described the creation of new teaching materials or resources for teachers – while the other half gave details of non-tangible outputs such as future links, new ideas, information, etc. 5.3 The characteristics of the 65 placements resulting in tangible outputs are: one third primary and two thirds secondary teachers (approximately in line with the overall take-up of EEBL amongst primary and secondary) one third non-consecutive structure and two thirds consecutive days (a slightly higher proportion of non-consecutive placements than at overall programme level) 26 Evaluation of Excellence in Education through Business Links (EEBL) Final report to Skills Development Scotland equally split between short (2 days or less), medium (3 days) and long (4 days or more) placements. 5.4 Initially it may seem disappointing that only 31% of all respondents came away from their placement with new teaching resources or other tangible outputs – but as the EEBL Policy and Practices Framework puts no particular emphasis on this, it could in fact be regarded as a positive outcome. 5.5 Examples of tangible outputs cited by respondents include: Variety of healthy eating recipes introduced to the curriculum. We have also registered with the 'Cooking Bus' and plan to arrange a visit to the school next year Excellent resources made from using photos from the archive and used in lessons - good comparisons from old places to new in their local area Teaching pack on sectarianism Global Citizenship booklet for secondary schools 5.6 A large proportion of responses to this question related to new ideas, new contacts, subsequent class visits, new links and improved information and advice to pupils. The tangible outputs from my placement were: Developing class materials for pupils on the topic of money Taking the class out to visit the museum for a whole day and explore the history of money Created a teachers pack of material on the topic Attended conferences and other events with the Scottish Government and Careers Scotland to showcase my experience of EEBL. One of the conferences was at Radisson Hotel Attended a workshop in Glasgow on enterprise and took a few pupils with me Produced a short film for Learning Teaching Scotland on financial education/ the placement, which includes interviews with myself, Museum representatives and pupils: http://www.ltscotland.org.uk/video/f/royalmileprimaryschool.asp?strReferrin gChannel=eie. – Teacher, Royal Mile Primary School Application of subject within the world of work 5.7 In its own survey of teachers, Careers Scotland asks teachers to explain the ways in which their placement has increased the practical application of their subject within the world of work, selecting from the following responses: raised awareness of technical skills needed; reinforced the relevance of my subject area to the world of work; helped me understand the soft skills required; improved my understanding of the labour market. 5.8 Many believed the placement reinforced the relevance of their taught subject (46%). Raising their awareness of the technical skills needed is also important (cited by 30%, Figure 5-1). One fifth of respondents felt that the placement had helped them in their understanding of the 27 Evaluation of Excellence in Education through Business Links (EEBL) Final report to Skills Development Scotland soft skills required within the world of work – perhaps a disappointing outcome, given that national employer surveys (Futureskills1, etc) so often highlight the need for soft / core skills and of these being the key shortcoming of young recruits. Figure 5-1 Application of subject within the world of work Raised awareness of technical skills needed Helped in understanding the soft skills required Reinforced the relevance of my subject Improved my understanding of the Labour Market 0 5 10 15 20 25 30 35 40 45 50 % of Respondents Source: SQW analysis of Careers Scotland EEBL data – base (212) Aspect of area of specialism that gained most from the placement 5.9 Careers Scotland also asks teachers to comment on which aspect of their area of specialism benefited most from the placement. Again, the questionnaire invites teachers to select from four options: improved context; helped me with some or all of the four capacities in Curriculum for Excellence; enhanced my technical knowledge; allowed me to share thoughts and ideas with others. 5.10 Nearly half of respondents report that the area of specialism that gained most from the placement was that it allowed them to share ideas and thoughts with others (47%). Over a third (35%) felt that their ability to deliver the capacities within Curriculum for Excellence had gained from the placement. Teachers also reported that the placement had giving them an improved context of their area of specialism (28%) as well as enhancing their technical knowledge (25%). 5.11 Given the objectives behind EEBL and Curriculum for Excellence, these are perhaps lower positive responses than might have been hoped. One explanation may be that teachers have not been able to make the linkages that should exist and may need support in defining these when planning and reflecting on placements. EEBL has the potential to make a real contribution to all areas of Curriculum for Excellence, and it is important that these linkages are emphasised, whoever takes over responsibility for the programme. For example, “Futureskills Scotland - Research Series - Scottish School Leavers and Their Understanding of the World of Work (2005) - http://www.scotland.gov.uk/Resource/Doc/919/0065462.pdf 1 28 Evaluation of Excellence in Education through Business Links (EEBL) Final report to Skills Development Scotland Figure 5-2 Area of specialism that gained most from the placement Improved context Capacities within Curriculum for Excellence Enhanced my technical knowledge Allowed me to share ideas/thoughts with others 0 5 10 15 20 25 30 35 40 45 50 % of Respondents Source: SQW analysis of Careers Scotland EEBL data – base (212) Dissemination of experience 5.12 In our e-survey we invited teachers to indicate, from a list of prompts, whether they had had any opportunities to disseminate their learning and experiences after their placement (Figure 5-3). The vast majority of teachers – with only two exceptions – reported that some form of dissemination had taken place. This is a very positive finding. 5.13 The main route for disseminating learning for secondary teachers was through informal discussion with other teachers (41%) followed by more formal dissemination through feedback to senior colleagues (24%). For primary teachers, sharing of learning was done through informal discussion (28%), feedback to senior colleagues (26%) and through ‘other means’ (31%) - generally a combination of the above options. Figure 5-3 Have there been any opportunities to disseminate the learning that you gained from your placement? 45% Primary Secondary 40% 35% 30% 25% 20% 15% 10% 5% 0% Informal sharing of Formal/structured information with sharing of other teachers in my information with school other teachers (e.g. staff meetings) Sharing of new resources/materials with other teachers Feedback given to headteacher and/or other senior collegues Other Source: SQW Consulting – e-survey of teachers (base: 191) 29 Evaluation of Excellence in Education through Business Links (EEBL) Final report to Skills Development Scotland 5.14 Of those who indicated ‘other’ dissemination opportunities, approximately one third reported that they pursued all or most of the options listed. Two reported that no dissemination had taken place. 5.15 Other specific comments included: report into school newsletter which goes home to parents as well as staff. Also article in local newspaper sharing information with staff (informally) within my Learning Community and with other coaches across the city feedback to our host at the centre outlining the potential of the location to help improve topics within the curriculum I invited MD of factory to shadow me for a day. 5.16 A small number of respondents reported that dissemination had taken place with pupils. Perhaps more interestingly, three teachers reported that parents were amongst the beneficiaries of dissemination (“sharing with teachers and parents during an information meeting”; “report in school newsletter which goes home to parents”; “presentation at parentteacher forum”). 5.17 Also on the theme of dissemination, we looked at Careers Scotland’s data on whether teachers recommended EEBL placements to others or talked to colleagues about the benefits of placements. As we can see in Figure 5-4, over half the of respondents (56%) reported encouraging other teachers to apply for or request placements by ‘word of mouth’, which is an encouraging indication of satisfaction with the programme. Almost one third (31%) of respondents also recommended EEBL to their head teacher. 5.18 Also of note is the fact that 11% of teachers subsequently gave presentations to their school or colleagues on their placement. Our survey and consultations suggest that there may be merit in trying to increase this figure: a number of teachers have specifically reported that it ‘would have been useful’ if they could have listened to such a presentation prior to undertaking their own placement. Figure 5-4 Encouraging others to go on EEBL placements Presentations to school or colleagues Case study on the school website/newsletter Word of mouth Recommended to Head Teacher 0 10 20 30 40 50 60 % of Respondents Source: SQW analysis of Careers Scotland EEBL data – base (212) 30 Evaluation of Excellence in Education through Business Links (EEBL) Final report to Skills Development Scotland Contribution of EEBL to continuing professional development (CPD) 5.19 We asked teachers to say how they thought EEBL placement compared to other forms of CPD. The vast majority (190 of 200 respondents) said that placements were a useful addition to other forms of CPD. Only four respondents thought that placements were less useful than other forms of CPD. When asked to expand upon their answer, feedback was generally very positive. The following quotes provide a flavour of teachers’ views on this subject: Could be a more 'normal' experience - at my school, only 2 of 18 teachers have done an EEBL placement. in 17 yrs, this was my FIRST one. Shouldn't be mandatory, but SMT should maybe get HGIOS ticks for teachers going on placement? EEBL Placements often are too shortnotice, or badly-timed (i.e. I won't apply for one when I'm doing report cards, Parents Nights, or in Holidays). Some of them sound very boring (but loads sound great - I am trying to sell them to other teachers, and indeed my SMT who would get a lot from the best EEBL Placements, I think) Make it compulsory part of staff development. And when that member of staff has returned they must go about devising some way of improving their curriculum from the knowledge gained. This could be in the form of a one off visit by the business that that member of staff organises or it could be something more embedded, such as a new topic taught. Every teacher should be encouraged/obliged to participate in at least one placement every few years. This would help remind teachers to encourage children to think positively about gaining meaningful employment. This is especially important for children who come from families who are longterm unemployed. There should be more opportunities for teachers to broaden their horizons I think these placement enhance our teaching practice and should continue to be fully funded so that supply cover never becomes an issue Both placements I have undertaken have been invaluable to my CPD. I have encouraged members of my department to take these forward as it is essential teachers are aware of what happens in the "real world" The most valuable professional development I have ever taken part in. It would be a great loss if it was to end Links with host organisations 5.20 Maintaining links with host organisations is one of the benefits we might expect to see arising from EEBL placements. Careers Scotland data (see Figure 5-5) shows that this is the intention of the majority of teachers. On average, 81% of teachers reported that they would maintain links with their host organisation after their placement had finished. Only 8% did not think this would happen. 31 Evaluation of Excellence in Education through Business Links (EEBL) Final report to Skills Development Scotland Figure 5-5 Teachers’ intentions to maintain links with host organisation 100 90 % of Respondents 80 70 60 50 40 30 20 10 0 SW SE NE HIE W Region Yes No Source: SQW analysis of Careers Scotland EEBL data, April 06 to December 08 (base: 715) 5.21 Careers Scotland have also asked host organisations to report back on whether they envisage that links with the visiting teacher (and associated school) will be maintained. Encouragingly, 76% of all host organisations surveyed by Careers Scotland intend to maintain contacts with teachers – indicating that employers intend to maintain links almost as much as teachers do (81%). 5.22 A more detailed analysis across the five regions illustrates a disparity in intentions, with 100% of host organisations in the North East intending to maintain links, compared to only 25% in the Highlands and Islands. Given that many teachers in Highland region are likely to undertake a placement outwith their normal travel-to-work area (for example, we know that several have done national placements in Edinburgh), it seems logical to assume that they may be less likely to maintain contact with their host after the placement ends. Figure 5-6 Host organisations intending to maintain links with teachers 100 90 % of Respondents 80 70 60 50 40 30 20 10 0 SW SE NE HIE W Region Yes No Source: SQW analysis of Careers Scotland EEBL data, April 06 to December 08 (base: 210) 32 Evaluation of Excellence in Education through Business Links (EEBL) Final report to Skills Development Scotland 5.23 Of course, the intention of schools and employers to maintain the links that have arisen from teacher placements may not necessarily come to fruition. There are indications from our telephone consultations with teachers and hosts, as well as from the existing data, that it can be difficult on both sides to find the time and opportunity to build upon placement activity no matter how much this might be desired. 5.24 Careers Scotland data give us an indication of the translation of ‘intention’ into ‘reality’, and what form any further links have actually taken in practice (Figure 5-7). One fifth of teachers report that they have further engaged the host organisation by involving them in school activities. A smaller number report that a member of the host organisation has subsequently visited the school (13%) and that other colleagues had requested placements from the host organisation (11%). Figure 5-7 Further engagement with host organisations Involved the organisation in school activities A member of the organisation visited my school Other colleagues have requested placements Work experience placements arranged 0 5 10 15 20 25 % of Respondents Source: SQW analysis of Careers Scotland EEBL data – base (212) 5.25 Comments from host organisations include: X’s EEBL placement has been an excellent example of how such links can be used to forge ongoing partnership working between schools and industry/business. EEBL placement has proved to be an effective way for teachers to learn and understand more about MFiF and hence be in a position to share this with young people. Have undertaken local partnership agreement with Hecklegirth Primary and have a programme of events arranged for forthcoming year. We intend to pilot the resources that will be produced as a result of the placement. We would like to continue to have teachers on a regular basis to ensure our resources remain relevant to the curriculum. Both teachers are going to trial some of the materials they contributed to within their schools. Fiona's students are contributing to a Power Point presentation which will be used as educational material next year. Important to keep up to date with new information within the education system passed onto teachers, also to have their input on new & exciting ways to promote the Safari Park through schools by assisting to develop new materials. 33 Evaluation of Excellence in Education through Business Links (EEBL) Final report to Skills Development Scotland Impact on pupils 5.26 Careers Scotland asks teachers to report back on what impact the placement has had on their pupils, using four indicators: pupils have a greater understanding of how the subject fits in context with the wider world; pupils feel that my EEBL placement added weight to aspects of my teaching; pupils found my relating of the experience interesting and informative; or too early to say. 5.27 Our analysis of the findings from Careers Scotland’s survey (seen in Figure 5-8) reveals that the greatest effect on pupils as reported by teachers is that they found the experience interesting and informative (33%) and that it gave a greater understanding of the subject in context (30%). These responses are perhaps below what might have been expected given the positive feedback on other issues reported above. Timing may play a part, as indeed might the relatively indirect routes to impact. However, this reinforces the concern raised above about the limited changes to learning plans or links to Curriculum for Excellence and suggests that in future such linkages need to be more clearly articulated. Figure 5-8 Effect of placement on pupils taught Greater understanding of subject in context EEBL placement added weight to teaching Found experience interesting and informative Too early to say 0 5 10 15 20 25 30 35 % of Respondents Source: SQW analysis of Careers Scotland EEBL data – base (212) 5.28 We sought to explore the impact of EEBL in more detail in our e-survey of teachers. In particular, we asked teachers about the impact that they felt the placement had had on their pupils. Both primary and secondary teachers reported predominantly positive impacts on pupils, with 73% of primary teachers and 78% of secondary teachers stating that there was ‘some’ or ‘a lot’ of impact on pupils (Figure 5-9). That said, many more reported ‘some’ impact, suggesting a fairly moderate effect. 5.29 The qualitative feedback from our survey indicates that the impact of EEBL on pupils, as perceived by teachers, falls into two main categories: increased ability of pupils to make links and connections and to understand the relevance of what they are studying; and broadening pupils’ horizons. Other impacts include gaining access to new equipment, and increasing pupil representation (‘pupil voice’). Some examples are highlighted below. Making links and connections: The knowledge learned has enabled me to make more interesting and relevant teaching resources, and pupils clearly see a better link between their lives and those of people in the past. 34 Evaluation of Excellence in Education through Business Links (EEBL) Final report to Skills Development Scotland Using my materials as evidence, students were able to stay focused. I was able to obtain different explanations of how Skara Brae came to be. This allowed students to think about using evidence and come to conclusions. Pupils can directly relate classwork to employment in Engineering and they also have a greater understanding of what will be expected/involved if they work for an engineering company. The class were invited to attend workshops on light technology. They were able to make links between the cutting edge science they saw, and the basic science they had been learning. The wide range of resources, the hands-on opportunities, & the enthusiasm of the organisers broadened their horizons significantly, and added purpose to Primary School Science. Pupils are interested due to relevance to them as it was a local company and some know people that work there. Prompted more discussion than normally would have. Placement led to project on stone circles near the school - pupils gained better understanding and appreciation of local history. Broadening horizons: Pupils are taken outdoors much more. Children are showing more interest in their local environment. My class have been involved in raising money to buy apple trees for our school grounds to make them nicer. They raised more money than expected so have bought trees for neighbouring community facilities as well. John Hancox (Children's Orchard) has been out to speak to the children about planting and looking after trees and is hoping to involve us in further events. New material on sectarianism were produced and piloted. Amendments to be made. Link with local catholic school made. Both schools jointly worked with Citizens Theatre on 'Divided City' drama workshops Access to new equipment: Children had a chance to use photography equipment that the school could never have afforded and also had expert tuition from the visiting artist. The photographs produced from the visit were used to produce a book and one image used as a Christmas card for a social enterprise project. Pupil voice: I learnt how a private organisation valued and encouraged staff for the most bringing out the best in them. Following my placement I made subtle changes to team work by thinking more about time, place and structure, and prompts. This was mainly in relation to management/leadership of staff but also impacted on pupils i.e. pupil consultation/pupil voice / pupils committees such as the Pupil Council and Eco Committees. 35 Evaluation of Excellence in Education through Business Links (EEBL) Final report to Skills Development Scotland Figure 5-9 Do you think there has been any impact on the pupils that you teach as a result of your placement? 60% Primary Secondary 50% 40% 30% 20% 10% 0% No impact Very little impact on pupils Some impact on pupils A lot of impact on pupils Source: SQW Consulting (base: 2010 5.30 Of particular significance, however, is an analysis of how impact on pupils is affected by the length of placement. Teachers who had been on longer placements felt that there was a greater impact on pupils than those teachers on shorter placements (Figure 5-10). The proportion of teachers who believe that their placement has had a lot of impact on the pupils they teach increases with the length of the placement. Figure 5-10 Do you think there has been any impact on the pupils that you teach as a result of your placement? 60% Short placement (3 days or less) Long placement (4 days or more) 50% 40% 30% 20% 10% 0% No impact on pupils Very little impact on pupils Some impact on pupils A lot of impact on pupils Source: SQW Consulting (base: 201) 5.31 There was very little difference on pupil impact depending on whether teachers had been on an individual or national/themed placement. Teachers whose placements were as part of a group reported slightly higher impacts on pupils than teachers on individual placements, with 79% of those on group placements reporting some or a lot of impact compared to 72% of teachers on individual placements. 36 Evaluation of Excellence in Education through Business Links (EEBL) Final report to Skills Development Scotland Figure 5-11 Do you think there has been any impact on the pupils that you teach as a result of your placement? 60% Individual Group 50% 40% 30% 20% 10% 0% No impact on pupils Very little impact on pupils Some impact on pupils (please describe) A lot of impact on pupils (please describe) Source: SQW Consulting (base: 198) 5.32 Impact on pupils was also analysed by the kind of placement that teachers went on; one over consecutive days or non-consecutive over a longer period of time (for example, a half day once a week for a number of weeks). A greater proportion of teachers who had been on a nonconsecutive placement reported that impacts on pupils would be more positive (88%) than those on a block placement (73%) (Figure 5-12). From the small amount of survey feedback we received on this theme, it is possible that this may be due to the ‘increased opportunity for reflection’ that some teachers gained from their placement. Figure 5-12 Do you think there has been any impact on the pupils that you teach as a result of your placement?2 70% Consecutive days Over a period of time 60% 50% 40% 30% 20% 10% 0% No impact on pupils Very little impact on pupils Some impact on pupils A lot of impact on pupils Source: SQW Consulting (base: 188) 2 188 responses 37 Evaluation of Excellence in Education through Business Links (EEBL) Final report to Skills Development Scotland “[The teacher placement] was a good idea because we learned so much from her on etching. The teacher told us about the print rooms, the tours and materials/ objects there. Afterwards we got to see the Gallery through the web-conference link, and we sent many different paintings and etches on mythology. It helped the teacher to develop new ideas in class, for example we have started a project on “surrealism”. We now have more knowledge of careers in art and know more techniques and skills. It has helped us be more imaginative and made us realise the opportunities there are.” (Fourth year pupils, Lossiemouth High School) Summary of teachers’ views 5.33 The final question in our survey of teachers was “do you have any other comments to make about your experience of undertaking an EEBL teacher placement?” Of the 208 responses received, 99 (almost half) provided a response. Of these 99, 86 gave overwhelmingly positive feedback. 5.34 The following quotes are typical of this positive feedback: The help and advice I gained from EEBL was exceptional. I gained great feedback from EEBL staff and my work was then passed on for other EEBL events. I have been on three EEBL placements and they've all been fab. I would like to go on more but my PT is not in favour so I do not think I will get the opportunity, which is very sad. As well as CPD, I think the opportunity to network with other teachers is invaluable - as is the opportunity to get out into the "real world". As a relatively new entrant into the teaching profession (I worked extensively in industry before), I perhaps appreciate (and miss) the "real world" more than most. I found it a fantastic experience, I was able to experience how Sensation worked and feel that I was able to make a practical impact on the workshops my school would be involved in. It was a brilliant opportunity that I would recommend to teachers and businesses. Careers Scotland link person - really helpful. Placement - great opportunity professionally and personally. I found the placement stimulating and enjoyable and would recommend such activities to other colleagues. More than a year later, I am still full of enthusiasm for my placement at the RGBE - it really was one of the best experiences of my life - I feel so privileged to have had the opportunity to meet and work with world experts in their field and that they shared their time and expertise so willingly. I encouraged two colleagues to undertake the same placement this year and they are as full of praise as I am - we are all so grateful for this opportunity of a lifetime and will never forget it. Thank you thank you thank you! 38 Evaluation of Excellence in Education through Business Links (EEBL) Final report to Skills Development Scotland 6: Challenges Key points There appears to be a very low level of awareness of EEBL amongst teachers. Local authorities vary in their approach to ‘gatekeeping’: some encourage Careers Scotland to market EEBL directly to schools, while others do not. Headteachers also vary in their willingness to advertise EEBL placements to teachers. Take-up of EEBL has been adversely affected by the lack of funding for supply cover. Teachers often lack clarity about their aims and objectives, and need assistance from EEBL co-ordinators in articulating what they hope to gain from the placement. The pre-placement process is important to the programme’s success, but in practice it is difficult for teachers and host organisations to find time to meet each other prior to the placement. Although the great majority of teachers are happy with their EEBL placement, some are disappointed that they spend ‘too much time listening and not enough doing’. 6.1 The findings from our survey of teachers and our initial consultations with schools, local authorities and EEBL co-ordinators confirm that there is a wealth of positive outcomes to report from EEBL. However, it is worth exploring some of the areas where feedback has not been universally positive. Whoever takes over responsibility for EEBL will need to be aware of some of the challenges that have arisen, in order to deliver the programme effectively. 6.2 In this section we discuss some of the areas that warrant further attention. We have also included a variety of quotes from our e-survey of teachers in response to the question: “how do you think EEBL could be improved?” Marketing and awareness raising 6.3 Our survey and our consultations both provide evidence that many teachers have simply not heard about EEBL and do not know that the opportunity for placements exists. A range of quotes from our teacher survey illustrates the problem: Make sure staff know about it - get those who have been on one to promote it More promotion of the placements within schools as teachers can be busy and not have the time to check the websites on a regular basis More advertisement and active encouragement of uptake within schools. Many of my colleagues haven't heard of EEBL and are not aware of what it does and how they can contribute. 39 Evaluation of Excellence in Education through Business Links (EEBL) Final report to Skills Development Scotland Teachers should have the opportunity to hear about EEBL placements current and past. More often than not the information is not shared any further than the school, more focus should be on c.p.d and encouraging business links to last. A higher profile in schools - maybe a Careers Scotland rep raising its profile at a staff event/meeting. 6.4 Our consultations suggest that several EEBL co-ordinators are frustrated that their efforts to promote EEBL to teachers are not as successful as they would like. There appear to be marked differences between local authorities: in some areas, local authority officers are happy for Careers Scotland staff to market the programme directly to teachers (e.g. by attending inschool events, giving presentations, etc), whilst in other areas information is only allowed to be circulated by local authority staff. 6.5 The channel of communication between Careers Scotland and schools generally prevents EEBL co-ordinators from communicating directly with teachers when marketing EEBL. Where direct approaches are possible, for example at INSET days, the feedback is reported to have been positive. In other cases, information about EEBL has to be directed to the headteacher of the school, who may or may not choose to circulate the information. There is little that can be done if the headteacher does not encourage teachers to consider doing a placement. 6.6 Potentially, one of the advantages of transferring responsibility for EEBL from Careers Scotland to local authorities (should the takeover of responsibility for EEBL be agreed with local authorities) is that this hierarchy of information-sharing will be reduced. Those local authorities that are supporters of teacher placements may benefit from the simplified communications infrastructure. The risk is that where local authorities are less keen, who will promote EEBL? Clarity of purpose 6.7 As our consultations have underlined, it is vital that teachers and host organisations develop a very clear understanding during the planning stage of what they hope to achieve from a placement. Clarity of purpose is crucial to overall satisfaction and subsequent impact. 6.8 Two quotes from our survey of teachers illustrate this issue well: The placement I went on was valuable but what was expected of us seemed to change over the week. I'm all for development and rolling with the punches but perhaps EEBL could detail whether or not there is a specific expectation of the placement or if the outcome is open to change. EEBL was fantastic and very informative, however I felt that there was a large discrepancy between what Careers Scotland wanted for me out of the placement and the company, in which I was placed. This led to some initial awkwardness as they 'bashed' it out. This perhaps should have all been clearly agreed prior to offering placements. 6.9 These quotes give an indication that teachers’ role in defining their aims can sometimes be rather passive. Indeed, consultations with EEBL co-ordinators confirm that some teachers appear to expect Careers Scotland to define their purpose for them, rather than taking 40 Evaluation of Excellence in Education through Business Links (EEBL) Final report to Skills Development Scotland ownership of the placement and being clear in their own minds about what they want from the experience. 6.10 In this respect, it might be argued that individually tailored placements have the advantage over themed or national placements, as they originate directly from teachers themselves, who might thereby be expected to have greater clarity of objectives. In practice, however (as we saw in the previous section), there was no evidence that individually tailored placements had greater impact than other kinds. 6.11 As we noted earlier in this report, some EEBL co-ordinators have reported spending a significant amount of time helping teachers to develop and clearly articulate their aims for an EEBL placement. This will be an important issue to bear in mind if the programme transfers to local authorities. Moreover, exemplifying the relevance to Curriculum for Excellence may better help link placements to direct curriculum issues. Preparation and planning 6.12 In section 3 we considered the planning and preparation process involved in EEBL. Overall the feedback from teachers and host organisations is positive, with more than three quarters approving the process highly. 6.13 Nevertheless, our survey and consultations suggest that there is probably room for improvement, as illustrated by the following quotes from our teacher survey: There could be more opportunity to prepare for the placement before starting, as there is little time during the placement. Also, maybe a follow up half day could be useful? Perhaps a meeting with the organisation before hand to discuss aims and objectives. More information given beforehand as I didn't really have any idea what we were going to do/learn until we got there. 6.14 We know from our consultations with EEBL co-ordinators that they make every effort to ensure that teachers have the opportunity to take part in some form of pre-placement briefing session. However, there are many reasons why teachers may be unable to take advantage of this: not least because it can be so difficult to take additional time out of teaching hours. Despite the best efforts of Careers Scotland co-ordinators, therefore, there will always be teachers who do not receive a thorough briefing. 6.15 This highlights one of the major challenges for EEBL: it expects that teachers will be able to be released from teaching hours not just to undertake a placement but to prepare for the placement. In practice this can be extremely difficult to achieve. Communication and co-ordination 6.16 From our consultations with Careers Scotland staff we recognise that the majority of EEBL co-ordinators are passionate about what they do and are highly committed to making the programme work. In the great majority of placements, communication and co-ordination is exemplary. 41 Evaluation of Excellence in Education through Business Links (EEBL) Final report to Skills Development Scotland 6.17 However, our survey of teachers indicates that there are a small number of instances where communication and co-ordination do not happen as effectively as they might, as illustrated by the following examples: The process of actually getting to the placement from the initial application was lengthy and in my view inefficient and disorganised. The personnel dealing with each application were lax in their approach. In fact, it took 3 years from my initial application (had to complete a further form because they seemed 'to have lost it') to actually getting the placement. Actually getting to do the placement was entirely due to my own persistence and not at all from the staff employed by the project to do so. My estimation was that the officers employed to follow this programme through did their job badly. I sent one teacher on a course last year and she travelled to Loch Lomond to the venue only to be told it had been cancelled - poor communication. If a person is unable to attend the pre-placement meeting the necessary information should be forwarded on or the meeting rescheduled to ensure a full understanding of what will happen on the placement. 6.18 In some cases it may be the fault of the host organisation, rather than the EEBL co-ordinator, that communication breaks down. Regardless of where the problem lies, it underlines the importance of good communication in delivering a programme such as this. Content of placement 6.19 As we know, the great majority of teachers are happy with their placement, but there are instances where people have been disappointed. 6.20 From our consultations we are aware that it is crucial for teachers to do more than simply listen to presentations during their placement. Teachers and EEBL co-ordinators stress the importance of work shadowing or getting involved in some way in the day to day workings of a business. This does not always happen: The placement I attended was not "hands on". We were given presentations by managers of different departments. If I was given the opportunity to do this again I would choose a more "hands on" placement. 6.21 Also important is achieving the right balance between learning about an organisation and using this knowledge to develop resources for the organisation. We were struck by the number of cases where teachers’ time appeared to be used to develop materials, e.g. for children who visited the host organisation. In such cases it is possible the teacher is restricted in how much they learn about the workings of the business. The appropriate balance will vary between different types of placement, and sometimes teachers feel it has not been achieved: Maybe one day on site and one day to develop resources or actually organise trips would be better 6.22 Similarly, some teachers feel that the host organisation has been overly concerned with promoting itself rather than making the placement relevant to education: 42 Evaluation of Excellence in Education through Business Links (EEBL) Final report to Skills Development Scotland Ensure it is relevant to teaching and benefiting pupils for the future rather than a PR exercise for the businesses involved 6.23 The policy and practice framework for EEBL makes clear that the opportunity to promote itself is a legitimate benefit of EEBL, but in the context of recruiting young people (one of the benefits to businesses described in the framework is “raising the profile of and career opportunities within the business”). If there are instances where businesses are simply promoting themselves to teachers as potential consumers of their product or services, this must be actively discouraged (but we believe that only a very small number of such instances have occurred). Barriers to using EEBL experience in the classroom 6.24 Finally, a key barrier to the success of EEBL lies not in how the programme itself is managed and delivered, but in teachers’ ability to make use of what they have learned from their placement once they return to the classroom. Careers Scotland’s teacher impact assessment shows that this can be a problem (Figure 6-1) and we have highlighted related issues in section 5 above. Nearly one fifth of teachers feel that not having ‘enough time’ is a barrier to using their EEBL experience in the classroom. A significant minority also feel that EEBL’s contribution to their personal or professional development is greater than its relevance to the curriculum – in other words, that the benefit was primarily to themselves rather than to their teaching. Figure 6-1 Barriers to using EEBL in the classroom Not enough time Curriculum is organised too far in advance Difficult to gain support from colleagues Helpful to CPD - not directly relevant to curriculum 0 2 4 6 8 10 12 14 16 18 % of Respondents Source: SQW analysis of Careers Scotland EEBL data – base (212) Funding 6.25 When EEBL was established in 2006 it was funded by the (then) Scottish Executive in two ways: block funding to Careers Scotland to cover management and delivery costs; and funding to local authorities under the National Priorities Action Fund (NPAF) to cover or contribute towards teacher supply cover. A maximum of five days teacher supply cover per placement could be funded through NPAF. 6.26 With the advent of the concordat between the Scottish Government and COSLA in 2007, however, local authorities gained greater discretion over how their funding was used. Our consultations with EEBL co-ordinators and teachers highlight a consistent message: take-up of EEBL has been adversely affected over the last year by the lack of funding for supply cover. As concerns continue about local government finance next year this must also be a worry for the future scale of EEBL activity. 43 Evaluation of Excellence in Education through Business Links (EEBL) Final report to Skills Development Scotland 7: Conclusions and recommendations Conclusions 7.1 In this section we address the key questions specified in the evaluation brief. How does EEBL affect the individual perception of education / basic philosophy by teachers? 7.2 In Section 5 we explored some of the findings from Careers Scotland’s own survey of teachers. Of particular note is the large proportion of teachers who felt they had benefited particularly from the opportunity to exchange ideas and thoughts with others (Figure 5-2). The sharing of ideas is an important element in the development of their philosophy of education. 7.3 In our electronic survey of teachers we invited respondents to comment specifically on the contribution of EEBL to continuing professional development. The feedback strongly indicates that not only is EEBL an important addition to other forms of CPD, but those who have experienced it believe that it should be regarded as a more ‘normal’ part of school routine. Many teachers question why so few of their peers have undertaken a placement, and suggest that everyone should be encouraged to do it at least once. 7.4 A senior member of staff from a local authority mentioned during our consultations that many teachers become tired of the profession after a number of years, and that EEBL can provide an opportunity to refresh their approach and give them a ‘new lease of life’. Anecdotally he is aware of teachers who have been on the point of leaving the profession, but have changed their minds as a result of doing a placement. The anecdote speaks powerfully of the potential of EEBL to retain and revitalise the workforce. This aligns with our survey findings of teachers undertaking placements to refresh their approach to teaching. How does EEBL develop partnerships, and what do these partnerships look like? 7.5 As with many such initiatives, the success of EEBL depends largely upon effective partnerships. In addition to the partnership between Careers Scotland and the Scottish Government at strategic level, the programme requires strong operational partnerships at local level between EEBL co-ordinators, local authority officers (Determined to Succeed and/or workforce development teams) and schools. 7.6 In some instances local authority Determined to Succeed and CPD teams have acted as positive gatekeepers for EEBL, allowing and helping Careers Scotland to promote EEBL to schools (in other cases the opposite appears to have happened, and EEBL has not been proactively marketed). Where such partnerships have developed then they appear to be mutually beneficial, enabling Careers Scotland, DtS staff and teachers to meet their objectives. Indeed, it is where such relationships are less developed that take-up is reported to be most difficult. 44 Evaluation of Excellence in Education through Business Links (EEBL) Final report to Skills Development Scotland 7.7 A further set of enduring partnerships seems to have formed between teachers and their host organisations. Both sides generally report a willingness to maintain contact beyond the placement. Although this then takes place less widely than planned, we have little doubt that the subsequent employer engagement is beneficial. How do employers / host organisations benefit from EEBL (e.g. staff development)? 7.8 Generally speaking, the feedback from Careers Scotland’s survey of employers reflects the findings from our consultations: the great majority of host organisations have had a positive experience and regard EEBL highly. 7.9 Much of the most useful feedback has come from organisations whose remit includes working directly with children – notably museums, galleries and other tourist attractions. As can be seen from the employers’ comments in Section 4 of this report3, such host organisations have benefited from the chance to understand the relevance of their provision to the school curriculum and to make appropriate changes. Placements involving this kind of host organisation are particularly likely to be followed up by class visits, which allow the organisations to test new materials or approaches that have been developed as a result of EEBL. 7.10 Other benefits cited by employers include gaining pedagogical insight into how to improve in-house training materials for staff, and being able to present to teachers why they offer a potentially attractive workplace to school leavers. What works, what doesn’t work, and why? 7.11 Based on our survey and consultations, it seems clear that the requisite factors for a successful placement are: absolute clarity of aims and objectives on the part of teacher and host organisation sound pre-placement phase (i.e. thorough briefing and discussion between host and teacher prior to placement) in order to achieve this clarity hands-on, active learning / shadowing during placement (not just listening to presentations). 7.12 Themed and national placements have just as much impact as individual placements, and the non-consecutive approach can be just as effective as block placements. Longer placements appear to deliver more benefits, but at a higher cost. 7.13 There are two levels of ‘gatekeeping’ for EEBL: at local authority level, and at school level. Some local authority officers encourage Careers Scotland to market EEBL directly to schools, while others do not; likewise, not all headteachers pro-actively advertise EEBL opportunities within their school. Clearly, EEBL works best where each part of the chain actively promotes take-up. 3 Although we have anonymised the quotes from host organisations, there is a preponderance of feedback from museums, galleries and other tourist attractions 45 Evaluation of Excellence in Education through Business Links (EEBL) Final report to Skills Development Scotland What triggers interest in a teacher participating in EEBL? 7.14 Over one third of teachers say that their placement was their own idea, while a slightly lower proportion were prompted by Careers Scotland into thinking about it. A large minority of teachers undertake an EEBL placement as part of the Scottish Qualification for Headship programme. 7.15 Motivation to take part in EEBL often arises out of an issue for the wider school, notably a school’s wish to gain a particular status such as eco-school or forest school. Other motivations include updating professional knowledge and gaining a better understanding of the labour market, in order to provide pupils with relevant information. How much influence on participation comes from the individual and how much is prompted by the local authority? 7.16 There is a general problem with awareness of EEBL: very few teachers seem to know about it. Gatekeeping issues are at play here: not all local authorities have allowed Careers Scotland to promote EEBL to schools, and not all headteachers choose to circulate EEBL opportunities internally. 7.17 The influence on participation comes from a mix of influences: some interest is selfgenerated, some is prompted. Certainly, local authorities and headteachers who regard EEBL positively have encouraged teachers to apply for placements. More than one third of teachers say that the idea to do a placement was their own, but it has not been possible to identify the circumstances in which placements are more likely to be instigated by individual teachers. How effective are individual placements versus national / regional themed placements? 7.18 There is a presumption on the part of some EEBL co-ordinators and other stakeholders that one-off tailored placements initiated by individual teachers are ‘better’ than themed or national placements initiated by Careers Scotland. Our survey shows that, if anything, the opposite may be true: teachers on themed or national placements were more likely to report impacts on pupils than those on individual placements. The difference is not significant, but it is clear that both themed and national placements have an important role to play. 7.19 The sharing of ideas and knowledge is one of the key benefits that teachers report from EEBL, and it is clear that group placements facilitate this more easily than individual ones. Themed and national placements are useful in allowing teachers from different schools, and even different local authorities, to exchange thoughts during the course of a placement: the potential impact of this opportunity should not be under-estimated. 7.20 Our consultations also found evidence that the provision of a ‘menu’ of themed placements at key points in the school year is well received by many local authorities. In providing a menu of placement opportunities, some of the barriers to EEBL can be broken down: it is easier to market EEBL to a wide audience the offer of a specific placement may help to stimulate interest 46 Evaluation of Excellence in Education through Business Links (EEBL) Final report to Skills Development Scotland 7.21 ready-made placements mean teachers spend less time researching which employer might be able to host them teachers can choose the most suitably timed placement to fit with school calendar. In our consultations it became clear that some themed and national placements were oversubscribed, while others just managed to recruit the requisite number of teachers. Indeed, we were told of one placement opportunity that was full in one year, received positive feedback but then could not recruit teachers the following year. This may reflect resource issues as much as the quality of the offer. The closer placements can be linked to direct labour market opportunities or areas of the curriculum the higher take up would be expected to be. How does EEBL impact in the classroom, both in terms of attainment and achievement? 7.22 As we knew from the very start of the evaluation, this question cannot be easily answered. We would argue that it is unreasonable to expect pupils’ attainment levels to increase simply as a result of one of their teachers undertaking a two-day placement. However, EEBL can have an impact on pupils. As we described in Section 5 of this report, teachers themselves have some insight into the impact of their placement in the classroom, and this has to be taken as a proxy for the views of young people. The two most common impacts that emerged from our survey of teachers were: pupils’ increased ability to see links and connections in terms of subject specialism (seeing the link between cutting-edge science in the workplace and basic science learned in class) between school and employment (greater understanding of how a particular subject relates to a certain career, e.g. science in school and engineering as a job) broadening pupils’ horizons spending more time outdoors and being more aware of environmental issues understanding social issues such as sectarianism. How does EEBL deliver a Curriculum for Excellence? 7.23 Since its inception, the aspiration for the development of the Curriculum for Excellence has been to enable all children and young people to become successful learners, confident individuals, responsible citizens and effective contributors to society and at work. 7.24 The Policy and Practices Framework for EEBL (produced by the Determined to Succeed Division of the Scottish Government in 2006) summarises the characteristics that should be developed in children and young people as a result of the Curriculum for Excellence: we reproduce these in Table 7-1, below. Based on the evidence generated by our evaluation, we have highlighted in bold the characteristics where a positive impact can be seen as a direct result of EEBL. 47 Evaluation of Excellence in Education through Business Links (EEBL) Final report to Skills Development Scotland Table 7-1 Contribution of EEBL to Curriculum for Excellence (EEBL contribution in bold) SUCCESSFUL LEARNERS WHO: CONFIDENT INDIVIDUALS WHO: are skilled in generic processes and activities such as core skills have a ‘can do’, ‘be all you can be’ attitude appreciate the relevance of what they are learning are self-aware and can reflect on their own particular strengths, development needs, interests and aspirations can use experiences in enterprise to reflect on other learning and make connections are skilled in planning, influencing, negotiating and teamwork are self-motivating and accept setbacks as learning experiences are creative, flexible and resourceful in managing change can generate and act upon original ideas have experienced challenging, entrepreneurial projects understand transitions throughout life and the importance of making connections between past, present and future experiences have developed a constructively critical attitude towards commerce and business take imaginative and informed approaches to problem solving involving calculated risks RESPONSIBLE CITIZENS WHO: EFFECTIVE CONTRIBUTORS WHO: have knowledge and understanding of the nature of work and social and economic enterprise have an understanding and appreciation of the world of work, the value of different occupations and entrepreneurship and their contributions to the economy and to society understand the roles, rights and responsibilities of individuals as employees, managers, employers, entrepreneurs, investors, customers and global citizens are willing to take responsibility for and accept the consequences of their own actions understand the positive and negative contributions of technology and business to society have knowledge and understanding of personal finance, as well as national and global economies have knowledge and understanding of wealth creation and wealth distribution both nationally and globally are willing to take the initiative and lead actively engage in school and community life in a positive manner to the benefit of the school and the wider community take imaginative and informed approaches to understand the roles and contributions to society of voluntary organisations have experience of engaging positively with a wide range of people in society. 7.25 As can be seen, we believe that EEBL has contributed in some way to all four values that underpin the Curriculum for Excellence. Recommendations 7.26 Here we present a number of key learning points from the evaluation, which should be of use to local authorities if they take on responsibility for delivering EEBL. Awareness of EEBL is generally low amongst teachers. This is partly due to ‘gatekeeping’ issues amongst local authorities and individual headteachers: some local authority officers do not encourage Careers Scotland to market EEBL to schools, and some headteachers do not promote EEBL within school. On the other hand, dissemination by teachers who have taken part in EEBL is high: the vast majority have recommended EEBL to their peers and their headteachers. The transfer of responsibility for EEBL to local authorities would remove a layer (Careers Scotland) from the communication hierarchy, thus in principle making it easier to market the programme. Recommendation: EEBL should be marketed effectively to schools, so that all teachers know that placements are available as part of the CPD offer. 48 Evaluation of Excellence in Education through Business Links (EEBL) Final report to Skills Development Scotland The pre-placement stage of planning and preparation is crucial to the success of EEBL. In particular, it is vital that teachers and their host organisations are both able to identify and articulate their respective aims for the placement. In order to achieve this, face to face meetings prior to the placement should take place if possible; when this does not happen, participants may have a less successful experience during their placement. Recommendation: sufficient resource should be given to preplacement planning and preparation. Themed and national placements are just as valuable as individual placements. This is potentially good news to local authorities if they take over responsibility for EEBL, as such placements are less resource-intensive to organise. However, it has other implications: in particular, the ending of Careers Scotland involvement means that national placements will no longer continue to be organised, unless whoever takes over responsibility for EEBL is able to co-ordinate such placements through a national network. Likewise, themed placements that have been operating across local authority boundaries (e.g. BBC Learning) may also be lost, or run the risk of duplication of contact as individual local authorities pursue their own employer leads. In this environment, if responsibility for EEBL is handed to local authorities it would make sense for groups of local authorities to employ a joint co-ordinator to work across boundaries. Recommendation: resource should continue to be given to coordinating placements across local authority boundaries, if necessary by sharing resources to employ joint co-ordinators. The points above indicate a need for significant management and co-ordination resource if EEBL is to operate effectively and at the same scale as in previous years. It is important that whoever takes over responsibility for EEBL recognises the role played by Careers Scotland staff and identify how this gap will be filled if responsibilities are transferred. Recommendation: the new providers should learn as much as possible about the role of Careers Scotland in managing the programme up till now, and make arrangements to resource a version of that role within the new delivery structures. Placements should be designed to be as hands-on as possible. Teachers benefit most where they are able to (sometimes literally) get their hands dirty. Work shadowing is much more useful than listening to presentations, and has more impact. Agreeing exactly what will happen during a placement is part of the planning process. Recommendation: the new providers should encourage host organisations to make sure that placements allow as much active learning / work shadowing as possible. Placements should have equal benefits to both parties. We are aware of a few instances where host organisations use teachers to produce materials that benefit the organisation, and where the benefit to teachers is not as clear-cut. Again, the benefits to both sides should be clearly articulated at the pre-placement stage. Recommendation: the new providers should liaise closely with teachers and host organisations to make sure that placements are mutually beneficial, with aims 49 Evaluation of Excellence in Education through Business Links (EEBL) Final report to Skills Development Scotland and objectives being clearly defined on both sides during the pre-placement phase. Longer placements (5 days) have more impact on teachers and pupils than short placements (2 days). This has obvious resource implications, and it is likely that a balance will need to be struck. Recommendation: where possible, and providing clear aims and objectives have been set by both parties, resourcing of 4- and 5day placements should continue alongside shorter placements. Some stakeholders assume that block placements (taking place over consecutive days) are more effective than non-consecutive placements (whereby a teacher attends for half-days over a period of time). This is not necessarily the case: there is some evidence that teachers and hosts benefit from non-consecutive placements, which can give more time for reflection. Recommendation: the benefits of non-consecutive placements should be investigated in more detail, with a view to promoting them if it becomes clear that they bring benefits. EEBL contributes directly to all four elements of Curriculum for Excellence and also to a wider range of outcomes. It has significant potential to develop and consolidate the links between schools and employers. Recommendation: EEBL should be regarded as integral to, not separate from, school-employer links; in particular, it has an important role in generating work experience opportunities for pupils. Dissemination of learning from EEBL has not been consistent, and could be improved. Recommendation: all teachers benefiting from EEBL should be expected to disseminate key learning points to their colleagues at school level, and Glow should be used to promote EEBL internally and externally. 50 Evaluation of Excellence in Education through Business Links (EEBL) Final report to Skills Development Scotland Annex A: Participation by local authority area Participation by council area Figure A-1 West 70 Total Participants 60 139 50 40 30 60 41 47 20 40 31 10 0 East Dunbartonshire East Renfrewshire Glasgow City Inverclyde Renfrewshire West Dunbartonshire Council Area April 06 - March 07 April 07 - March 08 April 08 - Dec 08 Source: SQW analysis of Careers Scotland EEBL data Figure A-2 South East 35 74 Total Participants 30 25 50 20 48 40 15 30 10 18 14 9 5 ot hi an g W es tL Sc ot ti sh id lo M St irl in Bo rd er s n th ia irk Fa lk n Ea st Lo th ia sh ire ck m an na n C la C ity of Ed in bu rg h 0 Council Area April 06 - March 07 April 07 - March 08 April 08 - Dec 08 Source: SQW analysis of Careers Scotland EEBL data A-1 Evaluation of Excellence in Education through Business Links (EEBL) Final report to Skills Development Scotland Figure A-3 North East 45 105 40 Total Participants 35 30 25 46 55 20 15 26 27 19 10 5 0 Aberdeen City Aberdeenshire Angus Dundee City Fife Perth & Kinross Council Area April 06 - March 07 April 07 - March 08 April 08 - Dec 08 Source: SQW analysis of Careers Scotland EEBL data Figure A-4 South West 35 69 Total Participants 30 25 59 48 20 31 26 15 25 10 5 0 Dumfries & Galloway East Ayrshire North Ayrshire North Lanarkshire South Aryshire South Lanarkshire Council Area April 06 - March 07 April 07 - March 08 April 08 - Dec 08 Source: SQW analysis of Careers Scotland EEBL data A-2 Evaluation of Excellence in Education through Business Links (EEBL) Final report to Skills Development Scotland Figure A-5 Highlands & Islands 70 134 Total Participants 60 50 40 30 20 21 17 37 28 18 10 0 Argyll & Bute Eilean Siar Highland Moray Orkney Islands Shetland Council Area April 06 - March 07 April 07 - March 08 April 08 - Dec 08 Source: SQW analysis of Careers Scotland EEBL data A-3 Evaluation of Excellence in Education through Business Links (EEBL) Final report to Skills Development Scotland Annex B: List of consultees Teachers Alison Roberts, Gavinburn Primary School Angela Buchanan, Calderhead High School Bennie Rafferty, Selkirk High School Brian Chalmers, Lossiemouth High School Carol McGurk, Whitburn Academy Debbie Raymont, Moray primary schools Fiona Shearer, Lossiemouth High School Gillian McNulty, Dalkeith High School Hannan al-Bassam, Royal Mile Primary School Marion Campbell, Merkinch Primary School Rachel Fagan, Selkirk High School Sharon Robertson, Crieff High School Sheena Forsyth, Dingwall Academy Susan Crooks, Selkirk High School Headteachers and Deputy Headteachers Brenda Gifford, Lossiemouth High School Paul Samson, Selkirk High School Careers Scotland (SDS) Staff Katy Gordon (Careers Scotland Manager, National Operations Team) Ron Mooney (EEBL Programme Co-ordinator) Anne Lawrie (Local Co-ordinator - South East) Laura Ferguson (Local Co-ordinator - South East) Irene Clarkson (Local Co-ordinator - South West) Margaret Houston (Team Leader - West) Marilyn Brown (Local Co-ordinator - West) B-1 Evaluation of Excellence in Education through Business Links (EEBL) Final report to Skills Development Scotland Claire Reekie (Local Co-ordinator - North East) Ian Carse (Head of Careers Services - Highlands & Islands) Rob Tyson (Local Co-ordinator - Highlands & Islands) Local authority Officers Caroline Hill (City of Edinburgh Council) George Mackenzie (Moray Council) Janyce Graham (Staff Development Officer, Renfrewshire Council) Robert Lamb (Inverclyde Council) Tony Bragg (Falkirk Council) Other Jean MacMillan (Scottish Government) David Cameron (Head of Children’s Services, Stirling Council) Colette Filippi (Independent Consultant) Michael Roy (Critical Mass Consulting Ltd) Host Organisations Alison Motion, Royal Highland Education Trust Iona Finlayson, Scottish Natural Heritage Joanna Mawdsley, National Galleries Scotland Katy Boal, Inverness Museum and Art Gallery Vicky Connor, Hilton Edinburgh Grosvenor Hotel B-2 Evaluation of Excellence in Education through Business Links (EEBL) Final report to Skills Development Scotland Annex C: Questionnaire used in on-line survey of teachers We have been given your name by Careers Scotland, who have commissioned us to evaluate Excellence in Education through Business Links (EEBL), the teacher placement programme. We would be very grateful if you would take a few minutes to answer some questions about your experience of EEBL. C.1 Please tell us which local authority you work for C.2 Are you a primary or secondary teacher? C.3 Can you remember what year you did your placement? (Note: if you have done more than one placement, please answer these questions with reference to the most recent one that you did.) C.4 C.5 C.6 2006-07 2007-08 2008-09 How long was your placement (tick one)? 1 day 2 days 3 days 4 days 5 days Other (please state how long) How was it structured (tick one)? Consecutive days Piecemeal over a period of time (e.g. half day one week, half day another week) Whose idea was it for you to do a placement (tick one)? It was my own idea (i.e. I actively sought the opportunity to do a placement) The idea came from the school The idea came from Careers Scotland Other (please describe) C-1 Evaluation of Excellence in Education through Business Links (EEBL) Final report to Skills Development Scotland C.7 What triggered your interest in doing a placement? (open question) C.8 What kind of placement was it (tick one)? I was the only participant; the placement was tailored to my individual needs I was one of several teachers who took part; the placement was generic to our needs C.9 What was your reason for doing the placement, i.e. what did you want to achieve from it? C.10 Did you achieve your aims (tick one)? yes no in part C.11 Did any tangible outputs arise directly from your placement (e.g. new resources or materials to use in class)? C.12 Which of the following benefits did you expect to achieve from your placement, and were these expectations met (tick all that apply)? Gain new insight into how an organisation works Gain additional motivation as a teacher Gain new confidence as a teacher Gain awareness of career opportunities for pupils Freshen my approach to teaching Produce new lesson plan/s Produce new teaching resources Make useful new contact/s New school-business link established for the longer term Class able to visit the organisation after my placement Pupils subsequently invited to do work experience at the organisation Other (please describe) C.13 For those that you did expect to achieve, how far was your expectation met, on a scale of 1 to 5 (where 1 is 'not at all', and 5 is 'met in full'). C.14 Do you think there has been any impact on the pupils that you teach as a result of your placement (tick one)? No impact on pupils C-2 Evaluation of Excellence in Education through Business Links (EEBL) Final report to Skills Development Scotland Very little impact on pupils Some impact on pupils (please describe) A lot of impact on pupils (please describe) C.15 Please explain your answer. C.16 Have there been any opportunities to disseminate the learning that you gained from your placement? (tick all that apply) C.17 C.18 C.19 Informal sharing of information with other teachers in my school Formal / structured sharing of information with other teachers (e.g. at staff meeting) Sharing of new resources / materials with other teachers Feedback given to headteacher and/or other senior colleagues Other (please describe) Was your headteacher generally supportive of you undertaking your placement? Yes No (if not, why not) Do you think that teachers will be more, or less, likely to undertake this kind of placement once responsibility for EEBL transfers away from Careers Scotland and into local authorities? (tick one) more likely (please explain your answer) less likely (please explain your answer) no difference How do you think EEBL placements compare to other kinds of staff development / CPD activities? (tick all that apply) Placements are a useful addition to other forms of CPD Placements are not as useful as other forms of CPD Other (please describe) C.20 How do you think EEBL could be improved? [open question] C.21 Please use this space to give us any other comments about your experience of undertaking an EEBL teacher placement …. C-3