Graphic Science Smarter UK Final Project Report November 2013 Introduction The Smarter UK project was designed to engage young people (aged 14-18) with the ethics of using drugs for cognitive enhancement. Smarter UK sessions were delivered in UK secondary schools between December 2011 and March 2013. The project was funded by a Wellcome Trust Society award and was a scaleup of Graphic Science’s earlier Smarter project which ran in Bristol and Glasgow in 2008 and was funded through a Wellcome Trust People Award. Sessions began with an eight-minute sketch which set the scene by highlighting all the major issues, followed by a short memory game to illustrate different types of memory. This was followed by a Vote with your Feet discussion based around five pre-set questions. Each session was delivered by two professional science presenters and supported by a neuroscience researcher who was available to share their expertise throughout the discussion and in a dedicated question and answer slot at the end of the session. Region Bristol and Bath Cambridge Glasgow and Edinburgh Oxford Wales Delivery Partner Explorer Dome Naked Scientists Glasgow Science Centre Science Oxford science made simple Neuroscience research organisation University of Bristol University of Cambridge Universities of Glasgow and Edinburgh University of Oxford University of Cardiff Neuroscientists were recruited by Graphic Science via each university’s neuroscience coordinator. They attended a one-day training from Graphic Science at their host institution in which they were also inducted as STEM Ambassadors. Delivery partners were trained in how to administrate and run Smarter UK in a two-day residential training attended by 2-3 presenters from each organisation. An external evaluation of Smarter UK was carried out by David Shakespeare/Square2Learning. The Smarter UK script was developed by playwright Toby Hulse. Project Objectives The project objectives for Smarter UK as stated in the bid to the Wellcome Trust were: To provide a platform to engage and stimulate discussion amongst 13-16 year olds (KS3 & 4 in England, S2-S4 in Scotland) with both the personal and societal implications of contemporary advances in Neuropharmacology. To improve teacher confidence in running debates in the classroom as expected by contemporary science curricula. To connect teachers and young people with real examples of cutting edge neuroscience research. To work with teachers to produce curriculum enhancing resources to support the teaching of neuroscience and the social implications of advances in science and technology. To train neuroscience researchers in public engagement and to equip them with the skills, confidence and opportunity to discuss their research in schools. 1 (Resulting in an embedded culture of engagement with schools into the practices of participating neuroscientists). To create a sustainable project which can continue through integration into the regular programmes offered by our delivery partners Delivery statistics Explorer Dome Glasgow Science Centre Naked Scientists science made simple Science Oxford Number Number of Total Sessions per Students of schools sessions students school per session 20 61 1404 3.05 23 Totals Target 21 21 60 61 1379 1699 2.86 2.9 23 28 18 21 55 64 1298 1641 3.06 3.05 24 26 101 100 301 300 7421 9000 2.982 3 24.8 30 One of the project’s targets was to try to reach schools that did not engage readily with STEM enhancement and enrichment activities. Lists of schools were provided by the regional STEMNET STEM Ambassadors contract holders for each area where Smarter UK delivery was taking place. The target was for 50% of all schools that received Smarter UK to be from these lists of hard to reach schools. Success in reaching these hard to reach schools varied from delivery partner to delivery partner. Hard to reach schools Explorer Dome Science Made Simple Naked Scientists Science Oxford Glasgow Science Centre % hard to reach schools 55 39 9.5 57 62 Comments on delivery targets The delivery targets set out at the bidding stage were largely met and in some cases, marginally exceeded. This did vary a little from delivery partner to delivery partner – some reflections on the reasons for this are discussed later in this document. We overestimated numbers of students in each class so although we delivered the full number of proposed sessions, the project only reached 82% of our target in terms of student numbers. Findings from the Smarter UK external evaluation Below is an extract from the executive summary of the external evaluation: The evaluation of this project aimed to secure evidence from simple ‘reaction level’ responses through to longer term changed behaviours and outcomes – and used direct observation and responses from the young people, researchers and teachers involved; as well as a number of on the- spot questionnaires and longer 2 term self-assessment diaries. Whilst a suitable amount of appropriate evidence was collected, the possible limitations of this evaluation process were also considered – which include consideration of degree of representation of the sample. The evidence is presented and considered for each of the project objectives, and concludes that the project has: engaged young people with a relatively new and controversial topic provided a platform for topics which would simply not have been explained or discussed under normal circumstances used a format (involving drama, ‘vote with your feet’ continuum and discussion, contact with and input from with a real-life scientist) that stimulates dialogue in schools very effectively enthused teachers to try similar ideas themselves - although other priorities, lack of time in the curriculum and lack of knowledge of neuroscience make this difficult enthused neuroscience researchers in terms of interest and confidence in public engagement and to engage more with schools A number of practical, shorter term recommendations are made which involve the way the sessions have been set up and run. Longer term recommendations are also made around the sustainability of the project if it is to continue – suggesting that either the project continues in the same way, but scales down its ambitions for influencing teachers, and teaching and learning around neuroscience and cognitive enhancement issues, in the longer term or the project could now focus having a greater impact on those involved by encouraging longer term engagement between individual teachers (and their schools), and the delivery partners and researchers. The suggestion is that a teacher, delivery partners and researcher co-plan, run and review Smarter UK sessions together in a school. In practice this may mean more intense targeting of efforts – engaging a smaller number of schools so as to have more impact on the young people, teachers and researchers involved in each. Training the Delivery Partners Summary: Delivery partners expressed universally favourable opinions about the training. They thought it was a great opportunity to familiarise themselves with the sessions, to team-build and to network. They particularly appreciated having a session delivered by a scientific expert. Smarter UK delivery partners attended a two-day residential training from 4-5 October 2011 at the Folly Farm Centre, Pensford, Somerset. The training was run by Graphic Science and led by Toby Hulse. The purpose of the training was to familiarise the presenters with the Smarter UK session format, and the script in particular; to give them factual background information that would assist them in facilitating the debate, and to understand the administration and evaluation requirements for the project. In addition to Toby and Graphic Science, a session on the project’s evaluation was delivered by David Shakespeare and a session on cognitive enhancement from a neuroscientist’s perspective was given by Professor Graeme Henderson from the University of Bristol. 3 Once Smarter UK delivery was underway, additional presenters were trained by the delivery partner organisations as the need arose (e.g. in cases where trained presenters moved to different roles within or outside the organisation). Feedback on the training The delivery partners all gave very positive feedback on the training. In particular they commented that they liked: The opportunity to network and team-build with colleagues from their own and other organisations Toby’s sessions on performance The high-level information in the session by Grahame Henderson which proved to be really valuable in the Smarter Sessions Gaining a thorough understanding of the whole project and their role Suggestions for future considerations were: To set some time aside (30 minutes) that would allow each of the delivery partners to work out the details of how they would run the project in their own organisation. Having advance access to the script and putting a greater focus on how to interpret the characters in the drama Holding the training earlier to enable booking to synchronise better with the pattern of school year – the autumn term is particularly busy for schools bookings, but the training was not early enough to allow delivery partners to make Smarter UK bookings. The timing of training sessions is complicated slightly by the later start of the university academic year compared to the school year, and the impact this has on organising and recruiting for neuroscientists’ training. It is likely that there would be a significant lag between training delivery partners and training researchers. Delivery partners would be put in a situation where they would be booking sessions into schools before the cohort of neuroscientists was trained to participate. This seems like a risky approach for all concerned. Working with neuroscientists Summary: The neuroscientists were considered by delivery partners, teachers and students, to be the key attraction of the project. Neuroscientists reported increased confidence as a result of their participation. Some would have liked greater opportunities to input during the sessions. Neuroscientists were recruited via the local neuroscience coordinator from each of the participating research organisations (see appendix 2). Recruitment was primarily aimed at early career researchers but small numbers of Masters Students and more established academics were also involved. Graphic Science trained 62 neuroscientists in total and at the end of the project, 45 of these neuroscientists (72.5%) had taken part in one or more Smarter UK sessions. In addition to the neuroscientists trained by Graphic Science, delivery partners had called in 4 the help of 11 other neuroscientists from their own networks. In total 56 neuroscientists were involved in Smarter UK delivery. Delivery Partner Explorer Dome science made simple Naked Scientists Science Oxford Glasgow Science Centre Totals No NS Trained by GS No Trained NS who delivered No Non-GS trained who delivered Total NS in delivery Days delivered by GS trained Neuro 17 9 1 10 19 13 12 2 14 15 Days delivered by non-gs trained NS 1 (ex ED employee turned researcher) 4 0 (but Hannah No days without NS Total Days of delivery 0 20 1 18 0 21 10 10 0 10 18 11 8 8 16 10 9 2 21 11 6 0 6 19 0 2 21 62 45 11 56 81 4 5 101 was NS for 3) The number of different neuroscientists involved in Smarter UK delivery ranged from 16 for Science Oxford to 6 for Glasgow Science Centre. Delivery Partner Explorer Dome science made simple Naked Scientists Science Oxford Glasgow Science Centre Percentage of trained NS who delivered % of NS who did >1 sessions Avg days/NS session 52.94% 2 50.00% 92.31% 1.3 28.57% 100.00% 2.1 60.00% 72.73% 1.3 31.25% 54.55% 3.5 100.00% There was some variability in the experience of working with neuroscientists for different delivery partners. This seemed to be a reflection of three main factors: The time delay between training and delivery of the sessions – Bristol and Wales both suffered a little from this. By the time the later Smarter sessions were arranged, the neuroscientists were busy with other priorities and much less available The neuroscientists trained in the first place – the Oxford cohort were largely clinicians, and this group did not seem to buy into the project. The enthusiasm of particular neuroscientists - Glasgow Science Centre in particular worked closely with a small group of very keen neuroscientists; the high level of buy-in to the project from Cambridge neuroscientists is a reflection of the close relationship between the Naked Scientists’ Hannah Critchlow who is a recent exCambridge neuroscience researcher and her colleagues. The percentage of neuroscientists who took part in more than one session for each delivery partner is an accurate reflection of how easy each organisation found it to book neuroscientists (a lower proportion of repeat bookings correlates with greater difficulty in securing neuroscientists for Smarter UK sessions). 5 On a few occasions, neuroscientists dropped out at short notice and the sessions had to go ahead without them. When this happened, the presenters chose one of themselves to act as the “expert”. All delivery partners agreed that involving a neuroscientist in the sessions was a central strength of the project as they provided students with a rare and inspiring opportunity to talk to a real scientist and brought authority and a depth of information to the sessions. This was also reflected in the feedback from students and teachers in the external evaluation. Delivery partners were reluctant to run sessions without a neuroscientist. However, sessions could be run effectively without a neuroscientist if absolutely necessary. The offer of honoraria, although not essential, and not always claimed by neuroscientists, seemed to help when securing the researchers’ involvement in delivery of sessions. Neuroscientists who took part in Smarter UK sessions reported increased confidence and enthusiasm in working with schools and in communicating their research more generally. In general feedback from those who took part in sessions was very positive. Considerations/recommendations when working with researchers in future Training should be as close as possible to delivery. Where delivery takes place over more than one academic year, additional recruitment and training sessions should be considered for each year PhD students and postdocs are much better placed than Masters Students to make time available and are much more confident about their own expertise Payment of honoraria makes it much easier to secure researcher involvement in school sessions Neuroscientists had some concerns about being referred to as “experts” when cognitive enhancement was not their subject of expertise. This needs to be accommodated better – perhaps by giving greater guidance to researchers during training and in briefing information about what they need to know and how to link their own expertise to the subject. It could also involve more time for them to speak directly about their own work e.g. by giving more time to the “Ask the neuroscientist” section. It would be useful for delivery partners to have more involvement in the researcher training day. This would give them greater insight into the qualities and abilities of different researchers and mean that they were better informed about people they would be working with before starting to deliver sessions. The budget implications of this would need to be considered in any future bids. Feedback and learning from schools Summary: Feedback from students and teachers about the Smarter UK sessions was predominantly positive. Over 80% of students who took part in the evaluation reported that the session was interesting and enjoyable. Over 90% reported that they had learned from the neuroscientist and over 80% that they had learned from the presenters. For teachers, the involvement of a neuroscience researcher was a key attraction. 6 Most sessions were delivered during a single period lasting between 45 to 60 minutes. A the lower end of the time range, fitting the full session in was a tight squeeze. Where content had to be cut, presenters generally decided not to run the memory game exercise. Presenters felt that the “Ask the neuroscientist” section was particularly valuable and tried to ensure that there was enough time at the end to make this worthwhile. Feedback from some teachers and students suggested that the Vote with Your Feet section went on for a bit too long. Given the perceived value of the researcher Q&A session, it would be worth trimming the Vote with Your Feet activity to allow a good 10-15 minutes for Q&A. Most delivery partners also offered a longer, double-period version of the session. Uptake for this was fairly low. However, it did allow more breathing space for the main session. One delivery partner estimated that the optimal running time for the main Smarter session was 1 hour 10 minutes. Where it was used, the advertising exercise worked well. However, this was much more of a usual school-lesson feature than other elements of the Smarter UK sessions. Some teachers stayed and got involved in the sessions, while others tended to keep a low profile. Presenters felt that sessions went better when the teacher was actively involved. Delivery partners estimated that around half of teachers took an active part in sessions and half did not. They did not consider it realistic to require teachers to be involved in sessions. Some teachers did pick up ideas about how to run discussions and debates in lessons. However, where this happened, teachers tended to view this as incidental rather than as intended outcome of the sessions. Often, teachers did not consider that Smarter UK could offer them a CPD opportunity. Furthermore, a number of respondents suggested that they and their colleagues would need further training to feel confident in running similar discussion activities. The teacher CPD objective of this project was the least successful and may simply have been an unrealistic aim. Project administration Sessions should be offered free, but an alternative budget model should be provided whereby delivery partners can charge a nominal fee to schools if they can make a good case that this will improve the success of the project. Consideration should be given to administrative overheads when agreeing the budget breakdown with delivery partners. Delivery partners would have liked more consideration of administration overheads to be incorporated into the budget and MOU. This would not necessarily require an increase in overall money available, but the time cost of booking sessions in schools and coordinating with a neuroscientist should be made explicit in the breakdown. The involvement of a neuroscientist, although noted by all delivery partners as a key component of the project’s success, increased the overall administrative load for the project considerably. Delivery partners estimated that administration took up to half a day per school. The challenges of booking sessions in schools varied from region to region. The difficulty of 7 securing bookings in schools that had been identified as hard to reach also varied. In some areas, it posed no additional problem, in others it proved to be very difficult. In general, bookings to schools were neither more, nor less difficult than anticipated based on experience with other projects. Most delivery partners felt that offering the sessions free of charge to schools was a good approach and some suggested that booking the sessions would have been much harder if schools had to pay. Offering free sessions also had some disadvantages – on occasion, schools were less wellprepared or the classes that received the sessions were different from those it had originally been booked for which meant that presenters could not plan appropriately. In Wales, free sessions seemed to be viewed less favourably than paid-for sessions resulting in poor organisation and last minute cancellations from some schools or disinterest from teachers that was transmitted to students and reflected in their behaviour. Science made simple have adopted a general policy of charging a nominal fee for all shows to offset this. However, they did not charge schools for Smarter UK. The experiment to use Twitter as a mechanism for sharing and monitoring experience for each session was not successful. Some presenters tweeted reflections while others did not, even with additional prompting from Graphic Science. It did not feel like a natural process for presenters unless they were already active Twitter users. Those who were active on Twitter felt that it was hard to share anything that offered genuinely meaningful learning to other delivery partners compared to what they would have learned through delivering sessions themselves. Delivery partners took part in one Skype conversation during the project. This was generally felt to be a useful activity. Delivery partners were keen to hear about each other’s experiences with the project and share learning. It was suggested that more frequent Skype sessions with a clear agenda and time-limit would have been useful. However delivery partners also made direct contact with each other from time to time where they needed specific support. This peer support may be worth cultivating in any future projects that use a similar model of delivery. Feedback on delivery It was perceived across delivery partners that Smarter sessions seemed to work particularly well with lower ability groups. This was not something that was considered when planning the evaluation, so no data was collected that could verify this insight. It may be something worth considering for any future projects in a similar format. Delivery partners enjoyed the sketch and felt it was something unique they could offer that the schools would not do themselves. They felt that the sketch worked well as an icebreaker, that students appreciated that the presenters were putting themselves on the line, and that this helped to create an atmosphere of open discussion later in the session. 8 Students were generally well-behaved and attentive during the sketch. Occasional groans or misunderstandings created amusement and added to the experience. On a few occasions, the teacher did have to intervene to regain order, but this was rare. Delivery partners used their own expertise to decide how best to facilitate Vote with Your Feet. Some adopted the “think, pair, share” technique (which was also suggested by David Shakespeare). Where the discussion was slow to get going, or students were reluctant to share their views, some presenters split the class into three groups, each of which was encouraged to explore and advocate for a different view, or facilitated discussion in each of the three groups that had naturally formed before inviting the groups to share views with each other. There was a general perception from presenters that boys tended to be more pro smart drugs than girls (this was particularly evident in single-sex classes). It was also observed that lower ability groups tended to be more in favour of the drugs, but only for themselves and not if everybody took them. There was also a sense that the Smarter sessions as a whole worked as well, if not better for lower ability groups who seemed to be more forthcoming with their views and generally more animated in the discussions. Delivery partners observed that less able and more unruly students engaged well and seemed to feel empowered by the idea that developmental changes in their own brains could be contributing to some of their teenage difficulties. Occasionally, the participating neuroscientist brought along a model brain or other visual prop. This was a really useful additional stimulus for Vote with your Feet. Students were generally very open about sharing their experiences of themselves, friends or family members taking medication for ADHD, epilepsy etc. and this made the sessions more relevant to them. Delivery partners based at science centres felt that Smarter UK offered them a useful professional development opportunity for working more with secondary schools. Legacy For some of the delivery partners, involvement in Smarter has resulted in closer working relationships with their local neuroscience coordinators and specific neuroscientists and some have entered into discussion about how to use this or develop other projects. Schools seemed to be particularly interested in the relationship between adolescent brain development and learning and how this could be applied to revision. One delivery partner was looking into the potential of using this as a starting point for a new version of Smarter or another related project. Using two presenters plus a paid neuroscientist limits the possibility of ongoing delivery of Smarter UK sessions on a commercial basis beyond the funded period. Most delivery partners run shows with just one presenter, and/or to a much larger audience for each session. The setup with Smarter is expensive for schools and delivery partners are not necessarily in a position to pay the neuroscientist for their contribution in addition to these costs. 9 It has been suggested that delivery partners might train the neuroscientists to be the second presenter to keep costs down. However current experience suggests that interest from schools for paid-for bookings will be relatively low. Collecting students views on the ethics of using drugs for cognitive enhancement During the Vote with your Feet section, students were asked to consider the following questions: If there were pills that could make you smarter, would you take them? If there were pills that could make you smarter, should everyone be taking them?" Should school age students be allowed to take these pills? Is taking medication to enhance your brain power any different from drinking coffee or red bull, taking vitamins or going for a run? These questions were primarily intended to stimulate open student-led discussion. However, they also created the possibility of collecting data about a large number of young people’s opinions as informed by the discussion. In order to capture students’ views on using drugs for cognitive enhancement, Graphic Science produced sets of likert scale colour-coded stickers marked “yes” (dark green), “probably yes” (pale green), “not sure” (yellow), “probably no” (orange) and “no” (red) and prepared sheets with each question and spaces to put the stickers. At the end of each question in the discussion, students were asked to apply the sticker that most reflected their final view on the response sheet for that question. Opinions were collected from over 6000 students for each question. Delivery partners reported that students really appreciated the idea that their views were being taken into account in some way and enjoyed using the stickers to collect data. It should be noted that despite the high sample number, these results should be viewed with a degree of caution. The primary purpose of the sessions was to create a forum for open discussion, rather than to collect data and different groups will have taken the conversation in different directions. These will have been influenced by the presenters, neuroscientist and peer pressures. The following charts show the final outcomes for each of the questions for all students from whom data was collected. Unfortunately, due to the data collection method, more detailed analysis based on factors such as gender and academic performance is not possible. 10 If there were pills that could make you smarter, would you take them? If there were pills that could make you smarter, should everyone be taking them? 10% 18% Yes 29% 7% Probably Yes Probably Yes Not Sure 18% 14% 52% 17% No 16% Should school age students be allowed to take these pills? 11% 17% 14% Not Sure Probably No No Yes 15% Probably Yes 40% No Is taking medication to enhance your brain power any different from drinking coffee or red bull, taking vitamins or going for a run? Yes 18% Not Sure Probably No Probably No 19% Yes 40% 9% Probably Yes Not Sure Probably No 20% 16% No Appendix 1: Delivery partners Bristol: Explorer Dome Ben Brown or Shaaron Levermont Explorer Dome Mailbox 42 179 Whiteladies Road Bristol BS8 2AG Tel: 0117 914 1526 Email: ben@explorerdome.co.uk or shaar on@explorerdome.co.uk Cambridge: The Naked Scientists Dr Hannah Critchlow Naked Neuroscience Naked Scientists Department of Pathology Cambridge University Tennis Court Road Cambridge CB2 1QP Tel: 07846 168684 Email: hmc39@cam.ac.uk Cardiff: science made simple Becky Holmes School of Physics and Astronomy Cardiff University 11 14-17 The Parade Cardiff CF24 3AA Email: susan.meikleham@glasgowscience centre.org Tel: 0141 420 5010 ext 234 Email: becky@sciencemadesimple.co.uk Tel: 02920 876 884 Oxford: Science Oxford Edinburgh: Glasgow Science Centre Susan Meikleham Glasgow Science Centre 50 Pacific Quay Glasgow G51 1EA Bridget Holligan Science Oxford Live 1-5 London Place Oxford OX4 1BD Email: Bridget.Holligan@scienceoxford.co m Tel: 01865 810020 Appendix 2: Neuroscience coordinators Anne Cooke – Bristol (no-longer in post) Vanessa Davies - Cardiff Martin Edmunds – Oxford Dervila Glynn – Cambridge Jane Hayley – Edinburgh Appendix 3: Advisory panel members, meetings and comments Development of the Smarter UK sessions and resources has been overseen by an advisory panel comprised of: Professor Barbara Sahakian, University of Cambridge (Chair) Dr Anne Cooke, University of Bristol Dr Hannah Critchlow, University of Cambridge Nick Hillier, Academy of Medical Sciences Bridget Holligan, Science Oxford Katherine Mathieson, British Science Association Jon Walford, Chew Valley School Appendix 4: Additional Smarter UK sessions In addition to the Smarter UK sessions paid-for from the project grant, Glasgow Science Centre ran four sessions paid for by the receiving schools. These were as follows: November 2nd – Balfron High – Neuroscientist – Callum 9th- Balfron High – Neuroscientist – Nneka 16th- Balfron High – Neuroscientist – Alasdair 12 December 3rd – Paisley Grammar School – Neuroscientist- Kieren November 2013 update: The Naked Scientists have recently booked paid-for Smarter sessions in three schools. Appendix 5: top tips for Smarter delivery Delivery partners offered a number of tips that relate directly to delivery of Smarter UK sessions. These are listed below. Use think, pair share or other techniques to get discussion going – eg. Split group into three positions, split facilitation so that one presenter takes each position. Avoid running Vote with your Feet near furniture – students will pick a position where they can sit down Drop the memory exercise it time is short – this means you can also cut the parts of the script that are only needed for the memory exercise Ask neuroscientist to clap at the end of the drama – starting a round of applause helps to break from the drama into the rest of the session. Sticker sheets can be disruptive. It works a lot better if the presenters hold the sticker sheets and students take just one sticker to put on the sheet rather than having their own sticker set. It is important to leave 10-15 minutes at the end of the session for the ‘Ask the Neuroscientist’ slot – this is a valuable part of the session and it is a shame if it is cut short. Starting the discussion with questions such as “does a brain bounce” helps students feel comfortable about asking any question 13