A Study of the Jolly Phonics Programme in the Teaching of Reading and Writing of English: Thaba Tseka, Lesotho 2021 Report Compiled by Sharon Flint sharon@dolencymru.org 20th December 2021 1 Introduction This paper presents the pretest and post-test reading scores of Kindergarten students as a means of studying the impact of “Jolly Phonics” on Kindergarten English literacy skills in multiple provinces throughout Thailand. Jolly Phonics is a multi-sensory method of teaching synthetic phonics. The methodology is highly student-centered and fast paced. This allows students to begin encoding and decoding right at the very beginning of the course and provides students with the necessary skills to blend words at a much earlier stage than other methods. Students are not required to memorise word spelling, but synthesize them using the knowledge provided during the classroom sessions, creating much more impactful and meaningful education. Scope of the Pilot Project In order to evaluate the impact of this method, Government and City Hall schools were chosen across six provinces in Thailand to take part in the study for the academic year beginning May 2022 through to April 2023. Six schools from each province were approached to take part in the training and 2 schools from each province were asked to act as “controls”. These schools would not be taking part in the training, but agreed to carry out the same assessment tests as the pilot schools, allowing a comparison of results to be carried out. Selection Criteria Only Government and City Hall schools were allowed to take part in the study to ensure a continuity o 2 What is Jolly Phonics? Jolly Phonics is the world’s leading systematic synthetic phonics programme. Synthetic phonics involves teaching the main 42 English letter-sounds in isolation, whilst simultaneously teaching children how to form, blend and segment these sounds to read and write words. Children are also taught “tricky words” that do not follow the sound system. The programme was created by international reading experts Sue Lloyd and Sara Wernham and is published by one of UK’s leading educational publishers, Jolly Learning Ltd. With Jolly Phonics, the synthetic phonics skills are taught alongside culturally appropriate stories, songs and actions that make Learners from Mazenod Primary School learning to read and write multi-sensory, child-centred and lots of fun! What do Teachers Say About Jolly Phonics? We always get great feedback from teachers around the world that are using Jolly Phonics. Typical comments refer to the fact that it is “easy”, “fun” and “quickly effective”. Here are some comments made by Grade 1 teachers trained in this pilot study: Phonics is the best method for teaching pupils to read and write. They are able to read and write sentences with correct spelling. Learner’s attendance is very good because of Jolly Phonics they are happy because they understand what they are doing. Mamokete Tholoana Grade 1 Teacher Learner’s progress is very good compared to the traditional method we were using in the past, this new method (phonetic) is useful. Grade 1 Learners are reading and writing very well with correct spelling. Anna Khusu, Grade 1 Teacher 3 Aims and Objectives The aim of this Jolly Futures project was to run a comparative pilot of the Jolly Phonics programme in the teaching of reading and writing of English in Government primary schools in Thaba Tseka District, Lesotho and assess its impact on early reading outcomes. The main objective of the evaluation was to determine if the synthetic phonics approach of teaching reading and writing of English, specifically the Jolly Phonics programme, leads to faster progress in the pupils’ reading and writing ability in English than those pupils not taught using the programme. Objectives o To evaluate the impact of Jolly Phonics on the English reading and writing skills of early grade pupils in government schools, so that an informed decision can be made by the government as to whether to roll it out across all schools. o To understand any challenges that may arise in the implementation of Jolly Phonics in government schools and how these challenges can be overcome for any future rollout of Jolly Phonics Research Questions 1. What, if any, impact does the introduction of Jolly Phonics in Grade 1 classes have on the initial literacy skills of pupils? 2. Are there any school or pupil characteristics that have affected any impact, produced by the introduction of Jolly Phonics, on pupils’ initial literacy skills? Methodology Research Design To answer the research questions, Literacy skills assessments will be undertaken with pupils in the 6 pilot schools, as well as in 6 matched control schools, at the start of the pilot study (baseline) and again at the end of the school year (end-line). The results of the Grade 1 pupils in the pilot schools will be compared with the Grade 1 pupils in the control schools in order to answer the first research question, on the impact of Jolly Phonics. Their results 4 will also be disaggregated in order to see whether any factors affected the impact, answering research question 2. The Schools: Selection and Context Twelve (12) schools were selected in consultation with the Thaba Tseka Education Office. Dolen Cymru have collaborated with schools there for 5 years and so there was familiarity with the programme by many school leaders. Due to the short time frame of the pilot, it was decided that the six pilot (6) schools who were familiar with Jolly Phonics should be selected so that the implementation could be as smooth and as quick as possible. In each of the six (6) pilot schools there were already Phonics Co-ordinators in place, so even though they were not Grade 1 teachers, they were able to support and mentor the newly inducted Grade 1 teachers. Table 1 below details the schools, their locations and, for the Jolly Phonics schools, how many teachers were trained in Jolly Phonics in total (some pf these prior to the Pilot). Table 1 – Details of the Schools Involved in the Baseline and End-line Studies SAMPLE SCHOOL LOCATION NO. OF TEACHERS TRAINED IN JOLLY GROUP PHONICS Jolly 1. Auray Primary School Mantsonyane 3 Phonics 2. Bocheletsana Primary School Mantsonyane 3 Pilot 3. Loti Primary School Thaba Tseka 2 Schools 4. Paray Primary School Thaba Tseka 3 5. Katlehong Primary School Thaba Tseka 3 6. Thaba Tseka Primary School Thaba Tseka 3 Control 7. Bereng Primary School Thaba Tseka 0 Schools 8. Kolbeg Primary School Thaba Tseka 0 9. Chooko Primary School Mantsonyane 0 10. Mahlakeng Primary School Thaba Tseka 0 11. Majara Primary School Thaba Tseka 0 12. Sefapanong Primary School Thaba Tseka 0 5 The Learner Sampling Procedures In both the pilot and control schools, a minimum of 17 learners from each Grade 1 class were randomly sampled from the register. An equal number of boys and girls were selected. This sample was used to administer the baseline assessments. These same sampled learners participated in the end-line assessments with some exceptions due to non-attendance and school dropout or transfers. Number of Pupils Sampled Table 2 SAMPLE SCHOOL GROUP Baseline End-line Number of Number of Pupils from Pupils Grade 1 Grade 1 Jolly Phonics Auray Primary School 21 24 Pilot Schools Bocheletsana Primary School 20 20 Loti Primary School 20 20 Paray Primary School 20 20 Katlehong Primary School 20 21 Thaba Tseka Primary School 21 20 Total Pilot Sample Size 122 125 Control Kolbeg Primary School 18 18 Schools Chooko Primary School 18 17 Mahlakeng Primary School 19 20 Majara Primary School 12 12 Sefapanong Primary School 20 20 Bereng Primary School 20 18 107 105 Total Control Sample Size 6 The average age across all Grade 1 samples was 6 years and 2 months at baseline and 6 years and 4 months at end-line. The pupils ages ranged from 6 to 8 years, although those aged 8 years were in the minority. School Demographics Location of Schools Location Pilot Schools Control Schools Rural 0 0 Semi-Rural 6 6 Urban 0 0 Facilities Facilities Pilot Schools Quality- Conducive Control Schools 0% Conducive 8.3% classroom, Conducive but needs blackboard, improvement 100 % improvement 91.7 % Unconducive 0% Unconducive 0% Conducive but needs charts, adequate desks and chairs/benches The Assessment Tool The assessment tool used to assess the pupils integrates existing EGRA and Phonics Screening assessment tools. There are 3 tests within this Phonics Screening assessment tool. Letter Sounds Test – This test assesses pupils’ knowledge of the sounds of the English language. It provides 41 graphemes and children are asked to pronounce the sound (phoneme) that they represent, on-by-one. Some of the individual sounds are represented 7 by two letters (diagraphs). All of the sounds are in lower-case, as this is easier for the earlier grade pupils. There is no time limit to the test. Word Reading Test –This test provides a list of decodable words, which pupils are asked to read one-by-one. They are given a score out of 40, based on how many words that they read correctly. Although it is not standardised, because the words are decodable. Some are real words and others are pseudo words such as dat and rin. The test provides an indication of how well the pupils have acquired the skills of phonic knowledge and blending, which provide the foundation to becoming a successful reader. Oral Vocabulary Test – This test contains 20 words, which each have a corresponding picture on a separate pupil stimulus sheet. The assessor reads each word and asks the learner to point to the corresponding picture. This provides an indicator of levels of English comprehension. Results The data gathered during these assessments were as anticipated. The majority of learners, in all schools, were low-scoring in all areas during the baseline tests. Many of these learners had missed pre-school due to the pandemic which impacted their ability to recognise letter sounds or read words. The third assessment provided more success, as many learners could remember the English words and recognise the corresponding images. 8 Mean Base Line Data Pilot schools: Control schools: Test Sample Group Mean Score Difference Letter Sounds Test Jolly Phonics 6.2 5.6 Control 0.6 Jolly Phonics 1.9 Control 0 Word Reading Test Oral Test Vocabulary Jolly Phonics Control 10.2 1.9 7.1 3.1 Table 3 Table 3 above shows that the Jolly Phonics Grade 1 group had a higher mean score on the Letter Sounds Test, Word Test and Oral Vocabulary Test than the Control Group, with a difference of 5.6 sounds, 1.9 words and 7.1 words. This suggests that groups were not evenly matched at the baseline. This could be for a variety of reasons including location in particular. Many of the control schools are out of town with a more rural population of learners. These teachers have not worked with Dolen and the Jolly Phonics methods at all. 9 Mean End-Line Data Pilot schools Control schools Table 4 Test Letter Test Word Test Sample Group Sounds Jolly Phonics Control Reading Jolly Phonics Control Mean Score Difference 32.8 32.2 0.6 19.5 0 Oral Vocabulary Jolly Phonics 13.7 Test 3.8 Control 19.5 9.9 Table 4 above shows that in the end-line assessments, the Jolly Phonics Grade 1 group had a higher mean score on the Letter Sound, Word Reading and Oral Vocabulary Tests than the control group. The pilot group scored more than 32 sounds (out of 41) in comparison to the control group in test 1. This is impressive given the short time frame of being taught with Jolly Phonics. The pilot group were able to read 19.5 words more than the control group. These results show that the learners were able to apply their phonetic knowledge to blend and read these words. The control group learners performed better in the oral 10 vocabulary test and were able to recognise some spoken words (3.8 words) however the pilot group scored significantly higher with over 9 more words. When you compare the variance from baseline to end-line for each group these results become even more apparent (see Table 5 below). The Jolly Phonics pilot group started with a higher base line and finished with a higher endline, with an increase of 26.6 sounds. In comparison, the control group started with a lower baseline but only increased with 0.6 sounds in the same 6 month period. Table 5 Group Test Baseline End-line Variance 6.2 32.8 26.6 Score 1.9 19.5 17.6 Oral vocabulary 10.2 13.7 3.5 0.6 0.6 0 0 0 0 3.1 3.8 0.7 Sound Assessment Score Grade 1 Pilot Words Assessment Sound Assessment Score Grade 1 Control Words Assessment Score Oral vocabulary The Jolly Phonics pilot group saw an increase of 17.6 words in comparison to the control group who were unable to read any words at both baseline and end-line. The control results could be attributed to the main method used for reading in the control schools. The ‘Look and Say’ or ‘Whole Word Recognition’ method is the most common approach which relies on pupils memorising the word as a whole and retaining this information over a period of time. The Word Test comprised of both real words and pseudo words such as dat and rin. These pseudo words are unfamiliar and with memory as an approach, they will be unlikely to read these words. The Jolly Phonics pilot group, at end-line, were able to read both real 11 and pseudo words as they were able to apply their phonetic knowledge and blending skills to read words regardless of if they had seen them before. The Jolly Phonics pilot group saw an increase in 3.5 words in the oral test in comparison to 0.7 increase in the control group. Disaggregated Data End-line performance by gender in pilot schools: It is interesting to note that performance according to gender was consistently equal throughout the 3 tests: Sounds – 32 Female / 33 Male Words – 19 Female / 19 Male Oral vocab – 14 Female / 13 Male This is extremely important for Basotho schools were generally female learners outperform male learners. These results indicate that the Jolly Phonics approach is effective for all learners regardless of their gender. 12 End-line performance by school: The following charts display the performance per school which is another extremely important finding to share with individual schools. Teachers’ confidence levels varied throughout the pilot and all schools were affected by COVID-19 measures which impacted attendance. Variance in learners sound recognition is minimal however their ability to blend to read words shows greater variance according to each school. These results highlight the importance of teachers modelling and explicitly teaching learners how to apply their letter sound knowledge and blend to read. Katlehong Bocheletsana Thaba Tseka Auray Loti Paray Katlehong Bocheletsana Thaba Tseka Auray Loti Paray The lower word reading scores can be attributed to classes shifting and having less time to practice blending as they move through the sets of sounds. This was noted by all teachers in August, so a refresher workshop was led by the local trainer team to ensure all Grade 1 teachers were confident in applying the 8-step approach to teaching each sound and making adjustments according to their particular context. 13 Katlehong Bocheletsana Thaba Tseka Auray Loti Paray The oral vocabulary test scores are good and relatively consistent between schools except the one that is slightly lower. This could be due to a number of factors including learner absence and shifting which impacts the ability of learners to remember English words without being able to read them. Bar Chart showing the increase in pupils’ abilities from May to November 2021 in Control and Pilot Schools Control Schools Pilot Schools 14 Control Schools Pilot Schools These charts show the excellent progress of learners in the Pilot schools across all 3 tests. Considering the challenges faced in 2021 this is an extremely pleasing result. Learners enjoyed their learning and teachers built their confidence. The sense of belonging created by the group of teachers and their Phonics Co-ordinators improved staff morale and productivity. 15 Successes and Challenges There were a number of factors that contributed to the success of this pilot and the improvements in the learner’s literacy outcomes. These factors included: Governmental Support: There has been strong, continuous support from the Education Department in Thaba Tseka. Two key District Resource Teachers enabled all monitoring and evaluation activities, training workshops, teacher mentoring and collaboration with Dolen Cymru. Detailed Initial Jolly Phonics Training: 2 days of initial teacher training provided teachers with a thorough introduction to the programme. Culturally relevant example stories, songs and images were provided where necessary, which ensured pupils’ maximum engagement in the learning experience. A follow up day long workshop was delivered to Grade 1 teachers alongside their school-based Phonics Co-ordinators to maximise collaboration and support moving forward. Regular Teaching of Jolly Phonics: It is recommended that Jolly Phonics should be taught at least 3-4 times a week, and ideally daily. In this pilot, the schools that showed the most progress had teachers that taught Jolly Phonics on a regular basis, which allowed the pupils to make good progress in their reading and writing ability. Weekly Zoom Meetings, WhatsApp & Jolly Phonics App for teachers: those with smart phones were able to access support on the WhatsApp Jolly Phonics groups and also use the Jolly Phonics Lessons App. Those with the app reported being able to check pronunciation of sounds was very valuable as well as recalling the tunes for the Jolly songs. Every weekly Phonics Friday Zoom meeting was accessible to Pilot teachers along with many other members of the education community engaged in Jolly Phonics teaching across Lesotho. 16 Principal Support: Principals accompanied their teachers to the Jolly Phonics trainings. The Principals who stayed regularly involved with the programme being implemented in their school and supported teachers to timetable Jolly Phonics achieved higher results. Learner attendance: Those schools who had higher learner attendance, during the challenging COVID-19 period, achieved better results as learning was not as interrupted. Challenges Experienced in the Pilot: Overcrowded classrooms and large class sizes: In some of the schools (pilot and control) classes held upwards of 60 learners in one stream. Although this problem is often unavoidable in this context, the programme must be sensitive to the number of children teachers are responsible for and provide support and guidance to the teachers in using the Jolly Phonics programme with such large pupil groups. Teachers with large classes commented that Jolly Phonics was so multi-sensory and fun that they achieved better class control and class engagement in the Jolly Phonics sessions than they did at any other lesson/subject. Attendance rates: Schools with lower attendance rates are likely to return lower test scores. As Jolly Phonics is ideally taught every day, or several times per week, without regular attendance, learners are likely to be left behind quickly. Again, although this problem is unavoidable to some extent, the programme should be sensitive to these circumstances, ensuring regular revision sessions and pupil catch up sessions where appropriate. This was exacerbated because of the COVID-19 pandemic. 17 Monitoring and evaluation – Leaders were trained to use the Jolly Monitor App on their tablet devices provided by Dolen Cymru. This was extremely problematic for some who have low confidence with new platforms. This did impact the reporting process and new plans for 2022 need to be implemented to ensure this can be used successfully moving forward. Conclusion Grade 1 Learners in Thaba Tseka Pilot schools can read and write independently. Grade 1 Teachers in Thaba Tseka Pilot have increased their confidence in teaching English and have improved their approach to child-centred learning. The Jolly Phonics approach to literacy teaching has the potential to change the life-chances for more Basotho children as they become able to access other subject areas through improved literacy levels, behaviour and attendance. 18 The aim of this project was to pilot the Jolly Phonics programme in the teaching of reading and writing of English in public Government schools in Thaba Tseka, Lesotho and assess its impact on early reading outcomes. Through this evaluation it has been determined that the synthetic phonics approach of teaching reading and writing of English, and specifically the Jolly Phonics programme, can lead to greater progress in the learners reading and writing ability in English than those pupils not taught using the programme, where the processes of the programme are carried out correctly. In the end-line assessments, assessors found that learners who had been regularly using Jolly Phonics were more confident and quicker at reading and recognising sounds. They made more plausible attempts at deciphering unfamiliar words and were not fazed by pseudo words. The learners who at base line had shown potential in reading, now armed with tools to help them blend and segment have made large improvements. The findings from this study show that if fidelity to the Jolly Phonics program is shown, pupils are able to quickly develop their reading skills and find enjoyment in learning. The similarity between English and Sesotho sounds also makes Jolly Phonics a very versatile programme, which will benefit all who are exposed to it. 19 Recommendations and Next Steps This document has outlined the findings from the Jolly Phonics pilot study in government schools as part of the Jolly Futures programme. The following are recommendations for the next steps in a future Jolly Phonics rollout in Lesotho. To see videos and pictures from the pilot schools please look at our Padlet: Jolly Phonics Pilot Project Thaba Tseka (padlet.com) We recommend after you have read this report, we schedule a Zoom meeting with Dolen Cymru and Jolly Futures for January 2022 to discuss the following: - Arrange for our Trainers to run a taster session with resources for Ministry staff Discuss the possibility of running another 6-month Pilot in another area of Lesotho (March-September 2022) Discuss the possibility of training Education Office staff in other parts of Lesotho Discuss the possibility of country wide roll out in 2023 Sharon Flint Dolen Cymru Lesotho Education Officer sharon@dolencymru.org We look forward to hearing from you and working with you to improve the English literacy levels of your learners through Jolly Phonics. 20 21