Evaluation of the Implementation of the “All Children Being Safe” (ACBS) Program In Three Preschools in NSW Prepared by: A/Prof Manjula Waniganayake Ms Karen Roberts Institute of Early Childhood Macquarie University Access Macquarie Ltd (December 2011) Access Macquarie Ltd – Commercial-in-Confidence Acknowledgements We offer our sincere thanks to the children, parents and staff at each of the participating centres for collaborating with us throughout the evaluation. In preparing this report we have attempted to capture their input in authentic ways. Likewise, we want to say a special thank you to the external members of the Project Steering Committee for their wise counsel and participation in all aspects of project design and implementation. From the start, Angela Walsh and Michelle Rose, as the NAPCAN Project Manager and ACBS Project Coordinator respectively, welcomed us, ensuring we had a sound orientation to the local community. Their involvement in liaising with all the stakeholders was critical to the success of the evaluation data collection and analysis. Thank you both for your support, local insights and professionalism in working with us throughout this project. We trust that this report provides a comprehensive overview of the successful implementation of the ACBS program in Kempsey, and the analysis provides sufficient learnings for use in future planning and development of this program in other early childhood settings. We felt that everyone involved in this project shared a lifelong commitment to keeping children safe. We hope that in someway our work contributes to achieving this goal. It has been an honour and a pleasure to work on this project with all of you. With best wishes Associate Professor Manjula Waniganayake and Ms Karen Roberts Institute of Early Childhood Macquarie University Access Macquarie Ltd December 2011 Access Macquarie Ltd – Commercial-in-Confidence An evaluation of the implementation of the “All Children Being Safe” (ACBS) program in three preschools An evaluation of the implementation of the “All Children Being Safe” (ACBS) program in three preschools Table of Contents Executive Summary Summary of Recommendations 1. INTRODUCTION 1 2. PROJECT BACKGROUND 2 3. 4. 2.1. Child protection in context 3 2.2. Centre profiles (as at December 2010) 3 2.3. Working with Aboriginal people and communities 4 2.4. Adaptation of the ACBS program for pre-schoolers 5 EVALUATION ORIENTATION 3.1. Evaluation literature 6 3.2. Evaluation framework adopted for this study 7 PRESENTATION OF THE FINDINGS 4.1. OUTCOMES FOR CHILDREN 11 11 4.1.1. Learning about safe and unsafe feelings 11 4.1.2. Learning involving safe people and places 14 4.1.3. An unexpected learning 18 4.2. OUTCOMES FOR EDUCATORS 18 4.2.1. Sound knowledge and best practice in relation to child Protection legislation as it impacts on educators 19 4.2.2. Developing skills in implementing the ACBS Program with children in their service 21 4.2.3. Building co-operative relationships with families to promote their participation in the ACBS program 24 4.3. OUTCOMES FOR PARENTS AND COMMUNITY 27 4.3.1. Parents and community awareness of the engagement In the delivery of the ACBS program 27 4.3.2. 29 4.4. Sharing the ACBS Program with children at home PROGRAM HIGHLIGHTS AND FUTURE DIRECTIONS 30 4.4.1. Program highlights and ideas for improvement 31 4.4.2. Suggestions for program improvements 34 4.4.3. 5. 6 Participants’ perspectives on future directions for the ACBS program DISCUSSION AND RECOMMENDATIONS 36 37 5.1. Program development 38 5.2. Knowledge and skills of educators 39 5.3. Extending children’s learning 41 5.4. Following up on participants’ suggestions 41 6. REFERENCES 44 7. APPENDICES 47 7.1. Modifications made to the ACBS preschool program 48 7.2. Evaluation framework 50 7.3. ACBS program evaluation instruments 54 7.4. Letter of thanks from preschool staff 61 An evaluation of the implementation of the “All Children Being Safe” (ACBS) program in three preschools An evaluation of the implementation of the “All Children Being Safe” (ACBS) program in three preschools EXECUTIVE SUMMARY This evaluation report is based on the NAPCAN sponsored adaptation of the “All Children Being Safe” (ACBS) program for early childhood, in Kempsey, a rural community with a high proportion of Indigenous families, located on the mid north coast of NSW. Macquarie University was invited to evaluate the adaptation and implementation of the ACBS project in three selected preschools in Kempsey. The ACBS is a protective behaviours program originally developed for use with primary school aged children. The Greenhill Primary School in Kempsey initially developed it in response to the request for the development of a culturally safe protective behaviours program that matched the child protection curriculum of the NSW State Government, administered by the Department of Education and Training (DET). The school children created the artwork for the storybooks and the training package was developed by NAPCAN with funding provided by the WESTPAC Bank. The CAGES Foundation is responsible for funding the extension of the program to preschoolers in Kempsey that covered the costs of the adaptation, implementation and evaluation of the Early Childhood ACBS program. The Early Childhood ACBS program was implemented in the preschools during 2011, and the evaluation report reflects the voices of the children, parents, educators and community members who participated in this project. The NAPCAN project manager and coordinator established a Project Steering Committee consisting of the preschool teachers and local Indigenous community services personnel to provide advice on all aspects of project design, implementation and evaluation. During December 2010, two of the selected preschools trialled the original ACBS stories with a group of pre-schoolers at their centres. Feedback from this pilot study was also taken into account in the development of the full package of resources for the modified ACBS program. The Project Coordinator, who was an experienced, university qualified early childhood teacher, had responsibility for reviewing the content of the primary ACBS program, and making the necessary modifications for use with 3-5 year old children in preschool settings. This included shortening the text of the five animal stories. An important addition was the development of an educator’s resource kit consisting of items such as puppets, music CDs, dress-up clothes, as well as art-craft materials for use in developing learning activities for pre-schoolers. These resource kits were supplied to each preschool to assist educators with the implementation of the ACBS program. As indicated in the evaluation, this resource kit was invaluable in establishing interest, adding depth and creativity as well as making the experiences meaningful for pre-schoolers. The ACBS Program outcomes were defined in terms of children, parents and community as well as preschool staff, and were used as the focus for this evaluation. In collaboration with the key local stakeholders matching evaluation strategies and instruments were designed. The aim was to ascertain evaluation data in a continuous way throughout the program implementation as well as prior to commencement and one month after completion. The overall evaluation plan was formally documented as an evaluation framework and is located in Appendix 2 of this report. Likewise, the full set of instruments matched against the specific objectives has been placed in Appendix 3. i An evaluation of the implementation of the “All Children Being Safe” (ACBS) program in three preschools Children, parents, educators and other community members actively participated in a variety of ways throughout the implementation of the ACBS program in Kempsey, and willingly contributed to the evaluation data collection. Based on the analysis of this data, the key findings from this study include the following: Consultation with and collaboration among key stakeholders is vital to the program’s success and needs to be coordinated by a qualified early childhood educator. In this instance, the stakeholders included local Indigenous people and community members, family members and early childhood staff. Early childhood educators implementing the program need to have a sound knowledge of and rapport with the children involved, as well as well developed skills in appropriately questioning children and eliciting responses. Training and support for early childhood educators, in relation to child protection responsibilities and general program implementation, needs to be available both prior to and during implementation. Long-term impacts of the program can be evaluated through a process of individual program assessment, continued follow up sessions and reviews including the children and families involved. The early childhood ACBS program was particularly enhanced by a range of resources and activities relevant to all areas of young children’s learning and development. Promoting key concepts about child safety relevant and meaningful to each local community can further enhance the early childhood ACBS program. These findings from the evaluation project were used in developing 11 recommendations for consideration by NAPCAN in enhancing the Early Childhood ACBS program further and expanding its implementation in other preschools throughout Australia. This evaluation highlights the potential benefits of the ACBS protective behaviours program when used appropriately by early childhood educators with sound expertise and experience of working with young children and their families. This project also reflects the strengths of an authentic community partnership that can facilitate the coming together of children, parents, professionals and the community. It is clear that adopting a whole of community approach can ensure that learning about child protection and safety can be examined in an integrated way across the community. Such an approach can thereby also make learning about keeping children safe sustainable as a life-long commitment by all involved. Manjula Waniganayake and Karen Roberts Institute of Early Childhood Macquarie University December 2011. ii An evaluation of the implementation of the “All Children Being Safe” (ACBS) program in three preschools SUMMARY OF RECOMMENDATIONS Recommendation #1: That NAPCAN recommends the appointment of an early childhood educator as the coordinator of ACBS project development and management in each community. Recommendation #2: That in planning, designing, implementing and evaluating the use of the ACBS program, consultation with the community is built into each community’s approach to the program. Recommendation #3: That when delivering the ACBS program in any early childhood setting, emphasise the importance of collaboration amongst staff in planning and implementing the program in a holistic way. Recommendation #4: That NAPCAN ensures that future ACBS programs are implemented by capable educators who can establish good rapport with the children and families participating in the programs. Recommendation #5: That NAPCAN provides an orientation session at commencement and finds ways of providing adequate support and guidance for educators during program implementation. Recommendation #6: That all educators involved in the implementation of the ACBS program have thorough prior training in their child protection responsibilities and with particular attention to ways of responding to a disclosure. Recommendation #7: That NAPCAN includes in the ACBS manual, program evaluation instruments used in this evaluation to enable early childhood settings to do their own follow up by assessing learnings with children, parents and community. Recommendation #8: That NAPCAN develops a longitudinal research study to assess the long-term impact of the ACBS programs across communities. Recommendation #9: That the ACBS educator’s resource kit supplied by NAPCAN includes both puppets and costumes for role-play and drama activities. Recommendation #10: That when developing further suggestions to use with the ACBS program, NAPCAN considers including activities and experiences that cover a broad range of learning and skill development. Recommendation# 11: That NAPCAN gives due consideration to the follow up suggestions offered by everyone involved in the Kempsey study, as listed in this report. iii An evaluation of the implementation of the “All Children Being Safe” (ACBS) program in three preschools iv An evaluation of the implementation of the “All Children Being Safe” (ACBS) program in three preschools 1. INTRODUCTION This report is based on the NAPCAN sponsored adaptation of the “All Children Being Safe” (ACBS) program for preschools, in a rural community, on the mid north coast of NSW. Macquarie University was invited to evaluate the adaptation and implementation of the ACBS project in the three selected preschools in Kempsey. Each of these preschools had a high number of children and families from Indigenous backgrounds. The project began with a pilot study to explore the possibilities of using the ACBS program during October – December 2010 with one group of children in three preschools. Due to difficulties with staffing, this phase of the project was implemented in two centres only. Based on the learnings from this initial phase carried out in 2010, the Project Manager, began modifying the stories included in the ACBS program to make it more appropriate for pre-schoolers. With input from the preschool staff and the Steering Committee members, a revised program was finalised by April 2011 and implementation began in each centre in Term 2. Participating preschools and key educators: South Kempsey Preschool: Dalaigur Preschool: ABC Learning Centre, West Kempsey: Project Period: Shirley Roslyn Melanie July 2010 to September 2011 NAPCAN Project Manager: Angela Walsh ACBS Project Coordinator: Michelle Rose ACBS Project Evaluators: Manjula Waniganayake and Karen Roberts Project Steering Committee: Shirley, Roslyn and Melanie (the educators) Janet Jensen, Marilyn Dean, Karen Hall and Delya Smith At the beginning of 2011, Michelle Rose, the Project Coordinator, examined the objectives of the ACBS project developed for implementation with primary school children, and modified these to make the learning more appropriate and relevant for children aged 3-5 years. In turn, in discussion with the project manager and coordinator, the evaluators used the modified learning objectives to develop an evaluation framework to assess the learnings from the ACBS project implementation during 2011. These objectives were categorised in three ways, targeting learning directed at the children, the preschool staff and the parents and community. The evaluation framework (see Appendix 2) was developed with input from the ACBS Project Steering Committee, comprising the preschool teachers, and key local personnel who were actively involved in child protection matters in Kempsey. 1 An evaluation of the implementation of the “All Children Being Safe” (ACBS) program in three preschools 2. PROJECT BACKGROUND “All Children Being Safe” (ACBS) is a protective behaviours program originally developed for use with early primary school aged children. The Greenhill Primary School, Kempsey, initially developed the ACBS Primary Program in response to the request for the development of a culturally safe protective behaviours program that matched the Child Protection curriculum of the State Government, administered by the Department of Education and Training (DET). Over 95% of the students at Greenhill Public School identify as Aboriginal. The program was developed with the support of local service providers and a team of community workers from NAPCAN, Kempsey, Schools as Community Centres, North Coast Area Health Service, and the Durri Aboriginal Medical Service (AMS). Acknowledging the effectiveness of the use of dreamtime stories when developing problem solving skills with the children in the school, Local Greenhill Elders were consulted in relation to which animals and birds were appropriate for the Greenhill community. The school children created the artwork for the storybooks and the training package was developed by NAPCAN with funding provided by the WESTPAC Bank. The goal of the ACBS program is to provide children with appropriate behaviours when interacting with children and adults in meaningful and nonthreatening ways to assist them to feel strong and stay safe in their communities. Five stories using bush animals found in the local community provide the foundation for learning in the ACBS program. They contain messages around safety and are explored with the children through a variety of learning activities. The ACBS program was piloted in one school, and then expanded to three others located within Kempsey. This pilot was well received, winning two awards for promoting health and wellbeing. The CAGES Foundation approached NAPCAN in 2010 in relation to funding the expansion of the ACBS Program into rural and remote Aboriginal Communities across NSW. The option of a preschool adaptation was discussed due to interest that had been expressed by Dalaigur Pre-school in Greenhill Kempsey. CAGES then agreed to fund the adaptation of the ACBS program and its implementation in 3 preschools in Kempsey, NSW. The NAPCAN Project Coordinator, an experienced and university qualified early childhood educator, then adapted the original ACBS program targeting primary school aged children, to make it more appropriate for use with preschool aged children in the local community (see Appendix 1 for details). This modified early childhood ACBS program was aimed largely at 4 to 5 year olds, and can be implemented over a period of at least 6 weeks. Overall, this evaluation project reflects an authentic community partnership as all aspects of the ACBS program design, implementation and evaluated included the key stakeholders - especially the children, their families and staff at the three preschools. 2 An evaluation of the implementation of the “All Children Being Safe” (ACBS) program in three preschools 2.1. Child protection in context The Convention on the Rights of the Child (UNICEF 1989), along with child protection legislation, identify that all children have a right to be safe from harm. The NSW Government, in their efforts to keep children safe, following Justice Wood’s Report of the special commission of inquiry into child protection services in NSW (2008), has been coordinating a range of prevention and early intervention strategies between governments, agencies and communities. A family support approach has been shown to be particularly valuable in helping families with their role of parenting and caring for their children. The value of a protective behaviours education program such as the ACBS is in the empowerment of children to keep themselves safe in all situations. It acknowledges that danger is not restricted to strangers, and that a person known to the victim commits the overwhelming majority of all types of child abuse. The protective behaviours program was initially developed in the USA in the 1970’s and the program was introduced to Australia in the 1980’s as an educational strategy to prevent child abuse (www.edsite.com.au/pbnt/protect/html). Following program evaluations, it has since been adapted to suit people of all ages in all situations where personal safety is a concern, particularly acknowledging issues of domestic violence and societal violence such as harassment and bullying. (Tomison & Poole, 2000, p.4). Protective behaviours encourage people to assert their right to feel safe, listen to what their body tells them, and follow up by taking action to solve problems or seek assistance from others who can help them (www.kidsmatter.edu.au/primary/programs-guide/protective-behaviours). The National Framework for the Protection of Australia’s Children was established in 2009. (FAHCSIA, 2009) In NSW, early childhood educators are mandatory reporters under the Children and Young Persons (Care and Protection) Act (1998). They are required to undertake training in identifying and responding to children at risk, and they have a duty of care to help ensure the safety and well being of all children in their care. Implementing a protective behaviours program requires an understanding of child protection issues and supports them in their duty of care. 2.2. Centre profiles (as at December 2010) Each of the three centres was selected for the ACBS program implementation for a variety of reasons. One of the key reasons included their location in the same area as the schools that implemented the ACBS. Other reasons included the high numbers of Indigenous children and/or staff in their services and their commitment to collaboration with Indigenous members of the local community. A brief profile of each of the preschools involved in this project was documented using data from the centre profile questionnaires completed by the educators at the start of the pilot study in December 2010. 3 An evaluation of the implementation of the “All Children Being Safe” (ACBS) program in three preschools Centre 1 This is a not-for-profit, stand-alone community owned and managed centre. It had the capacity to enrol up to 49 children aged 2 to 5 years. At the time of the pilot study, there were 28 children enrolled. All children except 2 were from an Indigenous background. It employed 5 educators, 2 assistants in each room and 1 floater. Indigenous staff comprised the educator/director and 2 assistants. Centre 2 At this preschool enrolments comprised 117 children aged 2 to 5 years. The centre was a not-for-profit, stand-alone, community owned and managed service. There were 14 educators (9 part time) and all but 5 were Indigenous. Almost all children at the centre came from an Indigenous background. Of these, 14 children were registered in the child protection system. Centre 3 This long day care centre catered for 76 children aged birth to 6 years. Of these, 31 children were from an Indigenous background, and included 6 children registered in the child protection system. It was a not-for-profit centre, owned and managed by a large national organisation. There were 11 educators employed with 2 working part time, and only one educator with a university degree in early childhood. 2.3. Working with Aboriginal people and communities The incidence of child abuse and neglect in Australia continues to be alarming, and for Indigenous children, the statistics are particularly concerning. In 2009-2010, Aboriginal and Torres Strait Islander children comprised 4.6% of Australian children and 26.6% of all confirmed reports of abuse or neglect. This means that Aboriginal and Torres Strait Islander children were 7.7 times more likely than other children to be the subject of a confirmed report of abuse or neglect than non-Indigenous children. (Beryln, Bromfield & Lamont, 2011, p.1) Under the NSW Government’s “Keep Them Safe” Plan for all children, an Aboriginal Impact Statement was developed to ensure a participative and collaborative approach to child protection work with Aboriginal children, families and communities, and address the over representation of Indigenous children in the child protection system. (NSW Government, 2010, p.1) The original ACBS program for school aged children involved a collaborative partnership with local schools and Aboriginal health and community organisations. It also respectfully acknowledged the Aboriginal traditional 4 An evaluation of the implementation of the “All Children Being Safe” (ACBS) program in three preschools owners of the land and the impact of colonisation on Aboriginal people. This collaboration with and respect for local Aboriginal people was vital to the success of the program, and has also been significant in the adaptation and implementation of the modified early childhood version of the ACBS program used in this project in 2011. A number of resources developed to support people working with Aboriginal people and communities describe effective approaches as operating on the following principles: Consideration of the historical context of colonisation, racism, discrimination, disadvantage and cultural dislocation and child removal from families Priority of cultural safety, Indigenous managed services with control of services and responsibility for outcomes, and appropriate and adequate resources and support for agencies (Higgins, 2010, p.1-3) Consultation and involvement of leading Aboriginal community members, involving as many Aboriginal elders and leaders and representatives from all local family groups and Aboriginal organisations Engaging in proper effective consultation in an equal relationship (NSW DoCS, 2009, p 30-34) These strategies indicate that, while the child protection situation is very serious for Aboriginal families, “there are also many strengths in Aboriginal communities and a deep commitment to the care of children on the part of community members” (NSW Government, 2010, p.3) The ACBS project development, implementation, and evaluation findings have also highlighted this strong commitment among Aboriginal community members in the Kempsey area. 2.4. Adaptation of the ACBS program for pre-schoolers Based on stories depicting Australian bush animals, and in consultation with Indigenous elders and others, the ACBS program was specifically designed for use with Indigenous children. It can also be useful for children from other cultural backgrounds. Modifications made to the ACBS program aim to have more relevance for preschool aged children and early childhood programs and curriculum. The preschool version of the ACBS program provided direct links to the National Early Years Learning Framework (DEEWR, 2009) and the National Quality Standard (DEEWR, 2010). To make it more user-friendly for pre-schoolers, a number of adaptations were made by the Project Coordinator (See Appendix 1) Some of the key changes included can be synthesised as follows. The ACBS storybooks, which are the centre piece of the program, were enlarged; the cover and pages made of thicker paper, and the cover colour changed to distinguish it from the primary school storybooks. The stories were shortened for use with pre5 An evaluation of the implementation of the “All Children Being Safe” (ACBS) program in three preschools schoolers. The overall program allows for flexibility in terms of timing and staging of how the stories are presented according to children’s interests and abilities. The introduction section in particular, has been shortened to offer more discussion of the relevant concepts being introduced or expanded on throughout the program. An educator’s resource kit consisting of props and creative media, such as puppets, costumes, books and pictures, music and movement CDs, and art and craft resources, suitable for the age group was developed to accompany the program implementation in preschools. Items such as colouring stencils and work sheets used with primary school children were excluded. Facial expression paddles and emotion cards were included to support learning and discussion about feelings. A map for each story was included to help children recall the previous story and be introduced to the next one. The stories can be set up for use on a SMART board or an Interactive white board, and thereby engaging children in the program through computer-mediated technology containing vivid images that can enhance the story messages. 3. EVALUATION ORIENTATION In preparing this evaluation report, a brief review of other evaluations of child protection programs was undertaken, along with a review of literature concerning child protection program evaluation projects. 3.1. Evaluation literature “As the main goal of an evaluation is to indicate whether a program is effective or not, it is important that an evaluation is conducted properly”. (Lamont, 2009, p.2) This can involve pre- and post- tests to ascertain change for participants against program goals. A comparison group is also usually recommended except in the case of child welfare and protection programs, where ethical consideration needs to be given in terms of preventing a control or comparison group of children from accessing the program. The final element involves follow-up testing to assess program success beyond the short term. Obviously, adequate data collection is the key to evaluating program effectiveness. Evaluations of prevention programs are generally qualitative and descriptive, and the use of focus groups and interviews with participants and staff are common. The evaluation of Braveheart’s prevention program called “Ditto’s Keep Safe Adventure” (Evans & Peck Services, 2008) included pre and post questionnaires for teachers regarding individual child participants, an observation sheet for teachers to use during the program, and a feedback form relating to teachers’ perceptions of the program. A parent survey was 6 An evaluation of the implementation of the “All Children Being Safe” (ACBS) program in three preschools also used by families to record children’s understandings and parents’ own thoughts about the use of an at home activity book, following the program. For children, a child activity sheet was included where they could draw what they had learnt from the program. An evaluation of the original primary school ACBS program involved post program interviews with children, while school teachers were surveyed immediately after the program was implemented and again several months later. The evaluation focused on children’s memory of stories and concepts as well as school teachers’ and children’s thoughts about the ACBS program. In assessing effectiveness of programs on protective behaviours, “program evaluations are fundamentally designed to assist with the planning of future programs and/or to improve pre-existing programs.” (Tomison & Poole, 2000, p.6) They need to be tailored to meet the needs of all stakeholders as well as to fulfil a specific and required purpose. 3.2. Evaluation framework adopted for this study A wide range of evaluation strategies and instruments were developed and utilised to capture data reflecting the learning that occurred throughout the implementation of the ACBS program during 2011. Each of these instruments was designed to match the participants’ learning perspectives and assess the anticipated outcomes as noted on Table 1. The evaluators designed these instruments with input from the preschool staff, the project coordinator, project manager as well as members of the Project Steering Committee. This consultative approach was important in ensuring the relevance and appropriateness of the proposed evaluation strategies and instruments to be deployed in this project. Some of the evaluation instruments were designed as a set of common questions to be used by the educators consistently throughout the weekly implementation activities. The evaluation instruments were used not just as evaluation tools, but also as teaching and learning strategies. That is, the questions were aimed at stimulating and engaging educators and children to think reflectively about child safety and protection with respect to children’s feelings and perceptions of safety and security. The educators also completed reflective questionnaires at different intervals, such as at the completion of each module and when the full program had been delivered at their centre. Asking children questions prior to, during, and following implementation of the ACBS program was an essential part of this evaluation. Early childhood educators are well placed to interview young children, who are likely to feel comfortable in their preschool environment and with the educator, especially if they are seen as similar and empathic. (Hill, 2005, p. 72) Reinforcing this perspective, Brooker (2001) states the children’s own teacher or carer working within their normal daily setting..(her) ongoing observations of the children, her familiarity with their backgrounds and behaviours, and her alertness to the effects of the research methods upon them, all help to produce good and reliable evidence. (p. 167) 7 An evaluation of the implementation of the “All Children Being Safe” (ACBS) program in three preschools A full list of the evaluation instruments used in this project is presented in Table 1, and the actual documents have been included in the Appendices at the end of this report. To assist with making sense of the findings of this project, the links between the evaluation strategies and instruments and the anticipated learning outcomes, are presented in Tables 2-4. Table 1: List of Evaluation strategies and instruments used Pre-program implementation – general 1. Centre profile questionnaire 2. Field visit notes by evaluators including discussions with individual teachers, project managers and the Steering Committee With Children 1. Pre-program questions with 1 -2 small groups one week before program implementation 2. Drawings of safe people and places (pre-program) 3. Post-program questions with 1 -2 small groups one month after program implementation 4. Children’s drawings of safe people and places (post-program) With educators 1. Teacher reflections on child protection legislation 2. Pre-program overview questions - teachers reflections 3. After each module – teacher reflections 4. Post-program overview questions – teacher reflections With parents and community 1. Pre-program questions to teachers (see overview questions 4 to 7) 2. Teacher documentation of relevant conversations with parents & other community members regarding their experiences with their children about the content included in the ACBS program 3. Community gathering and facilitated conversation by the evaluation team 4. Email questions to Steering Committee members With Program Coordinators Field notes, telephone conferences, email correspondence and entry and exit discussions during site visits to Kempsey in December 2010 and August 2011. The next three tables (Table 2-4) show the connections between the anticipated learning outcomes and the evaluation strategies and instruments used in this study. 8 An evaluation of the implementation of the “All Children Being Safe” (ACBS) program in three preschools Table 2: Outcomes and evaluation involving the CHILDREN ANTICIPATED LEARNING OUTCOMES EVALUATION STRATEGIES & INSTRUMENTS (see Appendix 2) To demonstrate an awareness of the concepts of safe and unsafe living through an ability to Identify safe and unsafe feelings and that every child has the right to feel safe. Identify people in their community who keep them safe. Feel safe, secure and supported. Develop the language to communicate safe or unsafe behaviours / situations. Identify people who can help them if they have been hurt. Interact in relation to others with care, empathy and respect. Develop a sense of belonging to groups and communities The evaluators developed a common set of questions for the three teachers to use with the children at their centres, to engage them in discussions in small groups at regular intervals as follows: Questions used one week BEFORE program implementation Questions used at the completion of each module Questions used one month AFTER implementation Pedagogical documentation – teachers also documented their observations of children’s learning using digital photography as well as preparing classroom displays and portfolios of children’s work that were developed during the implementation of the program. Table 3: Outcomes and evaluation involving the Preschool EDUCATORS ANTICIPATED LEARNING OUTCOMES To demonstrate an enhanced awareness of child protection strategies by: a) Participation by all preschool staff in training and information on child protection including support services in their community. b) Improving the standards of child protection practices, particularly in light of new legislation or information. c) Incorporating the modified ACBS program for early childhood into their preschool programs. d) Engaging parents and other family members in the delivery of ACBS program in their centres. EVALUATION STRATEGIES & INSTRUMENTS (see Appendix 2) Each teacher completed a Child Protection Questionnaire to indicate their current awareness of relevant matters before the commencement of the ACBS program The teachers also completed a questionnaire individually BEFORE the ACBS project was implemented. Likewise, AFTER the project implementation was completed, all teachers participated in a group discussion and completed questionnaires. 9 An evaluation of the implementation of the “All Children Being Safe” (ACBS) program in three preschools Table 4: Outcomes and evaluation involving the PARENTS AND COMMUNITY ANTICIPATED LEARNING OUTCOMES EVALUATION STRATEGIES & INSTRUMENTS (Appendix 2) To demonstrate an enhanced awareness of child protection issues in the wider community through a) Participation in community workshops about the ACBS program b) Engagement in the delivery of the ACBS program e.g. craft activities, storytelling, drama, music and movement. c) Sharing the ACBS program with their children at home through culturally appropriate literature. A centre profile survey questionnaire was used to ascertain information about child protection issues impacting on the local community, particularly the children and families attending the participating preschools. In addition, the evaluators asked the teachers to document relevant conversations with parents and other family members regarding their experiences with their children about content included in the ACBS program. Following the completion of the ACBS project implementation, parents and community members from each of the preschools were invited to a community gathering over lunch to discuss their perspectives on the ACBS project. The questions used during this discussion are included in Appendix 2. Members of the Project Steering Group were also sent a questionnaire by email, asking them to comment about their reflections on how the ACBS program was received by the Kempsey community. Overall, the preschool staff implemented the evaluation instruments developed to collate data on children’s learning. In turn, the evaluators engaged the preschool staff in individual interviews and group discussions to ascertain their perspectives on using a range of evaluation instruments. Three parents representing two of the three preschools participated in the community gathering and actively contributed to the evaluation. Two members of the Steering Committee also completed an email questionnaire, and thereby adding to the community perspectives on the ACBS program implementation. The project coordinator and the manager were also included in interviews and discussions throughout the program implementation. The evaluators also presented a workshop attended by the preschool educators and members of the Steering Committee, for two purposes. Firstly, by placing the ACBS project within a children’s rights perspective, the workshop looked at preschool educators’ child protection obligations. Secondly, the evaluation strategies and instruments designed for this project were discussed and refined to ensure that participants had a good understanding of the expectations of, and ethics involved in evaluating the implementation of the ACBS program with pre-schoolers. 10 An evaluation of the implementation of the “All Children Being Safe” (ACBS) program in three preschools 4. PRESENTATION OF THE FINDINGS In this section we present the data collected and analysed for this evaluation. This presentation is organised under FOUR sub-headings focusing on the anticipated outcomes of the project for children, educators, and parents and community as well as program highlights and improvements. When reading these findings, it is important to note the following: a) The educators who participated in this study, provided much data in the form of photographic images, video recordings and text-based data when responding to evaluation questionnaires and notes taken during discussions with small groups of children. This data contained input from children and their families as well as teacher reflections. To maintain the confidentiality of the participants, we have not identified the primary sources and provide summative comments based on our collation of the data. b) The ACBS project work began in 2010, and two of the centres implemented the program without major modifications to a group of preschoolers that year. Some of these children were present when the modified program was being implemented during 2011. When these children participated in the pre-program discussions, their exposure to the ACBS stories during 2010 can be seen in the responses recorded in 2011. 4.1. OUTCOMES FOR CHILDREN The anticipated outcomes for children have been collated under two subheadings focusing on children’s learnings about a) feelings and b) people and places, when responding to child safety matters. 4.1.1. Learning about safe and unsafe feelings One week prior to the ACBS program was implemented at each centre, the educators held discussions with the children to ascertain their current understandings and perceptions about safety and fear. When asked about what makes them feel safe, the children typically referred to their parents and other family members, especially their grandparents (See Table 5). They also identified their homes and the centres as being safe places. Some children specifically identified educators or people at the preschool, whilst others referred to various objects such as flowers and their beds. Other children, spoke of doing things such as ‘making cakes’ as being joyful and actions such as ‘holding hands’ and not touching hot things as preventative measures against danger and harm. Following the program implementation, children’s perceptions of proximity with parents and grandparents for safety were reinforced. Children’s responses in Centre 2 indicate a close connection with the stories included in the ACBS program. 11 An evaluation of the implementation of the “All Children Being Safe” (ACBS) program in three preschools Table 5: What makes children feel safe? Before ACBS After ACBS Centre 1: My family; mum; dad; nan; pop; preschool, the people at preschool. Centre 1: Being at home with dad; ice-cream; a hug; dancing and playing with friends; being with nan; rainbows Centre 2: Dad, mum, brothers, making cakes, fireworks, home, bed, flowers Centre 2: The mob; (what did the Joey do to feel safe again?) go in mum’s pouch; (Who makes you feel safe?) mum, dad, teachers, sisters, nan and brothers (How do you know you are scared?) NOTE: This change in question is relevant in actually eliciting feelings such as “heart beats faster”; and “eyes looking/ wide open”. Centre 3: Group 1: your dog, mum, dad, pa, poppy, nan/nanny, superheros, teachers Group 2: holding hands so no-one will run over you/so you can’t let go, crawl when you see smoke, walking, you take care of children and don’t let them touch hot things Centre 3: “ Staying with my Nan. Staying with my big cousins or if it is a little one don’t stay with them. My dad.” “ My mum and big sister and my cousin.” “Staying with my big sister and if you are staying with a person you don’t know, go and look for someone you do know.” “ Mummy and daddy.” “ When my mummy holds my hand when I cross the road so I don’t get run’d over.”” When my daddy loves me.” “Daddy and mum.” “Holding mummy and daddy’s hand.” “Holding mummy’s hand and holding Daddy’s hand.” As indicated in Table 6 children’s responses during these discussions, in relation to fear, were to provide a list of words of scary animals such as snakes, tigers, cockroaches and sharks, as well as imaginary creatures such as ghosts and monsters. Some children also referred to natural elements such as thunder and volcanos and others drew on personal experiences of scary events such as fireworks and scary dreams. Following the ACBS program implementation, one week after the conclusion of the last module, the educators held another round of discussions with the children about things that made them feel scared. Their responses at this time indicate a change in the manner in which they responded. As indicated in Table 6 the children’s language and expression, reflect a more complex engagement in the discussion, beyond the use of single words to explain their feelings. That is, whilst the children continued to refer to scary animals (eg dingo) and people (eg ghosts), they extended the discussion to explain the potential signs of danger created when for example, they were not able to find their mum. 12 An evaluation of the implementation of the “All Children Being Safe” (ACBS) program in three preschools Indicators of danger were noted within family homes as well as outside in the community, when it is dark or when alone. During the program implementation at Centre 2, for instance, one child was able to express that his “nana and pop growl” at him and this did not make him feel good. Some children were also able to identify the potential of being hurt by someone and others attempted to articulate the importance of speaking to people about their fears. Table 6: What makes children feel scared Before ACBS After ACBS Centre 1: Ghosts, snakes, thunder Centre 1: When I’m sick. Warki (Ghosts), When I can’t find mum. I never feel scared. Centre 2: Warki or warki man or ghosts, witches, monsters, dragons, sharks, tigers, cockroaches; fireworks Centre 2: Dingo. When someone’s in your room and they hurt you. Bogeyman is in your room. Aunty, what about when we can talk to people and tell them about something scary? I felt unsafe when I went out in the dark; on the road; in the bush; at the park (why?) cos naughty people could hide; bad people might kidnap you – Aunty, what if you wander off on your own?. Centre 3: Group1: a dingo; a gas bottle that erupts up in the sky; a volcano because of fire; crocodiles and a fox; very scary dreams in the night; Swat Fire on Ben Tennyson Group 2: monsters, scary dream, when you wake up and mummy and daddy are not there, mummy and brother. Centre 3: “ When I get lost and someone is not looking for me I phone the police and I will ask them if they can help me. When I do wheelies on mine and Tony’s Peewee 50.” “ A dingo. When I feel like I am going to fall off my bike.” “ When I have scary dreams and when I am falling off a swing and have no seat belt on.” “Scared by a monster, but I don’t like good dreams, I like bad dreams, cause monsters make me feel happy. The monster is a bear, I love him.” “ When I’m sleeping I have a bad dream and it makes me scared.” “ When the lights cut off.” “Snakes “When it’s night time and the light’s are off. “When something happens. When I’m busted.” The children were capable of identifying what or who makes them feel safe secure and supported. Their responses related more to people and things rather than feelings and how your body can tell you that something is wrong. This is due to the wording of the question as well as the need for children to have the language to talk about changes in their body and related feelings. Following the program implementation, which at Centre 2 included talking about bodily sensations, when the educator altered the question to “how do you know you are scared?” children were able to identify safe and unsafe feelings, such as eyes wide open and fast heart beat. 13 An evaluation of the implementation of the “All Children Being Safe” (ACBS) program in three preschools The use of props, such as emotion paddles and books about different emotions, during program implementation at all centres, further supported children to identify different feelings and what made them feel in a particular way. Educators recorded their comments at the end of each session. One child at Centre 3 for example, mentioned that he feels angry when he gets teased. At Centre 1 children talked about feeling happy in regard to being safe and also gave examples of safe feelings, including an excited tummy and feeling warm, 4.1.2. Learning involving safe people and places In the event that children felt scared or unsafe, they were asked to explain what they would do, and who and where they would go to, to feel safe. These responses are presented in Table 7. Table 7: Keeping safe - places and people Before ACBS Centre 1: Run home, or run to dad’s, nan’s, pop or the “mish” (The mission); Hide under a table, tell mum; (go to) Nan’s, Aunty’s, in my bed, to preschool. Centre 2: Kill them, get dad to kill them; hide under blankets; hide in bush; sleep with brothers; turn the light on; run to mum and dad; hide, ring the police/coppers; go and get sad; watch a movie; Go to sleep with mummy; go to bed; my place; go to Port; go that way; run away; Big W; lock the fence; Aunty’s place; motorbikes; cubbyhouse. Centre 3: Group 1: go to mum; if there’s a fire you stop down low; drop down low, roll and get that fire out; go to your house and nan’s; go to your cousins; your pop’s; your nan’s; your sister’s; aunty; police Group 2: go to mum and dad’s bedroom; go to the forest; go and sleep in the bedroom; stay with mummy and daddy; , your grandpa, nana, somewhere safe, on dad’s bus. After ACBS Centre 1: Go to my room; get into bed with dad; hide in a cupboard; tell mum; feel sad; hide and cry for mum; hide under a table; go in the bush. Centre 2: Go in the house; sleep in my room; tell mum and dad; tell my dad if there’s a dangerous thing; ell mum; run home; lock my room; tell sister; teacher; nan; pop; out house; my room at home; yeah with teachers; Centre 3: Call the police, ring a fireman, call 000; put the light back on; run away; go to dad and mum; my sister and brother; go in my bedroom. “If your friend lives next door, I would go there and stay the night.” 14 An evaluation of the implementation of the “All Children Being Safe” (ACBS) program in three preschools Table 8: People identified by children as being safe People identified Mum Dad Nan Pop Sister(s) Brother(s) Cousin Friend(s) C1 C2 C3 ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ Babysitter Aunty/Teacher Police The mob/ my family ✓ ✓ ✓ ✓ ✓ ✓ Fire-fighter Ambulance Nurse Doctor ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ At every centre, there were children who consistently identified their parents, grandparents, sisters and brothers as well as centre staff as representing safe people. The police were the other external agency identified as being a safe reference point by children in all three centres. Safe places identified by children also related to these people (eg. nan’s, home, police). Within the recorded responses provided by the educators, we were able to locate references to cousins and a babysitter only in the data from Centre 3. Likewise, references to friends and family or the mob collectively, were located in data provided from Centre 1 and 2. There was little or no variation in terms of who the children identified as safe people before and after the ACBS Program was implemented in Centre 2 in particular. However, the staff who implemented the ACBS program also noted that at Centre 1, prior to the implementation of the ACBS program, the children found it difficult to grasp the meaning of the words safe and unsafe. The staff used the word ‘scared’ together with body language, to clarify the meaning of some of the pre-program questions at this centre. Subsequently, when the questions were modified to ask the children ‘who will help you or stop you from getting hurt?’ these children also identified a comprehensive list including both family and community members such as doctors, nurses as well as friends. The children readily identified people who keep them safe and who can help them. These responses indicate that, in relation to learning about belonging, the children already had a good sense of this in relation to family. This understanding was strengthened during the ACBS program implementation, particularly in terms of incorporating a sense of belonging to both community as well as family. 15 An evaluation of the implementation of the “All Children Being Safe” (ACBS) program in three preschools Through the program and learning about being safe and unsafe, developing language to better communicate and understand a sense of belonging, children were also seen to develop more empathy, to interact in relation to others with care, and respect. Educators’ observations of children’s interactions and responses during and after the ACBS program highlight a change in this aspect (see Table 9). Reflective comments by educators also further confirm children’s developing communication skills and confidence at this time. The educator at Centre 1 was particularly pleased and impressed by the fact that the program allowed the children to “talk up”. She also felt strongly that having given children a voice we now need to listen to them. Likewise, the educator at Centre 2 explained that there were lots of children with emotional needs and otitis at her centre and this could impact on children’s ability to talk/express ideas. She emphasised “there needs to be a holistic approach; integrated into daily routines” and this could be reflected in asking questions such as “how do you feel today?” Such comments not only demonstrate the importance of the educator’s knowledge of local contexts, especially in terms of the children’s wellbeing and factors impacting on their learning. Table 9: Children interact with empathy Have you noticed any changes in children’s behaviours and/or attitudes that can be linked with the ACBS program in some way? Centre 1 “More nurturing, particularly of younger children in the yard” “Other children are now more confident to talk up, say ‘no’ to more aggressive children”. Centre 2 Children now have the words, a voice, rather than hitting out. Children got the feeling of being safe, reflect on incidents back to animal: I’m a kookaburra, I’m communicating with you. Children all sit together in a mob – social/sports (culturally appropriate). Two boys with issues particularly related to absorbed ideas, related to being a dingo at times; talked later about what he did (hide in cupboard) as not safe. Fire drill – kids would say “safe place”. Centre 3 Children expressed feelings better, understanding of the concept of different feelings - everyone feels differently about things. It’s ok to feel scared or angry; there are different ways of dealing with that. Showing empathy. Having the confidence to speak to an adult. 16 An evaluation of the implementation of the “All Children Being Safe” (ACBS) program in three preschools During the post-program implementation discussions, the children were also asked to indicate what they had learnt from the ACBS program, and these data are presented in Table 10. In documenting these comments, the educator in Centre 3 stated “I think the post-evaluation questions need to be reviewed as my group didn't really answer in detail to match their knowledge. It needs to be clear for all in this section if children can be prompted.” These comments highlight the importance of having educators with appropriate expertise in working with pre-schoolers directing the implementation of the ACBS program. In this way, teaching can be better targeted and children’s learnings can be more meaningful. Table 10: Learnings from the ACBS program as perceived by the children Centre 1 Centre 2 Centre 3 (The question was reworded: what things do you remember from the bush animals in the story?) Kangaroo told dingo to go away in loud voice; echidna had spikes and rolls up in a ball to stab dingo; snakes go in their hole to feel safe; birds have wings and got away to be safe. “Kangaroos were safe.” “Koalas, kookaburras, king parrot, echidna, dingo, baby joey, they’re all in a mob.” “They felt sad because of the dingo.” The babies getting hurt and the baby koala got bitten cause she ran down the tree, she wanted to get the other gum leaves, her mother told her ‘no’ and to stay in the tree, she didn’t listen to her mum, she felt afraid cause she thought her mum would smack her. She told her mum, she felt good after that. Her mum took her to the doctor’s to get a stitch, the baby told the doctor.” “dingo was unsafe” “the kookaburra felt safe he warned dingo was there”. “ The koala climbed down, he wasn’t listening to his mum he was unsafe cause the dingo bit her and he got bleeding, the mum was angry and she wouldn’t climb down ever again, the mum felt sad. “I remember the echidna and the king parrot and the baby king parrot.” “ I know, I remember the koala too.” “and the kookaburra too.” “Echidna, dingo and the aunties and the king parrots and the kookaburras.” “When the galahs went to have the party and the dingo wasn’t invited and he couldn’t eat the babies.” “When the echidna told her baby she’s not safe.” Birds fly a long way, flap their wings, have feathers, helped the animals, are higher than dingo, they’re safe. Koalas have fur, live up in trees, this is their happy place, they are safe. They were sad, they got happy, they got better 17 An evaluation of the implementation of the “All Children Being Safe” (ACBS) program in three preschools 4.1.3 An unexpected learning One of the preschool teachers, with agreement from the ACBS coordinator, used the ACBS puppets with a three year old called TJ. The teacher had wanted to use the puppets with TJ to support him with “language and communication skills regarding the increase in his aggressive behaviour.” The teacher had “spent quality time with him” using the puppets and reported that TJ now responds positively to 1:1 interactions with adults and children. The ACBS coordinator reported “it was very rewarding for the teacher when TJ drew a happy face.” Given below is an extract of an observation note documented by the teacher involving TJ. 26/8/11 TJ’s story He arrived at school with his toy transformer called Bumble bee and sat with me in the cubby house. “Aunty Shirley will you play?” and he thought for a bit before he said “you can be sister”. I said “yes” and he said “I will be Dad”. TJ then said “who will be Mum?” and he asked Aunty Donna, who said “yes”. He told Mum to go and sit in the lounge while he cooked tea, and also any child that wanted to play, had to sit in the lounge to wait for tea. I (SK) asked Dad (TJ) if he wanted me to go to the shop. He said “yes”. I said “do you want bread and milk?”; he said “yes”. He then made me stand in front of him and Dad (TJ) proceeded to button my cardigan up before he let me go to the shop. This type of play went on all morning. He fussed over everyone and if they were crying or feeling sick he would say “take them home”. I found that because he made me sister (SK) I had to listen to him and do what said all morning. He was calm and very thoughtful and his voice stayed soft and caring. This made me respond to TJ very differently to how we normally interact. In all my experiences of childcare, this was the first time I totally became the character and allowed the child to direct and totally control the play. He was so caring and kind and used a gentle voice and watched over us all. The other children would call me Nan or Aunty Shirl, but I would say “No, I’m sister.” They would smile at me and then call me “sister”. This child, TJ, was not formally part of the preschool group that was involved in the ACBS program at this centre. Use of the ACBS puppets with an individual child with language and emotional needs demonstrated an unexpected learning that benefited an individual child in the first instance. This case also highlights the potential benefits of using the ACBS resources in different ways with individual children in guiding their emotional or behaviour needs. 4.2. OUTCOMES FOR EDUCATORS Anticipated outcomes for educators focus on their understanding of relevant child protection legislation, their ability to implement the ACBS program with preschool children, their capacity to involve families in the ACBS program, and their perspectives on the future of the ACBS program. Data for these 18 An evaluation of the implementation of the “All Children Being Safe” (ACBS) program in three preschools outcomes was collected through written responses to a child protection questionnaire and both pre program and post program questionnaires, as well as participation in discussions with the evaluators prior to implementation and at the Community Gathering held in 2011. 4.2.1. Sound knowledge and best practice in relation to child protection legislation as it impacts on educators Prior to implementing the ACBS program it was important to ensure that educators were aware of child protection legislation and issues as they may relate to them in the workplace, in particular mandatory reporting requirements, and responding to a disclosure. Prior to the program being implemented, a brief child protection information session was therefore held with the educators, followed by a questionnaire for them to complete. One of the educators was unable to attend and her questionnaire was completed later with the support of the Program Coordinator. During discussion, each educator was also asked about her participation in more formal up-to-date child protection training, and all educators had attended a formal training session sometime ago prior to 2011. In terms of participation in child protection training, early discussion with each educator provided the following information: Centre 1: the educator felt that staff knew their role in child protection. She would appreciate having child protection training for them though – particularly on guiding children’s behaviour – guidelines, policies, role with parents, and developing a team approach. Centre 2: about half of the staff members at this centre had undertaken child protection training. Centre 3: the educator had done child protection training some years ago, and the other staff at this centre had not done any. Following a brief information session with the evaluators, educators were able to respond appropriately to more specific questions about their role in mandatory reporting of child protection matters as documented in Table 11.. 19 An evaluation of the implementation of the “All Children Being Safe” (ACBS) program in three preschools Table 11: Centre 1 1. Centre 2 Centre 3 What are the 3 pieces of legislation that relate to child protection & what are the features of each? The Children and Young Persons (Care and Protection) ACT 1998. Responsibilities of Community Services, other agencies. Authorised carers, parents. Outlines ways of working with children, young people and families to remain at home. Educators’ understanding of child protection legislation Young Children and Persons Act Mandatory Reporting Ombudsman 2. As soon as you see any sign that the child is not receiving “quality of care” from his/her carer. Children and Young Persons (Care and Protection) Act – mandatory reporters guidelines, what happens when a report is made, the rights of a child, what happens if a child can not stay in care with their family, information about types of child abuse Prohibited persons declaration 3. Working with children check What are the grounds for reporting risk of significant harm? Neglect Physical Sexual Domestic violence Pre-natal Psychological If you suspect on reasonable grounds that a child or young person is at risk of being neglected or physically / sexually / emotionally abused If a child’s basic needs are not being met – are not being properly fed, clothed or housed If a child’s medical needs are not being met If there are signs of physical abuse If a child disclosures information about abuse 3. How would you make a risk of significant harm judgement? Observation of child/parent. Questioning staff if they have a concern. What area of concern I would have. 3. Computerised reporting tree Phone Use the online mandatory reporters guide to help guide decision making. ABC has a team we can ring if we need assistance with child protection issues. They help to guide staff with their concerns and also with the use of the mandatory reporters guide. 4. What actions are required when you suspect a child or young person is at risk of significant harm? Report to the appropriate channels. All of the above in (No. 3) Child Protection office online (Mandatory report) Child Protection helpline 133627 Risk of significant harm report fax form. Documentation Reporting tree Call the Child Protection Helpline Ensure all concerns are documented when they occur 5. What do you do when risk is below significant threshold? Start documenting and collecting relevant information for evidence. Referral to someone that can assist Continue to monitor the situation and document any other concerns as they occur. Provide support and information to the family to assist them develop more knowledge in this area. So the family does not realise the focus is on them I often include relevant information in centre / room newsletters or handouts; complete cooking experiences, healthy eating experiences with the children; access posters, brochures from external services eg: RTA – appropriate car seats. 20 An evaluation of the implementation of the “All Children Being Safe” (ACBS) program in three preschools The information session also focused on responding to a child’s disclosure, as there is a strong possibility of this occurring during the implementation of a prevention education program. Educators were asked about their understanding of this in discussion following the session and they were able to respond appropriately. In post-program questions educators were also asked whether any children had any strong reactions to any aspects of the ACBS program and this did not happen at any of the three preschools involved in this study. . After program implementation the educators had still not been able to attend formal training in identifying and responding to children at risk, and this needs to be addressed. However, each centre does have, or has had, children and families involved in the child protection system, and educators have developed and demonstrated on the job knowledge and skills in child protection practices. 4.2.2. Developing skills in implementing the ACBS program with children at their centres Educators’ responses to five of the pre-program questions and a further two in the post-program questions relate to their confidence and ability to incorporate the modified ACBS program into their preschool program effectively and in meaningful ways with the children. Two of the educators had a distinct advantage in this, having participated in a trial run of the original ACBS program with a group of children in the previous year. The third educator was supported in implementation of the program during 2011 by one of the program coordinators, who also completed the pre-program questions with her. The relevant responses are presented in Table 12. 21 An evaluation of the implementation of the “All Children Being Safe” (ACBS) program in three preschools Table 12: Educators’ pre-program questions Centre 1 Centre 2 Centre 3 Do you anticipate any difficulties in implementing the ACBS program at your centre? No, not really. We will need to consider the 2 year old children in the toddler room, where the 3.6 year old children also share this room. Some difficulties will be the literacy level of the children and how they express their feelings. We would like to run the program in the preschool and kindy rooms, so we are going to have to put a plan in place about how this can be done with me running the program in both rooms. It may be difficult to ensure the kindy group leader is following up on the program and documenting any conversations had with children while I am not in the room. We have 3 Jehovah Witness families in the preschool room, so I am not sure if their beliefs about child protection will affect the implementation of the program with these children. How are you planning to address these concerns? To give the children I will put a written plan into place with myself and the kindy group leader to ensure we are on the same page and she words and feelings understands her role in the implementation of the program. and actions. I will speak to the 3 Jehovah Witness families on an individual level about the program to ensure that they support it Due to the limited concentration span and task persistence of the 2 year old children, we have decided to include the 3.6 year old children into our new room where the 4-5 year old children are. This way the 2 year old children can have quality time with a staff member, while all of the 3-5 year old children can participate in the ACBS program How do you assist children to gather information, ask questions, seek clarification and consider possibilities about their own lives? By consulting their extended families associated Group time At the start of an interest a brainstorming session is held with the children and lists are created: what do you know / what with our service, to support the child with the Show and tell do you want to know. Children’s responses are documented so they can be referred back to and also so the children inquiries at home and to then further extend this Special events and can see that their responses are valued. During the program if we find out an answer we come back to the list, so it can approach by inviting the families to our morning then Yarning, be ticked off. Children are encouraged to research information / explore / hypothesise in a variety of ways including: group time, where the children can share this Gathering participating in hands on experiences, experiments, documenting learning through photos, talking to parents who may information with staff and children. be experts in that field, looking at factual texts. How do you demonstrate that you acknowledge and build on the context and discourse of each child and family? Read their chosen books from home about families Family meetings Informal discussions and parent meetings are held to discuss each family’s values and areas that they feel are important and their culture. Through room and entry displays and expectations, to their child’s development. Through these interactions staff are able to develop a stronger understanding of each and by allowing extra time in the morning for home visits child/family and provide resources and experiences in the program that support and promote these values and goals. greetings and exchanging of ideas, information or Documentation is displayed throughout the room about individual and group work that depicts all children as being identifying individual needs for us to meet. confident and capable learners. All children’s attempts and milestones are celebrated and shared in positive ways with families. Community members from different cultural backgrounds represented at the centre are invited to share in special occasions. In what ways do you support children’s learning and use the cultural tools of the community to inspire all children’s thinking? We employ Indigenous relief workers and Yarning, gathering, Through observing children in their play we gain great insight into their understanding and beliefs about different encourage volunteers to support children’s learning. fire, members / groups in the community. This way we can acknowledge any bias that is occurring and incorporate We plan Indigenous gatherings for parents, families linking to culture experiences into the program to challenge this bias. Eg. we have a mother in the nursery who is a Police Officer. Her visits and extended families in the local community and tracking to the centre generated lots of discussion with the children (mainly the boys) about the role a female police officer plays. invite Indigenous artists who demonstrate their skills dance/songs and Some children didn’t know females could be police and others wanted to know if they could carry guns and do all the for our Cultural Days. elders same jobs as male police officers. These questions and comments were recorded and we decided to ask her to visit our room. The children were very open in asking their questions and she was able to provide the children with responses that acknowledged their thoughts and ideas as being valued and helped them to understand the important role that female officers play in the police force. 22 An evaluation of the implementation of the “All Children Being Safe” (ACBS) program in three preschools The educators’ responses to these questions demonstrate that they considered potential difficulties and ways to overcome them very carefully prior to program implementation. They also demonstrated a high level of professionalism and knowledge about the culturally appropriate ways of ensuring the implementation of an effective early childhood curriculum including the protective behaviours program. Post program responses similarly showed that the educators were capable of providing meaningful curriculum including the ACBS program, For the educator in Centre 1, who was the only one new to the program and was supported by the ACBS program coordinator, there was a marked increase in her level of confidence regarding her ability to implement it in future. While she was unable to attend for the first group discussion with the evaluators, she did participate enthusiastically in the community gathering following the implementation of the program at her centre was completed. One of her staff also wrote a letter in relation to her appreciation of the program and this is included at Appendix 4. Table 13: Educators’ responses to post-program questions Centre 1 Centre2 Centre 3 In terms of your hopes/expectations of this program, what are the highlights of the ACBS program implementation at your centre? Creativity, children enjoying craft, connect with stories. Children have voice – reduce reaction I was not confident at first but now feel much stronger about it (see letter from a staff member in Appendix 4) Engaging parents, let them have more knowledge of what was going to happen. Children’s creativity/words, extend concepts to other people not just mum and dad, connect to outdoors. All staff on board, community involvement. Children move from fantasy to reality – have words. They didn’t have before Creativity of children, great discussions, link to last year (program introduced informally) and safety concerns Of the strategies you used to assist children’s learning about matters covered in this program, which ones were most appropriate and effective? Books, props, craft especially drawing All – use multiple intelligences (Gardiner). All staff involved and all activities even cooking, each person use their strength and also kids learning style Drawing, craft, props – led to good discussion 23 An evaluation of the implementation of the “All Children Being Safe” (ACBS) program in three preschools 4.2.3.Building co-operative relationships with families to promote their participation in the ACBS program Building co-operative relationships with families is vital to early childhood educators’ work with both children and families, and is necessary for promoting their engagement with the ACBS program. Educators’ responses to three of the pre program questions focused on relationships with families. This topic was also covered in our initial meetings with individual educators. The outcome related to engaging parents in the program is covered in the following section related to out comes for families and the community. Early discussion with each educator provided the following information: Centre 1: The participating key educator had good communication skills and relationships with families, was able to talk with Aboriginal people, including males, about concerns about their children’s learning at the centre. When approaching problems and people she was very aware of key aspects - her connection with and feelings about the child; her connection with the family; her responsibility to staff; and her own role as an educator, as well as her responsibility to the Centre’s governing Board and the Government Department which was the licensing authority. Centre 2: Staff visited children’s homes to discuss matters connected with the preschool program at the centre. The key educator at this centre was able to talk with families and respond to their concerns within their own family settings. Parent involvement at this centre was strongly encouraged, along with providing an opportunity for information sharing – eg a Sports Day was organised and included parent workshops. Centre 3: Contact with families at this centre was largely informal, just daily contact with families as they drop off and pick up children. Educators’ responses to the pre-program questions and responses related to building family relationships were collated and presented in Table 14. From the beginning of this project, it was clear that the participating educators at the three preschools were fully aware of the importance of working collaboratively with the children’s parents and the wider community. In completing the pre-program questionnaire, the educators noted the following: All three educators commented about using a variety of strategies to actively engage parents and other family to work in partnership with them. Informal conversations when they came to drop off and pick up children, using newsletters as well as having a day book to maintain communication with families were some of the strategies mentioned. Although only one educator specifically mentioned collaboration with local artists and community elders in their programs, there was other evidence of this occurring in at least two of the centres. The educators also acknowledged that child protection matters required sensitive and respectful ways of communication with families. Their 24 An evaluation of the implementation of the “All Children Being Safe” (ACBS) program in three preschools responses reflect the importance of safe guarding child and family privacy and confidentiality as well as cultural beliefs and practices connected with family backgrounds. 25 An evaluation of the implementation of the “All Children Being Safe” (ACBS) program in three preschools Table 14: Educator pre-program reflections on working collaboratively with families Centre 1 Centre 2 Centre 3 How are partnerships encouraged in your setting? Ongoing, but difficult. We encourage Open door policy – more the merrier, All families are greeted when they arrive at the centre each morning and informal discussions are held about the morning tea gatherings and extra time displays, newsletters, actively invited. child / family eg: weekend happenings, access visits, interests, behaviour, etc. A day book including written for the morning greetings with the documentation and photos is completed each day and the parents are encouraged to read it each afternoon. families. The children are often keen to show their parents photos of themselves in the book completing experiences during the day. Sometimes if a question has arisen during the day during the group, we will write it in the day book and ask our parents if anyone knows the answer or might want to research the question at home with their child. A parent comments section is always included to encourage parent feedback or ideas. What opportunities do you provide for parents to engage with you in ways that respect and value their culture? Knowing that some members of the Parents are always welcome & Information is requested upon enrolment about cultural / religious beliefs and if any are identified by the parents families are Indigenous artists we encouraged to be part of our program they are discussed further to ensure we have a good understanding of their beliefs. Parents are encouraged to encourage Indigenous painting and share their beliefs with the children by joining the group for special occasions, cooking, music / dancing, storytelling, art/craft participation between families craft experiences. Informal discussions are had throughout the year when special events are happening so parents and children. The Elders show enjoyment are informed if there will be changes to their child’s routine. with telling dreamtime stories, dancing and the use of musical instruments from their home life. In what ways do you broach difficult issues that arise and engage in problem solving with families in relation to child protection issues? We have an open door policy, where we encourage Discuss discreetly with parents or community I ensure confidentiality is maintained at all times and that an open and honest relationship is our service users to discuss difficult issues with trained member or report depending on issue developed with all families. I speak to the families involved on an individual level and provide staff in privacy and with confidentiality. Where support in areas needed – finding out more information, developing strategies to be supported agencies are involved we work together networking to at home and preschool, providing information about other agencies that they can get support try and test suitable solutions to all parties involved, from. whilst respecting beliefs, attitudes, family history and culture. 26 An evaluation of the implementation of the “All Children Being Safe” (ACBS) program in three preschools 4.3. OUTCOMES FOR PARENTS AND COMMUNITY Anticipated outcomes for community was primarily taken from the perspectives of the parents whose children participated in the ACBS program that was implemented at the three preschools involved in this study. Data obtained through teacher reflections on parents’ participation and comments made by parents who attended the Community Gathering discussion led by the evaluators, and the responses from the Steering Committee are included in this section as findings. 4.3.1.Parents and community awareness of and engagement in the delivery of the ACBS program Before the implementation of the ACBS program in 2011, the educators were asked what plans they had made in terms of introducing the program to the children’s parents and other family. Their responses documented in the preprogram questionnaire are presented in Table 15. Table 15: Proposed strategies to introduce the ACBS program to families Centre 1 Centre 2 Centre 3 How do you propose to introduce parents/family members to ACBS program? Invite families to a morning tea/ storytime, where the children can demonstrate the use of the kit – with the costumes and the puppets. First engagement with parents and carers was on the 1st of April with a workshop. Then follow-ups with verbal language in the mornings and in newsletters. A parent night is being held at the centre on the 7th April where the program will be introduced to the preschool parents who attend. All the resources will be displayed and the parts of the program will be discussed. Parents will be encouraged to discuss their concerns / questions and how they can become involved eg: ways to discuss the program / safety / words to use with their children. All families in the preschool and kindy room will receive a letter outlining the purpose of the program and when it will be introduced in the rooms. As can be seen, each of the educators had planned to have a function to introduce and discuss with families the implementation of the ACBS program at their centre. They had also planned to display and refer to the use of the ACBS resource kit supplied to each centre. This kit consisted of various items in the shape of the animals depicted in the ACBS stories, including costumes, puppets, songs and various art/craft materials to be used with the children. Each centre also informed families about the introduction of the ACBS 27 An evaluation of the implementation of the “All Children Being Safe” (ACBS) program in three preschools program by letter, although only one educator had specifically documented this in the survey questionnaire they completed for the evaluation.. When the actual implementation took place, the educators and parents reported a mix of responses regarding the strategies that eventually occurred at each centre in terms of engaging families in the ACBS program. Table 16: Strategies actually implemented to engage families Centre 1 Centre 2 Centre 3 How did parents & other family members get involved with the ACBS program at your centre? Disappointed in self for not getting families involved more. Families knew it was happening but did not fully understand it. Hard to get them to talk; informal conversations only. Need to get information out to families. Community meeting to explain program to families. Sent permission notes, and did home visits – reassure that program can translate to other safety areas. At the end, held another community meeting with about 50% attendance. Parents took books home. Held a celebration with a 2 day festival concert during NAIDOC week Families read documentation. Some talk about children mentioning safe. More road safety/ car seats., asked Green Hills school to participate. As indicated in Table 16, Centre 1 had struggled to engage families in the implementation of the ACBS program. As noted, although families at this centre were informed about the program, no one had turned up at the morning tea planned, and the conversations remained at an informal level with only a few parents expressing an interest in the program. Also, no parents from this centre attended the Community Gathering arranged in August, inviting them to meet with the evaluators to discuss their participation in the ACBS program at their children’s preschool. In contrast, parents and other family were more involved in the ACBS program implementation at Centre 2 and 3. Both centres held a workshop to inform families about the program, prior to its implementation. Centre 2, included home visits and a second family function, aligned with NAIDOC week celebrations. It is also important to note that Centre 2 connected with the local primary school, where the ACBS program had been implemented originally. As indicated during the Community Gathering meeting in August 2010, this connection with the Primary School had been timely and fruitful as the school had presented this Centre with a box full of costumes of the ACBS animal 28 An evaluation of the implementation of the “All Children Being Safe” (ACBS) program in three preschools characters they had once used. It appears that having multiple copies of good quality costumes enabled more children at this centre to actively engage in the ACBS stories at the same time. Video-based data clearly depicted lots of children dancing, singing and dramatising stories from the ACBS program at the NAIDOC day celebrations at this centre, in the presence of many families. 4.3.2. Sharing the ACBS program with children at home A Community Gathering was held over lunch in August, to ascertain parent and family perspectives on the implementation of the ACBS program at the three preschools that participated in this project during 2011. Three parents – two mothers and a father, representing Centre 2 and 3 attended this event. When asked to comment about their awareness and involvement in the ACBS program at their children’s preschools, their responses indicated the following: Mother 1: This mother noted conversations with her child who was “excited about the program”, and mentioned her “talking about the dingo.” This mother also mentioned attending the parent meeting where the educator had provided information about the ACBS, and used the puppets and spoke about writing books using the characters from the stories. This mother also mentioned that her child had participated in the program in 2010, and giving the educator feedback on the planned modifications for 2011. She also “heard other families talking about it at the pool” where she worked, adding, “information doesn’t always reach everybody.” Mother 2: She described her child talking about XX the educator introducing the program to the children; and her child’s awareness of asking for permission to walk on the streets. She also noted the permission note sent to parents, and the workshop talk about keeping safe with children and what goes on in their lives. She was overwhelmed by how much more knowledgeable her child was about safety when walking on the streets. Father: He was very aware of the program and stated that it complemented the usual work at this centre. He explained, “My relationship with XX is both personal and professional.” His own job involved dealing with family violence issues in the Macleay valley and he therefore recognised “the importance of educational programs to act as a preventative measure to break the cycle of abuse” in the local community. These comments from the parents denote consistency with the educators’ reflections documented earlier, and suggest a close connection each parent had with their children’s preschool centre. All three parents referred to conversations with their own children about the program, reflecting a positive acceptance of the learning that was occurring, particularly in relation to community safety and security. For instance, 29 An evaluation of the implementation of the “All Children Being Safe” (ACBS) program in three preschools Mother 1: She commented that the ACBS program provided “a good way to bring up the topic of safety” adding "You can't just ask who is a bad person because it could be a dad or an uncle." Mother 2: She explained that her child had “a lot of fears” and “talking about the animals” in the stories helped with talking about these fears, and that her child has now “come out of her shell”. As can be seen these parents felt that the ACBS program provided an appropriate space to discuss difficult issues. They were aware of the existence of abusers or pedophiles in the community, with the father stating “many adults” were themselves “interfered with as children”. As such he asked “how do they then tell this community to change?” He went on to state that in attempting “to get pay back some have gone to jail. For me, it's a matter of importance how we keep our children safe. Let them know we’re on to them.” This father described the preschool as “a safe place and there needs to be a comprehensive protection measure for this centre." This father also articulated the value of the ACBS program from a broader educational perspective. In this regards, it is worth noting that each parent also referred to their children talking about the ACBS program with friends who came to play with them at home. The father described this as “an example of the extension of the learning beyond X (the preschool) to other children who are not at this centre.” At the same time, it is important to keep in mind the comments made by one of the mothers above that not all parents or families may be fully informed or aware of the ACBS program. The father described himself as “a community driver” rather than as a parent who was involved in the preschool directly. He saw this role as necessary because there were “people who think of child abuse in black and white terms or act as if they were crabs in a bucket - struggling to establish themselves as leaders. The mud-crab mentality means that there are unnecessary fights all the time. To be accepted by community there has to be high level people supporting and endorsing the program, and at the community level, not to knock it. Given the rate of sexual activity within the community, it is important to find ways to keep the children healthy and safe. In this way, children can have a childhood.” The staff at this Centre acknowledged the key role provided by this father by saying “having X has meant having a strong uncle to stand up for the children.” 4.4. PROGRAM HIGHLIGHTS AND FUTURE DIRECTIONS As a part of the evaluation, participants were also invited to comment on the effectiveness by identifying any highlights and by providing suggestions for implementation of the program. All participants, including the children, provided feedback on this: the children commented on what they 30 An evaluation of the implementation of the “All Children Being Safe” (ACBS) program in three preschools liked/disliked; the adults, comprising the educators, parents, the Project Steering Committee members and the program coordinator and manager, all had suggestions to offer in terms of improving the overall quality of the ACBS program. 4.4.1. Program highlights and ideas for improvement One month after the completion of the program implementation at each centre, the educators engaged the children in a round table discussion to ascertain their views on what worked and didn’t work with the ACBS program The children took this up seriously and offered the following feedback: Table 17: What the children LIKED about the ACBS program Centre 1 Centre 2 Centre 3 Reworded- what did you like about what we did when I came here?)) The dress ups, puppets, birds we made, painting, dancing and flying around the room, kangaroos, spiky echidna felt yucky, singing, bird clothes, dingo is sneaky and growly, playdough, drawing dingo, birds had a party because they were happy, dingo was hungry/had spiky teeth, alligator song, chocolate things (Use face paddles) “ Kookaburra and the kangaroo, dancing, puppets.” “Dingo tried to catch the baby joey and the daddy kangaroo told the dingo to go away. Warns all the animals”. “The celebration, we dressed up, danced, showing our costumes to our mum, dad and family.” “I liked all of it, dressing up, puppets, play dough echidnas and spikes and quills.” “Lillie Pillies, their safe place was the trees cause the dingo can’t climb. The echidna called help.” “ The stories, the smartboard, the songs, dancing, dressing up, activities – spikey echidnas, made kangaroo paper bag puppets, painted a person in my family that was mum, lego. Safe houses, draw a king parrot, kangaroos, koalas, echidnas. We also made hand puppet s and posters.” “The stories, dress up as a kangaroo and I was hopping.” “ The kangaroos and the kookaburra and the dingo and the echidna. The king parrot, the koala, the mummy kangaroo with the baby and the kookaburra.” “ “The echidnas.” “ When the dingo didn’t come to the party with all the babies. When the babies were safe.” “ When the dingo scares the babies.” “When the dingo bites me.” “About when the dingo tried to climb up the tree, but dingoes can’t climb trees. And when the mummy tried to keep the baby kookaburra safe.” M & C shrug shoulders; no response The children’s responses across the three centers were rather similar as can be seen in Table 17. Typically the children identified with the positive animal characters of those such as the kangaroos and the koalas, with some children referring to them by using family terms such as daddy or mummy kangaroo. It also appears that the children enjoyed the learning activities and the resources supplied with the kit, such as the face paddles, puppets and dress ups. 31 An evaluation of the implementation of the “All Children Being Safe” (ACBS) program in three preschools Table 18: What the children DID NOT like about the ACBS program Centre 1 No, dingo was scary, my dad’s strong like the kangaroo I liked that story Centre 2 Centre 3 “The dingo.” “The koala being bitten on the foot, I didn’t like that.” “The dingo and the poor little echidna and the other animals felt unsafe. The dingo tried to get them but he got the baby koala.” “The spikey quills. The koala got bitten by the dingo on the leg. I felt sad. I felt happy when she told her mum.” “When the koala went down the tree, the dingo cause he would eat me.” “ The dingo and the blood on Dingo’s foot from the spikes.” “ When Dingo bit Nanny Koala’s leg and when the koala slept and slept and didn’t have anything to eat.” “ The echidna was very good when he rolled up in a ball and when the dingo came, a spike got in his foot and all blood came out.” “ When Dingo didn’t come. I didn’t like Dingo biting them and eating them.” “ I don’t like dingoes.” “ Dingoes.” “When the dingo tried to eat the koala.” “When the dingo was trying to eat the little baby kookaburra.” As can be seen from the children’s responses documented in Table 18, in terms of identifying elements that they did not like about the ACBS program they focused on the animal characters and their actions in the stories. Typically the children were able to refer to negative behaviours that may be described as harmful or dangerous. For instance, the children expressed fear, sadness, disgust or abhorrence of biting and blood. Some children also expressed concern or were able to empathise in terms of the wellbeing of the baby kookaburra and the koala. In turn, the educators and the parents also had the opportunity to reflect on what they considered to be the highlights of the ACBS program implementation for them, as adults. As can be seen from Table 19, all educators highlighted children’s creativity. Each of the participating parents referred to children’s growth in confidence when talking about potentially scary topics using words such as safe and unsafe. The educator at Centre 1, where the ACBS program was run for the first time in 2011, acknowledged her own sense of confidence due to the success of this year’s implementation. The father who was present at the Community Gathering in August also felt that the ACBS program should be made available to all preschools: "every school should be doing this program with every child. There should be a list of identified behaviours and talk to parents about it as well." This was seen as a 32 An evaluation of the implementation of the “All Children Being Safe” (ACBS) program in three preschools way of removing the taboo around child abuse. He believed that “the whole of community approach” had the potential to enhance child protection. Table 19: Program highlights - perspectives of preschool staff & parents PRESCHOOL STAFF PARENTS Centre 1: “Creativity, children enjoying craft connected with the stories. Children have a voice” and this can help “reduce negative reactions.” The preschool teacher at this centre who was not confident at first now feels much stronger about running the ACBS program. Mother 1: She stated that her child was “more aware of strangers”, adding that the child has “pulled back on this” and “has more confidence now.” Centre 2: “Engaging parents” and providing more information about “what was going to happen”. Children’s creativity/words, extending concepts to other people not just mum and dad and connected with the outdoors. Having all staff on board, and community involvement. Seeing how the “children move from fantasy to reality. They have words they didn’t have before.” Centre 3: “Creativity of the children” added that they had some “great discussions” and were able to use words and “link to last year and safety concerns” Mother 2: She explained that her child was “now more open about learning” and knows what is real and not with regards to her “fear of the dark” in relation to “spirit stories and ghosts” (warki man); that she has noted children using the language of “safe and unsafe”. Father: Children’s awareness of what is not/appropriate behaviour by referring to the dingo, stating “don’t be a dingo”. Children’s use of words “safe and unsafe”. The Project Manager, Coordinator and the Steering Committee too were invited to comment about the highlights of the ACBS program from their perspective. In describing their highlights, the Project Manger and the Coordinator noted the following: Inclusion of an early childhood educator as a part of the project coordination team enhanced the two way learning between the centre staff and the project managers. Development of the ACBS program Resource kit – including the puppets, costumes, resource cards depicting different emotions etc Growth in children’s confidence in using appropriate language to talk about safety; watching children sitting, engaged, ready to listen and improved their behaviour Growth in staff’ confidence in Centre 1 in particular where it took the teacher some time to build trust, and in Centre 2 also. It is important to note that at Centre 1 and 3, the two educators were relatively alone in implementing the ACBS program, in comparison to Centre 2, where there was centre-wide commitment involving all staff and parents. In documenting their highlights, one member of the Steering Committee stated: 33 An evaluation of the implementation of the “All Children Being Safe” (ACBS) program in three preschools Although I have not delivered the program at the pre-schools, I have heard positive feedback from both South Kempsey and Dalaigur pre schools, in regards to the children’s’ better understanding about being safe, how to stay safe, and identifying someone who they feel safe with. The other Steering Committee member who had been involved with the ACBS program for three years, described it as a wonderful achievement for a partnership program with different services from Kempsey. Our town is often portrayed negatively, and I think this program highlights that we can develop and provide a quality resource and program to support the children of Kempsey. Having discussed the program implementation with each educator, this committee member also noted: It is a credit to the three preschools for the outcomes they have been able to demonstrate during the trial of the program. I have seen the photos and the extensive detailed information from Centre 3 – the feedback the children gave to X (the educator) was amazing. My discussion with X from Centre 1, was positive and the children really had enjoyed the program and activities. X (teacher from Centre 2) has been excited about implementing the program for the second time, the cultural aspects of the stories were able to be linked with the preschool environment and new ideas have been able to be included in the development of the ‘how to manual’ which is great. 4.4.2 Suggestions for program improvements When asked to comment about any program improvements, the educators offered no specific suggestions, stating instead “no, loved it all” and “no – all links well to community issues”. When asked to comment about what they would exclude or include from the existing program, there was agreement among educators that the introduction was too long to be completed in one session; Centre 1 did it over two days and Centre 2 and 3 did it over a full week. At the Community Gathering, one of the teachers at Centre 2 also added “An overview of the stories and the characters can be helpful for parents as well as health workers.” When asked about suggestions for improvements, the Project Coordinator and the Manager felt that it was important to “include a trauma framework” and to ensure that the “program was adaptable and flexible” so that it can be adapted appropriately for the context of delivery. It was also noted that the “Introduction session was too long for preschool children” and requires “adaptation” and shortening, and suggested a section with “expectations and outcomes broken down in steps.” They emphasised the importance of maintaining the “core fidelity of the program with add-ons or suggestions as to how to build on” what was listed in the manual. There was also a need to look into making the pre program activities with families more appropriate so that they can be encouraged to engage in meaningful way. 34 An evaluation of the implementation of the “All Children Being Safe” (ACBS) program in three preschools The Steering Committee members offered the following suggestions for improvement: Steering Committee Member 1: Having parent days where the children perform the stories and staff are then able to discuss in more detail the effects of CP issues have on their child/ren. It may also help to have someone like a Child Psychologist to give his or her expert advice at these days. Steering Committee Member 2: I think resources could be developed for parents or extended family members; it could be in the form of flyers, brochures, stickers, magnets, songs, rhymes, and children’s performance. I think if we could develop a play where children were able to dress up in the costumes and put on a performance for parents you would have ‘lots’ of parents participate, as they love seeing their children perform. The other idea could be to give the children a ‘certificate of participation/ completion etc.’ for each individual as they have completed the ACBS program, or some kind of display item that could be displayed in the child’s home. Eg. The door hanger, which is in the primary school manual. I would like some more ideas on craft, particularly if the program is done with one group and then introduced the following year with similar kids. For the services that have the interactive whiteboard technology (IWB), I would like some games developed for the IWB or perhaps puzzles using the animals specific to the story, songs, rhymes and posters. At the Community Gathering, the participating parents offered the following suggestions as a way of strengthening children’s learning about safety: Mother 1: There needs to be “something to teach parents and the wider community” (especially other professionals such as doctors and the police) “to use the same language as in the ACBS program (eg don’t behave like a dingo etc). We all need to know what each animal represents (eg kookaburra is the communicator). Involve other professionals working in places where children go such as the dentist and the doctor etc”. Mother 2: Implementing the program with “animals make it easier for children to understand and for adults to explain” (difficult issues about safety). However, there is a “need to be careful about how these animals may be viewed in different communities. For instance, the dingo may be a friend in another community”. Father: Children also “need to learn more about the actual animals and their real lives.” May be after the program is implemented “take the children to the koala park, billabong park etc to see the animals”. During this conversation also, staff at Centre 2 noted that it was important for staff to “take care of children who may disclose”. One of the staff commented that “domestic violence (DV) is a big issue here, and a few kids talked about DV.” She explained, “For some children, the koala had a bigger impact because it was actually bitten, and DV issues came out then as 35 An evaluation of the implementation of the “All Children Being Safe” (ACBS) program in three preschools someone was actually hurt. Children knew to tell parents anything even if it was wrong through this program.” The Project Managers also noted that at Centre 1, they had observed how the presence of DV issues in the children’s lives made them more unsettled in the morning. The teacher at Centre 3 also noted that during the program implementation, “some children felt the police were unsafe: put you in the cage”. The teachers at Centre 2 also commented “the police come into our children’s lives a lot.” They do know that the “police protect us” but need to “give them a positive role”. Participants’ perspectives on future directions for the ACBS program 4.4.3. When discussing future directions for the ACBS program, participants at the Community Gathering were in agreement that information about the ACBS program needed to be disseminated widely, beyond the preschools. In order to achieve this, participants suggested several strategies including the following: Use of t-shirts and community posters highlighting the ACBS program, what it was about, presenting koalas as helpers and urging people ‘don’t be a dingo’. One parent suggested that these posters could be placed in child and maternal health clinics and include the stories as a community resource available for various professionals. Use of “key people or representatives from the community to talk about the ACBS program in different areas.” "Medical teams can be introduced as safe people, and this can include the dentists.” This can build rapport between the different personnel that the children go to see within the community. The children will in turn be made to feel safer because these adults “will know what they are talking about” when referring to the storybook characters from the ACBS program - especially when it comes to sexual assault that involves family members. Use the Centre’s newsletter to promote ideas about child safety connected with the ACBS program. For example, use of symbols and slogans of safety such ‘a safe pair of hands’ and words ‘don’t be a dingo’. In documenting their responses to the post-program questionnaire, the educators also commented as follows: Centre 1: This educator wanted to pursue “more parent involvement”. She stated that this year’s implementation “gave children a voice ‘to talk’ or ‘speak up’ about child safety; and now we need to listen”. The teacher felt “very positive about it” and indicate she would “like to do things like X did (at Centre 2).” She also added “makes me think and react to children differently – really hear them”. Centre 2: “Feedback from previous families can encourage others. Do need to involve all staff, give them a group, and use their strengths. Community and family involvement means extending on school transition” as well as 36 An evaluation of the implementation of the “All Children Being Safe” (ACBS) program in three preschools working with other professionals such as “dentists and healthcare works; all need to be involved.” Centre 3: Reinforced the need for “more staff involvement” so that the ACBS program is not seen as “a one man band”. When the question was presented to the Steering Committee via email, the two who replied stated the following: Steering Committee member 1: Program to be run regularly at the ECC, ongoing promotion especially at parent information days, training for new staff is better done hands on when happening at the centers. Steering Committee member 2: Getting the key adult stakeholders “back together and brainstorm some ideas” including additional “craft activities” Inclusion of “photos and findings from the trial programs in the new Manual” to be produced “If the final conclusion of the evaluation is successful then I think that it would be really good if this program could be presented at early childhood / school forums or conferences to share the program to support other communities. I think a resource for parents could be developed to share the information about protective behaviours/child protection and the ACBS program. It could be included in the manual as a photocopy resource (so that it is uniform in content) as the information given to parents would be quite different depending on each service.” This person also described the ACBS program as modified and trialed in these preschools as “an early intervention program” and that it presented “a gentle introduction to the concepts of protective behaviours in young children. It has the potential to grow and further develop on several different levels with drama or music/song activities. It is great that we are able to enhance children’s resilience and knowledge about keeping safe at a younger age.” Accordingly, she was the view that this program “has the potential to grow and be developed for other communities – perhaps on a National level!” 5. DISCUSSION AND RECOMMENDATIONS The success of the ACBS program is due in part to the commitment of the NAPCAN program mangers to collaborate with members of the local Indigenous community throughout the development and implementation of 37 An evaluation of the implementation of the “All Children Being Safe” (ACBS) program in three preschools the program. Considering that there are plans to expand the program to other Indigenous communities throughout Australia, the same depth of consultation and collaboration with each local community will be vital. Based on the analysis of the findings of this evaluation, in offering our recommendations, the discussion has been organised under four subsections: 5.1 5.2 5.3 5.4 5.1 Program development Knowledge and skills of the educators Extending children’s learning Following up on participants’ suggestions Program development The perception of bush animals captured in the ACBS stories can vary across communities. That is, the dingo which is portrayed as an unpleasant character in Kempsey, may be regarded as a friend or a protector in another community. Therefore when using these stories, educators must take into account local perceptions in adapting the program for implementation within each community. It is difficult to predict how early childhood educators in other communities will adopt or adapt the ACBS program and resources developed specifically for Kempsey. Much of the success of this program can be attributed to the key roles played by early childhood educators, who had a sound knowledge and understanding of working with pre-schoolers. Having a project coordinator who was an experienced early childhood educator ensured that the program structure and the accompanying resources contained in the modified ACBS program were well designed for use with pre-schoolers. Awareness of Indigenous cultural knowledge including stories and beliefs about child safety, were used effectively in both designing the program as well as in the implementation experiences. Involvement of the parents and community members, in particular the elders and community artists, provided authenticity for the teaching and learning that took place in this community. Overall, this evaluation reinforces the importance of thorough consultation with and involvement of the local Indigenous community in each area where the program is to be delivered in the future, to ensure that any adaptations for the particular local community are adequately made. Accordingly, it is suggested the following aspects be built into the program guidelines that accompany the ACBS program manual. Recommendation #1: That NAPCAN recommends the appointment of an early childhood educator as the coordinator of ACBS project development and management in each community. 38 An evaluation of the implementation of the “All Children Being Safe” (ACBS) program in three preschools Recommendation #2: That in planning, designing, implementing and evaluating the use of the ACBS program, consultation with the community is built into each community’s approach to the program. The way that the program was implemented in each of the three preschools varied, taking into account the particular organisational circumstances of each centre. It was also clear that the program was a success in each of the three centres in different ways. The educators who implemented the program, were strongly of the view that program implementation was strengthened when all staff, parents and the community were actively engaged. They also agreed that adopting a whole of centre approach so that the program embraces every playroom, can be more meaningful and helpful when attempting to engage families in a challenging initiative such as “keeping all children safe”. Recommendation #3: That when delivering the ACBS program in any early childhood setting, emphasise the importance of collaboration amongst staff in planning and implementing the program in a holistic way. 5.2 Knowledge and skills of educators The establishment of positive rapport between educators and children is essential for introducing and continuing questions, with children being empowered in their responses. The context of questions needs to be meaningful, while open- ended questions will encourage much longer responses than closed questions. Changing questions rather than repeating them exactly the same way, can also assist children to respond in meaningful ways. (Westcott & Littleton, 2005, p.151) There were times during discussions with children that educators needed to re word questions or at least change a particular word in order to elicit a more meaningful response. Educators need to be competent and alert to this need, as well as having a sound knowledge of the children they are working with. The educators in this evaluation were very skilled at questioning children appropriately, knew their children well and had developed good rapport with them prior to the ACBS program being implemented. The evaluation instruments can be used as teaching/learning tools as was done in this project. However, as also noted, when working with children, at times it was necessary to adapt the questions to make them more meaningful/relevant to the children. This reinforces the importance of having well qualified educators to lead the pedagogical discussions with children and families. 39 An evaluation of the implementation of the “All Children Being Safe” (ACBS) program in three preschools Recommendation #4: That NAPCAN ensures that future ACBS programs are implemented by capable educators who can establish good rapport with the children and families participating in the programs. Recommendation #5: That NAPCAN provides an orientation session at commencement and finds ways of providing adequate support and guidance for educators during program implementation. Ethical considerations are key to interviewing children. When the topic or the focus of the interview involves discussion of concepts concerned with child protection matters, the possibility of a child disclosing abuse requires special consideration. (Hill, 2005, p.74) The evaluators and a parent at the community gathering highlighted children disclosing abuse, and the need for educators to respond appropriately to a disclosure. These concerns are also reflected in relevant literature, as there is a strong possibility when talking with and questioning children about safety and protection. This reinforces the need for training and heightened awareness of these issues prior to educators implementing the ACBS program. Early childhood educators are mandatory reporters and are required to undertake training in responding to child protection matters. As noted in the presentation of results, the educators in the ACBS study were specifically questioned about their knowledge of child protection legislation, where there was a major focus on ensuring their ability to manage a disclosure. Recommendation #6: That all educators involved in the implementation of the ACBS program have thorough prior training in their child protection responsibilities and with particular attention to ways of responding to a disclosure. Provision of appropriate professional development for staff implementing the program is obviously a high priority, and this has been supported elsewhere. For instance, general trends found among studies of prevention programs, in an Australian national audit of child protection research, were the need for: Adequate training for program workers; longitudinal studies to assess the long term impact of programs; and continued follow-up sessions of program support to refresh parents, families and children and also to assess their progress. (Higgins et al. 2005, p.23) Likewise, as the ACBS program implementation continues, NAPCAN may want to adopt a long-term strategy to monitor developments in different communities. Recommendation #7: That NAPCAN includes in the ACBS manual, program evaluation instruments used in this evaluation to enable early childhood settings to do their own follow up by assessing earnings with children, parents and community. 40 An evaluation of the implementation of the “All Children Being Safe” (ACBS) program in three preschools Recommendation #8: That NAPCAN develops a longitudinal research study to assess the long-term impact of the ACBS programs across communities. 5.3 Extending children’s learning During this project, a variety of creative methods were used to make the ACBS program more attractive to pre-schoolers. In particular, the educator’s resource kit was a particularly useful inclusion as these resources “give reign to children’s imagination” and was useful in understanding the children’s experiences. (Greene & Hill, 2005, p.14) Props and planned learning experiences such as drawings, writing, drama, role play and puppets, can “engage children’s interest, foster thought and reflection, and soften the effects of high-control, adult-dominant, question-and answer format.” (Brooker, 2001, p.166) It can be said that the implementation of the program benefited greatly from the inclusion of costumes and hand puppets in the resource kit that was supplied. The physical presence of these resources enhanced the whole of centre approach in the way the ACBS program was implemented throughout all the rooms at Centre 2 in particular. Children’s like and dislikes about the ACBS program activities suggest that the range of experiences offered was appropriate and enriching. It would be important to retain the diversity of experiences across key learning areas of the curriculum, demonstrating a mix of experiences that promote cognitive and social skills as well creativity and physical movement, literacy and numeracy. Recommendation #9: That the ACBS educator’s resource kit supplied by NAPCAN includes both puppets and costumes for role-play and drama activities. Recommendation #10: That when developing further suggestions to use with the ACBS program, NAPCAN considers including activities and experiences that cover a broad range of learning and skill development. 5.4 Following up on participants’ suggestions Both staff and parents were comfortable talking about child safety issues using the stories from the ACBS program because they were culturally appropriate, and allowed them to talk about confronting issues with reference to the animal characters in the stories. These stories in effect were one step removed from talking about real people, thereby making it easier for the adults to raise these issues. The suggestions for program improvement reflect the participants’ view of the need to make the learning from the ACBS program to be extended or taken beyond the preschool settings into the broader community. 41 An evaluation of the implementation of the “All Children Being Safe” (ACBS) program in three preschools All stakeholders offered a variety of suggestions for improvement and future directions as documented in the results section of this report. These suggestions have been summarised and categorised in two ways as follows: Suggestions for: 1)Program planning that NAPCAN can consider in rolling out the ACBS program to various communities; and 2)Local operational suggestions that individual communities can consider when implementing the ACBS within their local early childhood settings. These suggestions can be presented in the Early Childhood ACBS manual as a list of learning activities for communities to consider. Recommendation #11: That NAPCAN gives due consideration to the follow up suggestions offered by everyone involved in the Kempsey study, as listed in this report. As evaluators we have deliberately opted not to prioritise these suggestions as we believe, it would be more appropriate to discuss these with centres as well as other stakeholders, especially the Project Steering Committee members. In this way, centres in particular can ascertain the views of those involved in their own settings and define priorities in terms of local needs, resources and constraints. Program Planning suggestions for NAPCAN: Inclusion of a trauma framework to support the care of children who may disclose and looking after educators who may receive a disclosure. Adapt and shorten the introduction, breaking down outcomes and expectations into manageable steps. Strengthen the adaptability and flexibility of the program, and allow for revisiting and follow up sessions, include suggestions or add-ons for building on the activities listed in the manual – get key stakeholders together to brainstorm ideas Advice on developing community resources on keeping children safe for use with family members, the community, and other professionals working with children such as doctors, police, child and maternal health clinics. These could include flyers, brochures, posters, stickers, and T shirts. The aim of these resources is to encourage all concerned to use the same language (such as “don’t be a dingo” and “koalas are helpers”). Develop a resource for parents about child protective behaviours for inclusion with the manual as information to distribute to families. 1. NAPCAN to present the ACBS program at school forums and relevant conferences. 2. Include photographs and evaluation findings in the new manual. Local operational suggestions: Consider ideas for making pre-program activities more appropriate for families to encourage their engagement with the program in meaningful ways. (For example, use feedback from other families whose children have already participated in the program.) 42 An evaluation of the implementation of the “All Children Being Safe” (ACBS) program in three preschools Include parent/community days at the end of the program, where children perform the stories, as at Centre 2, and possibly involve a child psychologist for advice. Develop a certificate of participation or other display item, for children involved in the program. Promote child safety ideas and the slogans about the ACBS program messages in centre newsletters. (For example, appropriateness of the use of the slogan “don’t be a dingo” can vary within each community.) Include follow up visits to animal parks so children can learn more about the actual animals in local communities. Use key community representatives to talk about the program in different places with different people. In conclusion: This evaluation clearly indicated that adopting a whole of community approach can ensure that learning about child protection and safety can be examined in an integrated way across the community. The ACBS program is a powerful vehicle that can provide a cohesive approach to our work as early childhood educators in ensuring children’s safety and wellbeing beyond preschools, and involving both parents and other professionals who provide services to young children and their families. Having the same set of educational resources in each community, also means having a common language and an integrated approach to child protection issues practiced by a range of professionals from diverse disciplines representing education, health and welfare sectors. Such an approach can thereby also make learning about keeping children safe sustainable as a lifelong commitment by all involved. 43 An evaluation of the implementation of the “All Children Being Safe” (ACBS) program in three preschools REFERENCES Australian Children's Education and Care Quality Authority (ACECQA). (2011). Guide to the national quality standard. Retrieved from http://acecqa.gov.au/storage/3%20%20Guide%20to%20the%20National%20Q uality%20Standard%20FINAL.pdf. Berlyn, C., Bromfield, L. & Lamont, A. (2011) Child protection and Aboriginal and Torres Strait Islander children. National Child Protection Clearinghouse. AIFS: Melbourne. Brooker, L. (2001) Interviewing children. In G. Mac Naughton, S.A. Rolfe and I. SirajBlatchford (Eds.) Doing Early Childhood Research. International perspectives on theory and practice. Sydney: Allen & Unwin. Department of Education Employment and Workplace Relations (DEEWR). (2009). Belonging, being & becoming: The Early Years Learning Framework for Australia. Canberra, ACT: Australian Government. Retrieved November 22, 2010 from http://www.deewr.gov.au/EarlyChildhood/Policy_Agenda/Quality/Documents /A09057%20EYLF%20Framework%20Report%20WEB.pdf Evans & Peck Services. (2008) Bravehearts Inc Program Evaluation. Ditto’s Keep Safe Adventure Program. FAHCSIA (2009) National Framework for the Protection of Australia’s Children. (retrieved November 2010) (www.fahcsia.gov.au/sa/families/pubs/framework_protecting_children/Documents/ sec2.htm) Greene, S. & Hill, M. (2005) Researching children’s experience: methods and methodological issues. In S. Greene & D. Hogan (Eds.) Researching children’s experience. Approaches and methods. London: Sage. Higgins, D.J. (2010) Community development approaches to safety and wellbeing of Indigenous children. National Child Protection Clearinghouse. AIFS: Melbourne. Hill, M. (2005) Ethical considerations in researching children’s experiences. In S. Greene & D. Hogan (Eds.) Researching children’s experience. Approaches and methods. London: Sage. Lamont, A. (2009) Evaluating child abuse and neglect intervention programs. National Child Protection Clearinghouse, AIFS: Melbourne. 44 An evaluation of the implementation of the “All Children Being Safe” (ACBS) program in three preschools NSW Government, Department of Community Services (2009) Working with Aboriginal people and communities. A practice resource. NSW Government, Department of Premier and Cabinet. (2010) Keep Them Safe. A shared approach to child wellbeing. Annual Report 2009-10. Tomison, A.M. & Poole, L. (2000) Preventing child abuse and neglect. Findings from an Australian audit of prevention programs. National Child Protection Clearinghouse, AIFS: Melbourne. Westcott, H.L. & Littleton, K.S. (2005) Exploring meaning in interviews with children. . In S. Greene & D. Hogan (Eds.) Researching children’s experience. Approaches and methods. London: Sage. The “All Children Being Safe” Program (Green Hills Pilot Program). North Coast Area Health Award 2009. (retrieved August 2010) www.awards-expo.health.nsw.gov.au 45 An evaluation of the implementation of the “All Children Being Safe” (ACBS) program in three preschools 46 An evaluation of the implementation of the “All Children Being Safe” (ACBS) program in three preschools APPENDICES Appendix 1 Modifications Made to the ACBS Preschool Program Appendix 2 Evaluation Framework Appendix 3 ACBS Program Evaluation Instruments Appendix 4 Letter of Thanks from Preschool Staff 47 An evaluation of the implementation of the “All Children Being Safe” (ACBS) program in three preschools Appendix 1: MODIFICATIONS / CHANGES MADE TO THE ACBS PRESCHOOL PROGRAM BY MICHELLE ROSE 1. The ACBS Preschool Program has been linked to The Early Years Learning Framework and linked to the seven areas of the National Quality Standard, which contribute to the quality of early childhood education and care. 2. Included animal/bird puppets and dress up animal/bird costumes for the benefits and reasons that are as follows: Approaches that will assist children to follow through with their anxieties and fears. Opportunities for children to express or vocalise feelings and emotions through puppet play. Assist shy or difficult children to settle and become engaged. Puppets can be a confidant for children. It can be their friend, where some children can be reluctant to confide in an adult. A puppet can make it easier for a child to be more willing to discuss how they are feeling or why they are upset. Some children may feel overwhelmed when in a group. Using a puppet could help a child to communicate more confidently. 3. The program provides flexibility when linking to the emergent curriculum. This allows further flexibility to leave the program and return to it or just do part of the program, whilst connecting the program to other areas or topics the children are working on. 4. The 5 bush animal stories have been shortened to align with the 4-5 year old children’s attention and concentration span. 5. The “Introduction” has been shortened to offer more clarification and discussion of the meaning and relationship to each child with the concepts being, family/mob, community, safe/unsafe concepts, danger and feelings. By using this approach the children’s prior knowledge of these concepts will be built upon and extended, therefore supporting them during pre-program and throughout the storybook. 6. Non-inclusion of colouring sheets and worksheets in the ACBS program, due to the fact of them being too product based, not open ended or child centred. Worksheets lack expression in what the children are thinking or feeling. Best Practice in Early Childhood is when the children have control over the materials they use, along with autonomy in decisions and choices they make. This enables an increase in emergent and creative experiences in the ACBS program to include drawing and painting of pictures by the children. 7. Music and movement has been added using “The Ultimate Collection.” CD by Don Spencer. This CD offers song and movement associated with the Australian bush animals, sea creatures and indigenous musical instruments. The power of music will enhance the emotional and mental well being of the children, along with inspiring and enriching their lives. 8. A kit bag on wheels, making the transportation more mobile, has been established to include all art/craft resources, literature, A4 pictures of the 6 animals/birds, wattle tree, bird nest, red coral tree flowers, lilli pilli fruit and witchetty grubs, as a further introduction to the stories along with music, musical instruments, puppets and costumes. 9. A map has been included alongside each title of the 5 stories. The map has used all the children’s drawings which are in the book. Using this approach enables the children to 48 An evaluation of the implementation of the “All Children Being Safe” (ACBS) program in three preschools 10. 11. 12. 13. 14. 15. continually identify with the same appearance of the animals/birds, whilst following the storyline in sequence. This includes where the animals/birds have travelled and where they live in the bush animal community. The map also provides an opportunity for the children to recall the story from the previous week and to be introduced to the next story. The experiences throughout the book, focuses on small groups of 4-6 children in total at any one time. With small groups of children in place, communication between children and staff make recording more meaningful, whilst providing a wider timeframe for the children to gather their thoughts. To enhance the children’s language abilities, provision of facial emotion cards and wooden facial expression paddles have been included. This provides an opportunity for each child to identify and discuss ‘our feelings.’ The ACBS storybooks have been enlarged by 5 cms to 35cm x 25cms horizontally and vertically and the cover and pages are a thicker material, making the book easier to handle by educators and the preschool children. The cover has been made blue background for easy identification from the early primary story books, which are terracotta clay colour. Discussion regarding respect of diversity, similarities and differences between the bush animals and people has been included. In strengthening confidence in language (verbal and non-verbal) cooperation with others and negotiation of roles and responsibilities in play and group experiences, the children will gradually understand different ways in contributing through play projects in a non-threatening manner. This can provide a further opportunity for the children to comment and/or question and respond with relevant gestures and actions. ACBS stories have been set up to be used on a SMARTboard/Interactive White Board. This will engage the children with vivid images enhancing the ACBS story messages getting across in a powerful visual representation. Pre-Program Module will introduce and inform parents/family and the local community services regarding the implementation of the forthcoming ACBS program at their child/chn’s preschool setting. This will include: Parent/Family/Community/Staff gathering to inform the preschool service users of the contents associated with the ACBS program. Written permission to be given by parents/families for their child to be included in the ACBS program, along with having photos of their child and their artwork displayed in the classroom and used for program evaluation. Consultation with the parents/families and community as to the ways to best keep them informed regarding the implementation and progress of the ACBS program at the centre .Examples to be given – newsletters, displays, day book entries, digital photo frame, mid program parent/family/community/staff gathering. Information regarding ACBS Program to be sent/faxed to the local community services ie. Police, Fire Brigade, Ambulance, local district hospital, Community Health Services, FACS, local library and FaHCSIA. 49 An evaluation of the implementation of the “All Children Being Safe” (ACBS) program in three preschools Appendix 2: EVALUATION FRAMEWORK EVALUATION FRAMEWORK TO EXAMINE THE ADAPTATION AND IMPLEMENTATION OF “ALL CHILDREN BEING SAFE” (ACBS) PROGRAM, IN THREE PRESCHOOLS, IN KEMPSEY, NSW Project brief: Over a 12 month period NAPCAN will project manage the adaptation of the “All Children Being Safe” (ACBS) program for preschools, in one rural and remote Indigenous community – Kempsey, on the mid north coast of NSW. The NAPCAN project coordinator has established a Steering Group consisting of the preschool teachers, indigenous consultants and indigenous services and will adapt the original ACBS program to be a developmentally appropriate protective behaviours program for pre-school children aged 3-5 years old. The coordinator will then train child care centre staff in the new ACBS program, and will assist with the delivery of the ACBS program in three preschools in the Kempsey area, in partnership with an Indigenous consultant and Indigenous services. Macquarie University has been invited to evaluate the adaptation and implementation of the ACBS project in the three selected preschools in Kempsey. Participating preschools and educators: South Kempsey Preschool: Shirley Dalaigur Preschool: Roslyn ABC Learning Centre, West Kempsey: Melanie Project period: July 2010 to September 2011 Project coordinator: Michelle Rose Project Evaluation Group: Michelle Rose, Angela Walsh, Karen Roberts and Manjula Waniganayake Project Steering Group: Shirley, Roslyn, Melanie (the educators), Janet Jensen, Karen Hall, Marilyn Dean and Delya Smith. Project objectives: The anticipated outcomes of the ACBS program as modified for preschools have been specified for children, parents and preschool staff and evaluation strategies to collect data to assess these outcomes are defined as follows: The ACBS Program outcomes for children: To demonstrate an awareness of the concepts of safe and unsafe living through an ability to a) Identify safe and unsafe feelings and that every child has the right to feel safe. b) Identify people in their community who keep them safe. c) Feel safe, secure and supported. d) Develop the language to communicate safe or unsafe behaviours / situations. e) Identify people who can help them if they have been hurt. 50 An evaluation of the implementation of the “All Children Being Safe” (ACBS) program in three preschools f) Interact in relation to others with care, empathy and respect. g) Develop a sense of belonging to groups and communities Evaluation strategies: Teacher-led small group discussion with children (4-6 at a time) about their current understandings about safe and unsafe living (including feelings and people) one week before the ACBS program is implemented. One month after the implementation of the full ACBS program has finished, each teacher works with a random sample of 6-10 children at their centre who participated in the ACBS program, to discuss and draw what the child remembers from the ACBS program implementation. It is recommended that each preschool teacher leads these discussions, using a common set of questions that the Evaluation Team will provide, and modify these as appropriate to the ongoing conversation with their own group. It is also recommended that an observer/notetaker is present during these discussions to document the discussions verbatim – especially the children’s words, as much as possible. At the end of each session the teacher will lead a discussion with the children, and complete an evaluation using a proforma focusing on the children’s learning. We trust that there will be appropriate questions in the revised ACBS program documentation to assist with this process. The Evaluation Team will be provided with either an overall summary of key points from these evaluations for each preschool (ie, a summative evaluation) or/and a copy of the evaluations completed after each session at each centre (ie, a formative evaluation). During and after the implementation of the ACBS program, each teacher continues to collect pedagogical documentation reflecting children’s developing understandings about child protection matters discussed, reflective of outcomes. This includes teacher observations, children’s artefacts and conversations of interactions between children and adults. Appropriate consent from the parents/guardians will be obtained for children’s participation in the ACBS program evaluation activities. Information about working with children on the evaluation will be provided during the evaluation workshop we will present to staff. We are also planning to incorporate ways of doing pedagogical documentation into our evaluation workshop. ACBS program outcomes for pre-school staff To demonstrate an enhanced awareness of child protection strategies by: a) Participation by all preschool staff in training on child protection including use of support services in their community. b) Improving the standards of child protection practices, particularly in light of new legislation or information. c) Incorporating the modified ACBS program for early childhood into their preschool programs. d) Involving parents and other family members in the ACBS program in their centres. Evaluation strategies: Using the NAPCAN evaluation proforma provided with the original ACBS program, at the completion of each module, documentation of each teacher’s reflections on what they achieved and require reconsideration in the way the next ACBS modules are implemented with their group of children. The Evaluation Team will be provided with 51 An evaluation of the implementation of the “All Children Being Safe” (ACBS) program in three preschools either an overall summary of key points from these evaluations for each preschool (ie, a summative evaluation) or/and a copy of the evaluations completed after each session at each centre (ie, formative evaluation). A survey questionnaire on child protection legislation and required practices following participation in the professional development sessions – or prior to, as appropriate. This will be done on a 1:1 basis with each preschool teacher and the Evaluation Team. Participation in discussions with the Evaluation Team at the start of the implementation and at the end, to discuss, their approach and its impact on the children, families and other staff at their centres. We will use the notes from our discussions during the December visit and the completion of the survey questionnaire sent in December for this purpose. We also anticipate using discussions held on the proposed training day and completing this work at a meeting in August when all centres have completed their project implementations. ACBS outcomes for parents and community: To demonstrate an enhanced awareness of child protection issues in the wider community through a) Participation in community workshops about the ACBS program b) Engagement in the delivery of the ACBS program e.g. craft activities, storytelling, drama, music and movement. c) Sharing the ACBS program with their children at home through culturally appropriate literature. Evaluation strategies: Centre profile survey questionnaire to be completed by the NAPCAN project coordinator to ascertain information about resources and support provided centrally to all three preschools. Teacher documentation of relevant conversations with parents and other family members regarding their experiences with their children about content included in the ACBS program. The Evaluation Team will facilitate a Focus Group/Community Gathering to capture parent/family perspectives about the implementation of the ACBS program. (July) Discussions with the Project Steering Group about their reflections on how the ACBS program was received by the Kempsey community will also be included in this data. Proposed timelines and milestones for the evaluation: January Send proposed evaluation framework to Angela for consideration and confirmation Teleconference #1: Proposed Monday 24 January at 9.30am with the ACBS Project Executive Group: Angela, Michelle and Karen H. (NAPCAN team) and Karen R. and Manjula (Evaluation Team) to discuss proposed evaluation framework February -March Teleconference #2: Follow up to confirm arrangements for March/April at a date to be set in January, with the ACBS Project Executive Group Discussion and feedback from the evaluation team regarding the modified ACBS program prior to implementation with preschool staff March Finalise the design of the evaluation instruments and training material; copy of the revised ACBS program must be sent to the Evaluation Team by 11 th March. 52 An evaluation of the implementation of the “All Children Being Safe” (ACBS) program in three preschools Visit to Kempsey on 30 March to run professional development for the preschool staff on the project evaluation strategies. Recommend that the ACBS program coordinator presents an orientation to preschool staff on the same day. April - June Project implementation and documentation as per the evaluation framework Teleconference #3 and #4 with the ACBS Project executive group in May and June for a status update on project implementation. Dates to be set in March. As implementations finish, submission of data collected at each centre to the Evaluation Team July Initial collation and analysis of evaluation data based on children’s learning Final evaluation data collection by the Evaluation Team - and focus group/community gathering - with a visit to Kempsey August Final collation and data analysis Preparation of the report on the evaluation – we can do a presentation to the community if required. Proposed professional development sessions: ACBS Program Orientation to be presented by Michelle on the same day as evaluation framework session, so that staff are made aware of the content of the program and what they are expected to do in implementing it. Evaluation Workshop: We will do 2 sessions with all preschool staff, based on the evaluation framework as follows: a) Children’s Rights within a program evaluation (1 hour) – introduction to the program evaluation framework, its purpose and their role as teachers. b) Involving children in the project evaluation (2 hours) – how to use the evaluation tools and strategies by aligning it with the EYLF and pedagogical documentation. 53 An evaluation of the implementation of the “All Children Being Safe” (ACBS) program in three preschools Appendix 3: ACBS PROGRAM EVALUATION INSTRUMENTS Instrument #1: PRE-PROGRAM QUESTIONS TO CHILDREN Instrument #2: POST-PROGRAM QUESTIONS TO CHILDREN Instrument #3: TEACHER REFLECTIONS AFTER EACH MODULE Instrument #4: TEACHER REFLECTIONS ON CHILD PROTECTION LEGISLATION Instrument #5: TEACHER REFLECTIONS - PRE-PROGRAM OVERVIEW Instrument #6: POST-PROGRAM EVALUATION QUESTIONS SENT TO STEERING COMMITTEE MEMBERS Instrument #7-9: POST-PROGRAM EVALUATION QUESTIONS: These were used with parents at the Community Gathering (instrument #7), individual teacher reflections (instrument #8) and those presented to the NAPCAN program manager and ACBS coordinator (instrument #9) NOTES: Children’s learning was formally assessed by using a series of questions one week prior to program being implemented at each centre (Instrument #) and one month after the implementation was completed (Instrument #2). In both instances, the children’s responses to these questions were documented by an observer (in this instance, the ACBS program coordinator) whilst each teacher led the discussion with the children. Teachers were also given a set of questions to use after each module was implemented (Instrument #3). Each teacher reflected on the ACBS program, one week before the program was implemented (Instrument #5) and one month after the completion of the full program (see Appendix 8). Teachers were asked to reflect on child protection legislation (Instrument #4) prior to program implementation. These questions (Instrument #6) were sent to each participant and responded by email. Since there were common questions asked of parents, teachers and the program manager and coordinator, these instruments have been collated in a table format.(instruments #7, 8 and 9) 54 An evaluation of the implementation of the “All Children Being Safe” (ACBS) program in three preschools ALL CHILDEN BEING SAFE Kempsey Evaluation Project for NAPCAN (2011) Instrument #1: PRE-PROGRAM QUESTIONS TO CHILDREN Children’s responses to these questions were documented by an observer (such as the ACBS program coordinator) whilst the teacher is leading the discussion with the children. Bring together one or two small groups of 4-6 children who will participate in the ACBS program. Use the following questions to identify children’s current understandings about safe and unsafe feelings, people and places. This activity needs to be completed one week BEFORE the implementation of the ACBS program begins. 1. What makes you feel safe? 2. What makes you feel scared or unsafe? 3. What do you do when you feel scared or unsafe? 4. Where do you go when you feel scared or unsafe? 5. Are there people that you feel safe to be with? If you feel scared or unsafe, who would you go to feel safe? 6. Who will you tell if you feel scared or unsafe? 7. Who can help you if you are in danger or unsafe? Drawing safe people and places: Get each child to draw a picture of people and places that they feel safe about. Discuss their work and write down any words to help explain what the drawing indicates. Please put each child’s name and the date on which this activity was completed. Collect the artwork and store in a secure location to be given to the Evaluation Team. 55 An evaluation of the implementation of the “All Children Being Safe” (ACBS) program in three preschools ALL CHILDEN BEING SAFE Kempsey Evaluation Project for NAPCAN (2011) Instrument #2: POST-PROGRAM QUESTIONS TO CHILDREN Children’s responses to these questions were documented by an observer (such as the ACBS program coordinator) whilst the teacher is leading the discussion with the children. Use the following questions to identify children’s current understandings about safe and unsafe feelings, people and places following the implementation of the ACBS program. This activity needs to be completed one month AFTER the implementation of the ACBS program has been completed. 1. What makes you feel safe? 2. What makes you feel scared or unsafe? 3. What do you do when you feel scared or unsafe? 4. Where do you go when you feel scared or unsafe? 5. Are there people that you feel safe to be with? If you feel scared or unsafe, who would you go to feel safe? 6. Who will you tell if you feel scared or unsafe? 7. Who can help you if you are in danger or unsafe? 8. What do you remember from the ACBS program? 9. What did you like best about the ACBS program? 10. Was there anything that you didn’t like about the ACBS program? Repeat the drawing activity completed before the program started, and reflect on any changes in understanding/awareness that are visible in this work. Write a brief note or be prepared to talk to the Evaluation Team about any changes. Make sure the drawings completed before and after program implementation are kept separate. 56 An evaluation of the implementation of the “All Children Being Safe” (ACBS) program in three preschools ALL CHILDEN BEING SAFE Kempsey Evaluation Project for NAPCAN (2011) Instrument #3: TEACHER REFLECTIONS AFTER EACH MODULE Complete one of these feedback sheets for each module and circle which session: INTRODUCTION KOOKABURRA KANGAROO KOALA ECHIDNA KING PARROT Following the implementation of each module, teachers are invited to document their reflections regarding the learning that took place using the following questions 1. How did the children respond to this module? 2. What key learnings about feeling safe emerged from this module? 3. What did the children enjoy most about this module? 4. What did the children have difficulty in grasping about this module? 5. Was there something you could do to address these difficulties? 6. What were the reactions/comments/feedback from parents and other family members to the ACBS program? ALL CHILDEN BEING SAFE Kempsey Evaluation Project for NAPCAN (2011) Instrument #4: TEACHER REFLECTIONS ON CHILD PROTECTION LEGISLATION 1. What are the 3 pieces of legislation that relate to child protection and what are the features of each? 2. What are the grounds for reporting risk of significant harm? 3. How would you make a risk of significant harm judgement? 4. What actions are required when you suspect a child or young person is at risk of significant harm? 5. What do you do when risk is below significant threshold? 57 An evaluation of the implementation of the “All Children Being Safe” (ACBS) program in three preschools ALL CHILDEN BEING SAFE Kempsey Evaluation Project for NAPCAN (2011) Instrument #5: PRE-PROGRAM TEACHER REFLECTIONS 1. What are you excited about in undertaking the ACBS program? 2. Do you anticipate any difficulties in implementing the ACBS program at your centre? 3. How are you planning to address these concerns? 4. In what ways do you broach difficult issues that arise and engage in problem solving with families? (EYLF Educators’ Guide 2010, p.18) – in relation to child protection issues? 5. How do you propose to introduce parents/family members to ACBS program? 6. How are partnerships with families encouraged in your setting? (EYLF Educators’ Guide, 2010, p.19) 7. What opportunities do you provide for parents to engage with you in ways that respect and value their culture? (EYLF Educators’ Guide 2010, p.22) 8. How do you assist children to gather information, ask questions, seek clarification and consider possibilities about their own lives? (EYLF Educators’ Guide 2010, p.22) 9. How do you demonstrate that you acknowledge and build on the context and discourse of each child and family? (EYLF Educators’ Guide 2010, p.28) 10. In what ways do you support children’s learning and use the cultural tools of the community to inspire all children’s thinking? (EYLF Educators’ Guide 2010, p.28) 58 An evaluation of the implementation of the “All Children Being Safe” (ACBS) program in three preschools ALL CHILDEN BEING SAFE Kempsey Evaluation Project for NAPCAN (2011) Instrument #6: QUESTIONS FOR THE ACBS STEERING COMMITTEE 1. Can you please tell us about your involvement in the All Children Being Safe Project being run in the three preschools in Kempsey? 2. From your perspective, what are the highlights of the ACBS program being implemented at these preschools this year? 3. How has the ACBS program influenced the way you may think about child protection issues? 4. What aspects of the ACBS program could be improved to increase parent/family involvement in child protection work? 5. Is there anything you would like to see included or excluded from the current program to strengthen or improve children's learning about all children being safe? 6. Now that the program implementation has finished, looking into the future, what would you like to see happen with the children, parents and staff who participated in this program? 59 An evaluation of the implementation of the “All Children Being Safe” (ACBS) program in three preschools POST-PROGRAM EVALUATION QUESTIONS used with parents, teachers and program manager & Coordinator Instrument #7: COMMUNITY GATHERING questions 1) 2) 3) 4) 5) 6) 7) 8) 9) Instrument #8: Post-program TEACHER reflection questions How did you find out 1) In terms of your about the ACBS program hopes/expectations of this being implemented at program, what are the your children’s centre(s)? highlights of the ACBS program What were your initial thoughts and implementation at your feelings about the program before centre? it was implemented? 2) Have you noticed any How were you involved in changes in children’s the ACBS program at your behaviours and/or attitudes child’s centre? that can be linked with the Can you share with us, ACBS program in some way? anything your child/ren, YES/NO Please comment. talked about the ACBS 3) Do you feel that the program with you at topics/issues discussed were home? appropriate or meaningful for From your perspective, the children at your centre? what are the highlights of YES/NO Please comment. the ACBS program being 4) Did any children have any implemented in your strong reactions to any children’s preschools this aspects of the program? year? YES/NO Please comment. Are there any aspects of 5) Of the strategies you used to the ACBS program which assist children’s learning about you feel could be matters covered in this program, strengthened to improve which ones were most children's learning about appropriate and effective? all children being safe? 6) How did parents and other How has the ACBS program family members get involved influenced the way you may with the ACBS program at your think about child protection centre? (any conversations issues? that were documented??) What aspects of the ACBS 7) Is there anything that you program could be would include or exclude from improved to increase the ACBS program that you parent/family involvement implemented? YES/NO in child protection work? Please comment. Looking into the future, 8) Were there any unexpected what would you like to see developments or surprises happen with the children, during the implementation of parents and staff who the program at your centre? participated in this YES/NO Please comment program? 9) Are there any aspects of the ACBS program which you feel could be strengthened to improve children's learning about all children being safe? 10) Looking into the future, what would you like to see happen with the children, parents and staff who participated in this program? Instrument #9: NAPCAN PROGRAM MANAGER & COORDINATOR questions 1) 2) 3) 4) 5) 6) 7) In what ways do you believe, the ACBS program enhanced the understanding of child safety/child protection issues by children and their families at each of the three centres? What impact do you believe, the ACBS program has had on the teachers’ understanding and approach to child protection/safety matters? Is there anything that you would include or exclude from the ACBS program that you have developed? YES/NO Please comment. In terms of your hopes/expectations of this program, what do you consider to be the highlights of the ACBS program implementation in these three centres? From your perspective as program coordinators, were there any unexpected developments or surprises during the implementation of the ACBS program? YES/NO Please comment Are there any aspects of the ACBS program which you feel could be strengthened to improve children's learning about all chn being safe? Looking into the future, what would you like to see happen with the children, parents and staff who participated in this program? 60 An evaluation of the implementation of the “All Children Being Safe” (ACBS) program in three preschools Appendix 4: LETTER OF THANKS FROM PRESCHOOL STAFF To whom it may concern Firstly, we would like to say thank you for including us in your programme. We are most grateful for the benefits our children and our staff have received. This includes not only new equipment to use but the knowledge gained from this programme. It was fun and exciting for both staff and children. Our children seem more aware now about where to go or who to go to if in danger. The stories and activities each week all reflected around stranger awareness and their feelings. This helped the children to talk about themselves and their feelings. One large puzzle in particular which was donated was very popular with the children, and the staff were able to extend on this activity by allowing the children to pain a large kangaroo themselves. This was achieved by drawing around the puzzle using it as a template then cutting it out on cardboard,. The children then painted it using cotton buds giving it the effect of a dot painting. Again, thank you very much for giving us this opportunity through your programme. From all our staff at xxx preschool. 61