Doctorate in Educational and Child Psychology Pamela Hamilton Case Study 1: An Evidence-Based Practice Review Report Theme: Interventions for children with Special Educational Needs What is the effectiveness of the Triple P programme (Positive Parenting Programme) on reducing or preventing child behavioural/conduct problems? Summary This systematic literature review evaluates the evidence base for the effectiveness of the Family Behavioural Intervention called Triple P. Triple P was designed by Sanders and colleagues (1999) in response to the reduced success of previous behavioural family interventions (BFI) for families in which parenting challenges are but one of the many obstacles in which they have to overcome (Sanders, Markie-Dadds & Bor, 2000). Triple P claims to be one of the most effective evidence based parenting programmes in the world (www.triplep.net/glo-en/home).Triple P is a multi-level family support strategy which consists of five intervention levels designed for parents of children from birth to sixteen years of age who are at risk of developing conduct problems. This review aims to evaluate post 2010 research into the influence of Triple P on child behavioural outcomes. Five papers were identified to address the review question. Overall results found that Triple-P was effective in reducing/preventing child conduct problems. Promising evidence was also found for the effectiveness of Triple P in Asian countries and children with developmental disabilities. 1 Doctorate in Educational and Child Psychology Pamela Hamilton Introduction What is Triple P? Triple P’s authors have suggested that Triple P is one of the most effective evidence based parenting programmes in the world (www.triplep.net/glo-en/home). Triple P is a multi-level family support strategy which consists of five intervention levels designed for parents of children from birth to sixteen years of age who are at risk of developing conduct problems. Conduct problems can be considered to refer to a wide range of “acting out” behaviours which ranges from minor oppositional behaviour i.e. temper tantrums to more serious physical aggression and theft (Peacock & Holland, 2003). Triple P was designed by Sanders and colleagues (1999) in response to the reduced success of previous behavioural family interventions (BFI) for families in which parenting challenges are but one of the many obstacles in which they have to overcome (Sanders, Markie-Dadds & Bor, 2000). Through a multi-level and tiered approach which increases in strength, families are provided with the level of support which meets their needs. Table 1 describes each level of support. Table 1 Levels of Triple P (information obtained from the Triple P website) Level Description Level 1 Also known as universal Triple P, this model is designed to read a large volume of people through positive 2 Doctorate in Educational and Child Psychology Pamela Hamilton parenting information messages provided through the media. Level 2 Also known as the “light touch” intervention, this model provides a brief one or two session consultation for parents who are coping well but have slight concerns about their child’s behaviour or development. Level 2 can also be delivered through seminars. Parents can attend any number of three 90-minute seminars (Power of Positive Parenting; Raising Confident, Competent Children; and Raising Resilient Children) or any of the three seminars in the Teen Triple P Seminar Series (Raising Competent Responsible Teenagers; Teenagers; and Getting Raising Teenagers Connected). Level 3 This is a four session consultation model for children with mild to moderate behavioural problems. It is mainly used in the primary care setting and take the form of a face-to-face or telephone intervention with a primary care worker. Tip sheets and positive parenting booklets are used to reinforce strategies. Alternatively it can take the form of two hour small discussion groups targeting specific issues. 3 Doctorate in Educational and Child Psychology Level 4 Pamela Hamilton Also known as Standard Triple P, the intensity of the Triple P programme greatly increases at this level. Level 4 consists of 8-10 sessions and is typically for parents with children who are displaying more severe behavioural problems such as aggression. It can be delivered through individual, group, self-help or online course formats. Level 5 This programme takes two formats. Enhanced Triple P is for families facing parenting concerns in addition to other obstacles, such as parental mood disturbance. This model best reflects the diversity of parental support offered by the programme due to it being individually tailored to meet the needs of the family. Alternatively, at this level Pathways Triple P is offered to families where child maltreatment is a risk. Pathways Triple P provides assistance with anger management and coping strategies along with other behavioural strategies at an enhanced level. Parents must complete a level four programme before accessing level 5. 4 Doctorate in Educational and Child Psychology Pamela Hamilton Basis in Psychological Theory Behavioural principles drawn from social learning theory, cognitive and developmental psychology strongly underlie the Triple P model. In addition, Triple P also draws upon research into the risk factors mentioned above which can influence parenting. Social learning theorist Bandura (1977) suggested that behaviour is learned via the environment through observation. Children will observe the way in which others (models) around them behave and copy such behaviour. Children are presented with many models in their environment, such as friends and individuals from popular culture. However, their primary caregiver, their parent(s), can be considered to be their most influential model. Other Social learning theorists, such as Patterson (1982), have developed Bandura’s thinking. Research into the reciprocal and bidirectional nature of parent and child interaction has identified that parents may directly train their child to perform antisocial behaviours. Parents may be non-contingent in the positive reinforcement of pro social behaviours and their sanctions towards challenging behaviours. This results in many challenging daily behaviours by the child being reinforced. The child then uses their challenging behaviour to reduce aversive intrusions by family members and the behaviour then becomes functional. This causes coercive behaviours for all parties to increase and escalate, often resulting in the child learning to control the family by coercive means (Patterson, DeBaryshe, & Ramsey, 1989) As a result of this Triple P teaches parents positive child management skills as an alternative to coercive and inadequate parenting. Triple P teaches parents that, as social 5 Doctorate in Educational and Child Psychology Pamela Hamilton learning theory suggests, a child’s behaviour will improve if it is appropriately reinforced. Therefore, good behaviour will increase if it is rewarded and bad behaviour will decrease if it is ignored or adequately sanctioned. In addition, Cognitive Behavioural Therapy (CBT) has greatly influenced Triple P at level 5. CBT techniques such as mood monitoring and challenging dysfunctional cognitions are frequently used at this level. Sanders, Dadds and Turner (2003) note that social information processing models has influenced the content of Triple P. This theory emphasises the role in which parental attributions, beliefs and expectations influence parental self-efficacy, decision making and behaviour intentions. Triple P aims to address this by facilitating the parent in reframing their understanding of the child’s and their own behaviour. Relevance to EP practice Behavioural, social and emotional difficulties (BESD) have a significant impact on a child’s learning outcome (Amstrong & Hallett, 2012) and life chances. BESD and learning difficulties are not mutually exclusive. Students with BESD can have behaviour problems which may frustrate their access to the curriculum. For example, a child with BESD who is frequently excluded from school for aggressive behaviour will miss out on social and academic learning experiences which will ultimately influence their academic attainment and peer relationships. Educational psychologists are frequently involved in BESD cases and BESD is a substantial part of the EP’s remit. In 2010 the Association of Educational Psychologists and Educational psychology (AEPEP) submitted a memorandum to the UK government which 6 Doctorate in Educational and Child Psychology Pamela Hamilton focused on the role, associated benefits, and additional resources that educational psychologists can provide to schools in order to effectively manage challenging behaviour and discipline (AEPEP, 2010). However, whilst the report focused on school based interventions there is a strong link between parenting and child behaviour. Negative parenting style characterised by harsh and inconsistent discipline have been found to be associated with severe child antisocial behaviour. In addition, lack of involvement and poor child monitoring strongly predicts antisocial behaviour (Loeber & Stouthamer-Loeber, 1986). Reid, Webster-Stratton & Baydar (2004) also note that permissive and inconsistent parenting, along with the use of more violent and critical discipline is more likely to result in a child with anti-social behaviour. The Department for Education have suggested that appropriate parenting programmes which reduce coercive parenting practices, improve positive parenting, and reduce child antisocial behaviour would be beneficial in tackling the above (Scott, Doolan, Beckett, Harry, Cartwright & the HCA team). A range of behavioural family intervention programmes have been identified by the DfE, which includes standard Triple P parenting programme (www.education.gov.uk/commissioning-toolkit/Programme/Detail/36). Whilst the above research has used the terms BESD and anti-social behaviour they can both be identified as characteristic of conduct problems. However, whilst the Standard Triple P programme has been suggested, an Educational Psychologist has to consider the financial implications and cost effectiveness of the family behavioural intervention in this 7 Doctorate in Educational and Child Psychology Pamela Hamilton time of austerity. Therefore, providing a range of less costly Triple P services that have a reliable evidence base would be advantageous. This, suggestion is particularly supported by the fact that the whole Triple P programme has been found to produce predictable decreases in child behaviour problems (Graff, Speetjens, Smit, de Wolff & Tavecchio, 2008) at both level 3 and 4 for parents from various populations and family types. Review Question (101 words) Research into the effectiveness of Behavioural Family Interventions for pupils who have or at risk of developing conduct problems/disorder have looked at child outcomes. However, the most recent meta-analyses of Tripe P at all levels was in 2008 (Nowak & Heinrichs, 2008). Five years have passed since then and additional studies info the effectiveness of Triple P has been released. The aim of this study is to evaluate post 2010 research into the influence of Triple P on child behavioural outcomes. The review question is “What is the effectiveness of the Triple P programme (Positive Parenting Programme) on preventing/reducing child conduct problems?” 1. Critical Review of the Evidence Base 1.1 Literature search 239 A systematic search of the electronic databases PsychInfo, Educational Resource Information Centre (ERIC) and Medline was undertaken on the 18th December 2013. The 8 Doctorate in Educational and Child Psychology Pamela Hamilton following relevant search terms and directions were used to obtain studies relevant to the review. PsychINFO Key word searches were carried out using the following words: 1. Triple P OR Triple-p (Title) ERIC Key word searches were carried out using the following words: 1. Triple P OR Triple-p (Title) Medline Key word searches were carried out using the following words: 1. Triple P OR Triple-p (Title) Using the above mentioned databases’ search engine, the initial search was limited by the inclusion criteria identified in 1a, b, c and d (Appendix A), and combined. Once the duplicates were removed, the titles and abstracts of 26 studies were screened for eligibility. A full article search was undertaken of the remaining studies which gave insufficient information within their titles or abstracts to be able to decide upon the desirability of the study for the review. The inclusion criteria were applied to the 10 studies which left five that were suitable for the review question. Figure 1 depicts the entire process for identifying studies from the initial searches to screening and selecting studies for the review. The inclusion and exclusion criteria are described in Appendix A, and the excluded studies, 9 Doctorate in Educational and Child Psychology Pamela Hamilton following the screening of the full article, plus rationale for exclusion are listed in Appendix B. Table 2 identifies the names of the studies that were selected based on the result of screening process. 10 Doctorate in Educational and Child Psychology Figure 1 Study Identification Process Articles identified from the electronic databases (PsychINFO, ERIC and Medline) Limits applied and duplicates removed (N=124) Excluded bases on criteria 1a, b, c and d Titles and abstracts screened (N=26) Full text and articles screened (N=10) Studies included in the review (N=5) Figure 1. Flowchart if 11 (N=98) Excluded bases on criteria 1a, 1e, 1f, 3a, 3c, 4b, 4c, (N=14) Excluded bases on criteria 1a, 2d, 3c,4b,4c (N=5) Pamela Hamilton Doctorate in Educational and Child Psychology Pamela Hamilton Table 2 List of Studies Eligible for the Review 1. Fujiwara, T., Kato, N., & Sanders, M. R. (2011). Effectiveness of group positive parenting program (Triple P) in changing child behavior, parenting style, and parental adjustment: An intervention study in Japan. Journal of Child and Family Studies, 20(6), 804-813. 2. Heinrichs, N., Kliem, S., & Hahlweg, K. (2013). Four-year follow-up of a randomized controlled trial of triple p group for parent and child outcomes. Prevention Science, 1-13. 3. Leung, C., Fan, A. & Sanders, M.R. (2013). The effectiveness of a group triple p with chinese parents who have a child with developmental disability: a randomized controlled trial. Research in Developmental Disabilities, 34, 976-984. 4. Sanders, M.R., Baker, S. & Turner, K.M.T. (2012). A randomized controlled trial evaluating the efficacy of triple p online with parents of children with early onset conduct problems. Behaviour Research and Therapy, 50, 675-684. 5. Sofronoff, K., Jahnel, D., & Sanders, M. (2011). Stepping Stones Triple P seminars for parents of a child with a disability: a randomized controlled trial. Research in Developmental Disabilities, 32(6), 2253-2262. 1.2 Quality and Relevance Appraisal The studies were initially summarised to identify main information (see appendix C). The studies were then coded based on a coding protocol adapted from the Task Force on 12 Doctorate in Educational and Child Psychology Pamela Hamilton Evidence-Based Intervention in School Psychology which was produced by Kratochwill (2003). This was followed by identifying the weight of evidence of each study by following the weight of evidence protocol identified by Harden and Gough (2012). The weight of evidence framework identifies 3 dimensions for weighting studies. These are: a) Methodological quality: The quality of execution of the study in relation to the quality standards for the study type. b) Methodological relevance: The appropriateness of research design for addressing the review question. c) Topic relevance: The appropriateness of focus of study to review question. The weighting of each study is identified in Table 3 and the detailing on how each judgment was formed and scores awarded can be found in Appendix D under the relevant headings. Table 3 Weighting of Studies Study 13 A B C D Doctorate in Educational and Child Psychology Pamela Hamilton Methodological Methodological Topic Overall Quality Relevance Relevance Weight of Evidence Fujiwara, Kato & 1.5 2 2 5.5 Sanders (2011) Low Low Low Low Heinrichs, Kliem & 3.5 4 Hahlweg (2013) High High Laung, Fan & 3 Sanders (2013) 3 10.5 Medium High 2 2 7 Medium Low Low Low Sanders, Baker & 4.5 4 4 12.5 Turner (2012) High High High High Sofronoff, Jahnel & 1.5 Sanders Low 1.3 Critique and Appraisal of Studies 1.3.1 Sample 14 3 Medium 3 7.5 Medium Low Doctorate in Educational and Child Psychology Pamela Hamilton In determining an appropriate sample size it was decided that the suggestion articulated in a Triple P meta-analysis by Graff et al. (2008) that a sample size of 10-50 is too small would be taken on board. Therefore, a small sample size below 50 would be considered small, 51-100 medium and 101+ large. Sample size was calculated from the point of allocation of families to the control or intervention group. de Graff et al. (2008) in their results identified that the overall mean effect size for improved child behaviour which was observed by parents was d = 0.88. However, according to Cohen (1988) an anticipated effect size of .88, with a statistical power of .8 and probability level of .05 would require a minimum total sample size of 34 participants (one-tailed) and 44 participants (two-tailed). For the purpose of this review the sample size suggestions put forward by Graff et al. will be taken on board as their suggestions specifically relate to Triple P. However, when addressing issues relating to small sample size the comments of Cohen will be taken on board if necessary. Three out of the five studies included in this study had a large sample size of over 100 participants. However, there were two studies that did not have an adequate randomization of participants. Fujiwara et al. (2011) had 91 participants in the intervention group and 24 participants in the control group and Heinrichs et al. (2013) had 186 families in the intervention group and 84 families in the control group. However, this was a result of only 23% of parents declining to participate in the intervention and not the anticipated 50% following stratified randomisation to assign treatment to schools. Sofronoff et al. (2011) also comment on their small sample size limiting the power of effects. Heinrichs et al. (2013) used hierarchical linear modelling as it models the covariance structure of the data and 15 Doctorate in Educational and Child Psychology Pamela Hamilton Fujiwara et al used a multivariate analysis of variance (MANOVA) to adjust for the covariates that differed significantly. In addition, Sofronoff et al. (2011) did not state if there were any significant differences in the demographic characteristics between the intervention and wait list control group. Significant differences between the samples were reported in the Heinrichs et al. study (2013). Heinrichs et al. reported a higher percentage of single mother participants (34 vs 15.6) in the intervention group. As Fujiwara et al. (2011) implemented a quasi-experimental design their groups were also not equally matched. All studies had a noticeable difference in the number of fathers compared to mothers participating in the intervention. This is something that has been consistently picked up in previous Triple P meta-analysis. However, the Heinrichs et al. (2013). study does go some way to remedy this due to the large number of two parent families within its study (N=219). Interestingly, the Heinrichs et al. and Sanders et al. (2012) studies had a small number of families from low socio-economic backgrounds. This finding assists in evaluating the effectiveness of Triple P for families who have a higher socio economic status as many Triple P studies using the group format have successfully targeted low income families (Nowak & Heinrichs, 2008). In addition, as reported by pervious meta-analysis overall there was a much higher proportion of parents who had identified problem behaviour relating to a son and not a daughter. The recruitment process took different forms ranging from promotion posters and application forms distributed to parents of children receiving support for neurological impairment (Leung et al., 2013) to the stratified randomisation to assign treatment to schools. The Sofronoff et al. (2011) study stated that participants indicated an interest, they 16 Doctorate in Educational and Child Psychology Pamela Hamilton did not say how the participants were contacted. It was only the Sanders et al. (2012) study that placed an academic eligibility criteria upon parents in order to be eligible for the study. This study required parents to read English to a year 5 level. In addition, parents were also required to have broadband and internet connection, as it was an online intervention. What should also be noted is that whilst some studies included in this review have exclusion criteria relating to child disability, enrolled siblings and the accessing of other interventions (Heinrichs et al., 2013; Sanders et al.), other studies do not have such criteria or are actively researching the disability group. In addition, none of the studies were undertaken in the UK. Nowak and Heinrichs (2008) have identified that the average attrition rate across all Triple P interventions is 19.5%. The only study that had an attrition rate above this was the Sofronoff et al. (2011) study which had an overall attrition rate of 30%. Attrition was calculated as the amount of participants who did not complete the post-intervention questionnaires following allocation to intervention or control group. In addition, the Heinrichs et al. (2013). study did not state the amount of participants who completed the post intervention questionnaire. However, the study did state the amount of participants who completed the 4-year follow-up questionnaire. Overall, this study had an attrition rate of 10% even though it had a substantial follow-up period. 1.3.2 Design Four studies were randomised control (RCT) group design. However, the Heinrichs et al. (2013) study was only randomised up to the point that the preschools were selected. 17 Doctorate in Educational and Child Psychology Pamela Hamilton Furthermore, the numbers of schools in the treatment (N=11) and control (N=6) differed considerably. Fujiwara et al. (2011) used a quasi-experimental design. Four studies collected baseline data of child behaviour using either the Strengths and Difficulties Questionnaire (SDQ), Eyberg Child Behavior Inventory (ECBI) or Child Behaviour Checklist (CBLC). This affected the overall quality of methodology in the study and was reflected with low ratings on the weight of evidence table due to multi-method data not being obtained. However, Heinrichs et al. (2013) did provide multi-source information by providing separate questionnaire data obtained from both parents. This may have been due to the large sample of two-parent families participating in the study. Sanders et al. (2012) used three behavioural indicators. These being the SDQ, ECHI and a 40 minute observation of the parent and child. This was coded by research assistants using the Family Observation Schedule (FOS) (Sanders, 2000). However, the power of the observational data was low due to the low observation sample size. Fujiwara et al. (2011) reported on the reliability of the SDQ for a Japanese population, Heainrichs et al. (2013) on the internal consistency of the German version of the CBCL and Leung et al. (2013) on the reliability of the Chinese version of the ECBI. Sofronoff et al. (2011) reported on the internal reliability of the ECBI. Sanders et al. (2012) commented on the test-retest reliability and internal consistency of the ECBI and SDQ and the inter-rater reliability of the observations. The studies also differed on their level of follow up for the control group with one studying following up at three months, three 18 Doctorate in Educational and Child Psychology Pamela Hamilton studies following up at six months and the Heinrichs et al. study following up at set times in a four year period. 1.3.3 Intervention Only the Leung et al. 2013 and Heinrichs et al. 2013 studies stated that the Triple P intervention groups were delivered by accredited providers. Furthermore, only the Leung et al. study stated that protocol adherence was discussed and that trainers adhered to the Triple P manual and resource materials. All trainer led Triple P interventions had a specific end date. However, these studies do not make clear if Triple P completion required all the sessions to be attended. Sanders et al. in their Triple P online study highlights that there was a considerable difference in the log on and completion times of the participants. However, the paper does state that after the completion of the study (i.e. after 9 months) participants in the internet-as-usual group were allowed access to the intervention. This suggests that some participants may not have completed the intervention whilst completing evaluation data. 1.3.4 Weight of Evidence Summary The Fujiwara et al. (2011) study scored low for methodological quality. The study gained points for providing enough information for the sample size to be calculated, having more than 51 participants and establishing group equivalence. However, the study did not have a RCT design, use multi source or multi method data, use measures for specific target audience, conduct follow up analysis on control and intervention group. In addition, the study did not state attrition information so points could not be gained. 19 Doctorate in Educational and Child Psychology Pamela Hamilton The Sanders et al. (2012) study scored high for methodological quality. They were able to demonstrate all of the methodological quality criteria (see WOE A in appendix D). The Heinrichs et al. (2013) study also scored high. However, unlike Sanders et al. they were unable to demonstrate multi method and multi-source data collection. The Leung et al. (2013) study scored medium for methodological quality. They were able to demonstrate the use of a RCT design, measures specific for target audience, provision of enough information for effect size to be calculated, more than 51 participants, low attrition and group equivalence. However, they did not demonstrate the addition criteria to score high. The Sofronoff et al. (2011) study scored low for methodological quality. The study demonstrated a RCT design, more than 51 participants, follow-up on intervention and control group and group equivalence. However, they failed to demonstrate the other criteria. The Fujiwara et al. (2011) and Leung et al. (2013) studies scored low for methodological relevance. Whilst they were able to demonstrate the obtaining of pre and post-test measure for all groups involved they were unable to meet the criteria to obtain higher scores (See WOE B in Appendix D). The Sofronoff et al. (2011) study scored medium for methodological relevance. The study was able to demonstrate the use of a control group and pre, post and follow up measures for the intervention and control group. However, the study did not meet the criteria to obtain a higher score (See WOE B in Appendix D) The Heinrichs et al. (2013) and Sanders et al. (2012) studies both scored high for methodological relevance. Both studies were able to demonstrate the random assignment 20 Doctorate in Educational and Child Psychology Pamela Hamilton of participants to the control or intervention group. They studies also adjusted for confounding variables and used pre, post and follow-up measures for all groups involved with the means and standard deviations reported. The Fujiwara et al. (2011) study scored low for topic relevance. The study demonstrated the use of Triple P with participants aged between 2-11 years and the collection of pre and post measures. The study was also able to demonstrate that participants were identified as having elevated levels of child behavioural problems through the analysis of established child behaviour measure(s). However, it did not meet the criteria to obtain a higher score. The Sofronoff et al. (2011) and Heinrichs et al. (2013) studies scored medium for methodological relevance. The studies were able to demonstrate the use of Triple P with participants aged between 2-11 years who have been identified as having elevated levels of child behavioural problems through the analysis of established child behaviour measure(s) or parents requesting to undertake Triple P training. In addition, the study also used pre, post and follow up measures (3 months or more). However, the studies did not meet the criteria to obtain a higher score. The Sanders et al. (2012) study was the only study to score high for topic relevance. The study was able to demonstrate the use of Triple P with participants aged between 2-11 years who were identified as having elevated levels of child behavioural problems through the analysis of multiple and established sources of information (e.g. ECBI and observations) or parents requesting to undertake Triple P training. The study also used pre, post and 21 Doctorate in Educational and Child Psychology Pamela Hamilton follow up measures (6 months or more). Finally, the study reported the means and standard deviations for the child behaviour subscales of the child behaviour. See the final paragraph of Appendix D (WOE D) for how the overall WOE was calculated. 1.4 Outcomes 1.4.1 Child behaviours In a meta-analysis of Triple P Nowack and Heinrichs (2008) identified the overall effect size of Triple P for child problem behaviour to range between 0.35 and 0.48 for between-groups. Similarly de Graaf et al. (2008) found a mean effect size post intervention of .42 (studies used Cohen’s d). Therefore, for the purpose of this study effect sizes below 0.35 will be considered to be low, effect sizes of .36 to .47 as medium and effect sizes above .48 will be considered to be high. However, this review recognises that this classification does not correlate with more established effect sizes classifications. Cohen (1988) identifies 0.2 as a small effect, an effect size of 0.5 as a medium effect and an effect size of 0.8 as a large effect. However, the suggested classifications will be used as it specifically relates to Triple P and the effect size findings of previous meta-analyses. The mean effect size of interventions that provided post and follow up data will be listed in the overall weight of evidence column. Interventions that did not provide follow up information will have their post intervention effect size repeated in the overall weight of evidence column. 22 Doctorate in Educational and Child Psychology Pamela Hamilton The Heinrich et al. (2013) study used the Child Behaviour Checklist (CBLC). The CBCL focuses on the child’s internalising (emotional) and externalising (conduct) behaviours. The data provided by the study does not provide individual data for the emotional and conduct behaviour. Whilst this review had focused on data which specifically focuses on the conduct of a child the Heinrichs et al. study was still included due to it having a four year follow up period, data from both partnership mothers (PM) and fathers and single mothers (SM). Partnership fathers in this study did not report a significant change in behaviour at any time. Therefore, their data is not included in the table. Effectiveness of Triple P on reducing child behavioural/conduct problems 23 Doctorate in Educational and Child Psychology Pamela Hamilton Table 4 Effectiveness of Triple P intervention Study Outcome Effect size – Effect size Overall measure Cohen’s d Cohen’s d Weight of (Post (Time and Evidence intervention) Follow up) Studies using the Strengths and Difficulties Questionnaire (SDQ) Fujiwara, Kato & SDQ- conduct .11 Sanders (2011) Sanders, Baker & Not Low undertaken SDQ- conduct -.58 Turner (2012) -.42 High (6 months) Overall effect size - .24 -.42 Studies using the Eyberg Child Behavior Inventory (ECBI) Sanders, Baker & ECBI-problem -.70 -.60 High Turner (2012) ECBI-intensity -.89 -.74 High (6 months) 24 Doctorate in Educational and Child Psychology Leung, Fan & ECBI-problem Pamela Hamilton -.21 Sanders(2013) Not undertaken Low for control ECBI-intensity -.29 Not undertaken Low for control (6 months) Sofronoff, Jahnel & ECBI-problem .36 Sanders (2011) Not undertaken Medium for control ECBI-intensity .16a Not undertaken Cannot for control calculate (3 months) Overall effect size .41 .67 Studies using observation Sanders, Baker Observed child IDb .14 Turner (2012) disruptive (6 months) behaviour Studies using the Child Behaviour Checklist (CBLC) Heinrichs, Kliem & CBCL-PM .14 .16 Hahlweg (2013) CBCL-SM .37 .52 (1 year) 25 Low Doctorate in Educational and Child Psychology Pamela Hamilton CBCL-PM .06 CBLC-SM .45 (2 years) CBCL-PM .12 CBLC-SM .19 (3 years) CBCL-PM -.01 – CBLC Scores of intervention group slightly above control CBLC-SM Overall effect size .41 .23 .24 Overall effect size PM .08 Overall effect size SM .39 a Effect size found to be not significant. b Study does not provide p-value and only states that it is not significant. 26 Doctorate in Educational and Child Psychology Pamela Hamilton Strengths and Difficulties Questionnaire The two studies that measured child behaviour using the SDQ reported vastly different effect sizes which differed by .47 post intervention. Unfortunately, the Fujiwara et al. (2011) study did not undertake follow up evaluations. Therefore, follow-up evaluation data could only be obtained from the Sanders et al. (2012) study. This data highlights that the maintenance effect of the intervention over a six month period dropped from -.58 to .42. However, the follow up effect size does fall within the Triple P effect size range previously identified by Nowak and Heinrichs (2008). Therefore, the follow up effect size using the SDQ will be considered medium. Eyberg Child Behavior Inventory (ECBI) Three studies measured child behavior using the ECBI which reported vastly different effect sizes. These ranged from Sofronoff et al. (2011) reporting no significant effect for ECBI intensity to Sanders et al. (2012) reporting an ECBI intensity of -.89. The difference in effect size may be due to the Leung et al. (2013) and Sofronoff et al. studies focusing on the effectiveness of Triple P with children with developmental disabilities. Children with developmental disabilities and their parents may have different needs, characteristic and backgrounds which Triple P cannot individually cater for. In addition, children with developmental disabilities may present more complex behaviours which are more resilient to change (Roberts, Mazzucchelli, Studman, & Sanders, 2006) and continuous parental supervision of the child’s behaviour to prevent harm to self or others is required (Plant & Sanders, 2007) 27 Doctorate in Educational and Child Psychology Pamela Hamilton The problem and intensity subscales of all of the studies were merged together to identify an overall effect size post intervention and at follow up. The mean effect size post intervention for the ECBI subscales was .41 which is a medium effect size. The mean effect size at follow up was large at .67. However, this figure should be treated with caution as it is only representative of the Sanders et al. (2012) study as follow up was not conducted by the other studies. Child behavior observation Sanders et al. (2012) was the only study that used child behavior observation. They reported a small effect size of .14. However, the effect size may have been significantly reduced due to the small number of participants which was the result of a technical issue. Effectiveness of Triple P on preventing child behavioural/conduct problems Child Behaviour Checklist There was a considerable difference between the post intervention effect size of PM’s (.14) and SM’s (.37). This trend was continued throughout the four year follow up (apart from in the third year). The overall follow-up effect size for PM’s was .08 and for SM’s .39. The results for SM’s shows strong intervention maintenance effects for the effectiveness of Triple P in the prevention of child conduct problems. The strong intervention effects for SM’s may have been caused by SM’s reporting higher levels of 28 Doctorate in Educational and Child Psychology Pamela Hamilton dysfunctional parenting which was significantly reduced during the course of the intervention and follow up. Overall behavioral evaluations Triple P has a strong maintenance effect of up to four years. In combining the overall individual effect sizes of the SDQ, ECBI and CBCL the overall combined effect size for these behavioural measures is .49. which is a large effect size. 2. Conclusions This systematic review aimed to evaluate the effectiveness of the Triple P on reducing or preventing child behavioural/conduct problems. The studies overall weight of evidence provided mixed results, with only the Heinrichs et al. (2013) and Sanders et al. (2012) study receiving a high rating (three of the studies included in this review are coauthored by Sanders who is the founder of Triple P). The high rating was primarily because of the low methodological quality rating given to the other studies due to multi-source and multi-method data not being obtained. Nearly all studies used questionnaires are their only evaluative tool. These are subjective self-report measures and as noted by Webster-Stratton and Lindsay (1999) solely relying on parent reports alone can misrepresent findings as parents can be biased by personal factors. The finding of this study support previous research which suggests that Triple P is effective in reducing or preventing child behavioural/conduct problems. The 29 Doctorate in Educational and Child Psychology Pamela Hamilton Triple P effect sizes range from .11 to .89. However, the Sanders et al. study, which was the most rigorous, reported an effect sizes of over .6 post intervention and at follow up. Triple P is considerably more effective for children who are not disabled. This may be due to the suggestions articulated earlier that children with developmental disabilities and their parents may have different needs which Triple P cannot cater for. In addition, children with developmental disabilities may present more complex behaviors which are more resilient to change. The effectiveness of Triple P in non-western societies also has room for improvement. The studies in this review conducted in Japan and Hong Kong have considerably low effect sizes. This may be due to the difference in parenting styles between western and Asian culture. Whilst the Heinrichs et. al. (2013) study also report low effect sizes this, as Nowak and Heinrichs (2008) point out, can be attributed to fathers showing considerably less improvement in child behaviour compared to mothers. However, until many more fathers start to participate in studies of the effectiveness of Triple P this finding should be treated tentatively. In addition, TPOL has been found to produce sufficient effect sizes whilst costing the least money. Whilst the overall weight of evidence for the Leung at al. (2013) and Fujiwara et al. (2011) studies are low they do highlight that Triple P can work in countries where English is not the first language. However, these studies did not conduct follow-up evaluations with the control group. In addition, none of the studies undertaken in this review were in the UK. This raises questions as to Triple P’s effectiveness for a UK population, especially in highly diverse communities. 30 Doctorate in Educational and Child Psychology Pamela Hamilton As discussed earlier the interventions for children with developmental disabilities scored considerably lower effect sizes. Sofronoff et al. (2011) suggests that this may be that parents have re-evaluated the behaviour. Therefore the behaviour may still be persisting but parents no longer perceive it as problematic. If this is the case multi-source evaluation data would be highly valuable in proving this hypothesis. Triple P is an effective programme for reducing and preventing child behavioural and conduct problems. Hopefully future research will help to iron out some of this issues brought up in this review. 4.1Recommendations In order to improve the weight of evidence of studies into Triple P multi-source and multi method data needs to be obtained when undertaking research. In addition, further research should obtain follow up data of one year minimum from both intervention and control group. There was much inconsistency in the follow-up of the studies included in this paper. Rigorous and lengthy follow-ups with low attrition rates would enable judgements on the long term maintenance effect of Triple P to be made. It would be financially advantageous to look into the effectiveness of online Triple P with the UK population. Online Triple P (which is a level 4 intervention) would enable more families to access the programme whilst requiring less money from the government. Due to this reason I would recommend the use of the intervention in this way. 31 Doctorate in Educational and Child Psychology Pamela Hamilton References AEPEP. (2010). Memorandum submitted by The Association of Education Psychologists. Armstrong, D. & Hallett, F. (2012). Private knowledge, public face: conceptions of children with SEBD by teachers in the UK – a case study. Educational and Child Psychology, 29, 77-87. Bandura, A. Ross, D., & Ross, S. A. (1961). Transmission of aggression through the imitation of aggressive models. Journal of Abnormal and Social Psychology, 63, 575-582. Bandura, A. (1977). Social learning theory. New Jersey: Prentice Hall. Cohen J. (1988). Statistical power analysis for the behavioral sciences (2nd ed). New Jersey: Erlbaum. de Graaf, I., Speetjens, P., Smit, F., de Wolff, M., & Tavecchio, L. (2008). Effectiveness of the triple p positive parenting program on behavioral problems in children a meta-analysis. Behavior Modification, 32(5), 714-735. Scott, S., Doolan, M., Beckett, C., Harry, S., Cartwright, S. and the HCA team. (2010). How is parenting style related to child antisocial behaviour? Preliminary findings from the Helping Children Achieve Study. Department for Education: Research Report DFE-RR185a. 32 Doctorate in Educational and Child Psychology Pamela Hamilton Ferrari, A.J., Whittingham, K., Boyd, R. Sanders, M & Colditz, P. (2011). Prem baby triple p a new parenting intervention for parents of infants born very preterm: Acceptability and barriers. Infant Behavior & Development, 34(4), 602-609. Fujiwara, T., Kato, N., & Sanders, M. R. (2011). Effectiveness of group positive parenting program (Triple P) in changing child behavior, parenting style, and parental adjustment: An intervention study in Japan. Journal of Child and Family Studies, 20(6), 804-813. Harden, A., & Gough, D. (2012). Quality and relevance appraisal. IOE. Heinrichs, N., Kliem, S., & Hahlweg, K. (2013). Four-year follow-up of a randomized controlled trial of triple p group for parent and child outcomes. Prevention Science, 1-13. Hodgetts, S., Savage, A. & McConnell, D. (2013). Experience and outcomes of stepping stones triple P for families of children with autism. Research in Developmental Disabilities, 34 (9), 2572–2585. Kratochwill, T.R. (2003). Task force on evidence-based interventions in school psychology. American Psychology Association. Leung, C., Fan, A. & Sanders, M.R. (2013). The effectiveness of a group triple p with chinese parents who have a child with developmental disability: a randomized controlled trial. Research in Developmental Disabilities, 34, 976-984. 33 Doctorate in Educational and Child Psychology Pamela Hamilton Loeber, R., & Stouthamer-Loeber, M. (1986). Family factors as correlates and predictors of juvenile conduct problems and delinquency. Crime & Justice, 7, 29. Nowak, C., & Heinrichs, N. (2008). A comprehensive meta-analysis of Triple P-Positive Parenting Program using hierarchical linear modeling: Effectiveness and moderating variables. Clinical child and family psychology review, 11(3), 114-144. Patterson, G. R. (1982). Coercive family process. Oregon: Castalia Publishing Company. Patterson, G. R., DeBaryshe, B. D., & Ramsey, E. (1989). A developmental perspective on antisocial behavior. American psychologist, 44(2), 329. Peacock, M.L. & Gimple, G.A. (2003). Behavioural problems of young children. New York: Guilford Press. Petra, M. & Kohl, P. (2010). Pathways triple p and the child welfare system: A promising fit. Children and youth services review, 34(2), 611-618. Plant, K. M., & Sanders, M. R. (2007). Predictors of care‐giver stress in families of preschool‐ aged children with developmental disabilities. Journal of Intellectual Disability Research, 51(2), 109-124. 34 Doctorate in Educational and Child Psychology Pamela Hamilton Reid, M. J., Webster-Stratton, C., & Baydar, N. (2004). Halting the development of conduct problems in Head Start children: The effects of parent training. Journal of Clinical Child and Adolescent Psychology, 33(2), 279-291. Roberts, C., Mazzucchelli, T., Studman, L., & Sanders, M. R. (2006). Behavioral family intervention for children with developmental disabilities and behavioral problems. Journal of Clinical Child and Adolescent Psychology, 35(2), 180-193. Sanders, M.R., Baker, S. & Turner, K.M.T. (2012). A randomized controlled trial evaluating the efficacy of triple p online with parents of children with early onset conduct problems. Behaviour Research and Therapy, 50, 675-684. Sanders, M.R., Stallman, H. & McHale, M. (2011) Workplace triple p: A controlled evaluation of a parenting intervention for working parents. Journal of Family Psychology, 25(4), 581590. Sanders, M. R., Markie-Dadds, C., & Turner, K. M. (2003). Theoretical, scientific and clinical foundations of the triple p-positive parenting program: A population approach to the promotion of parenting competence (Vol. 1). Queensland: Parenting and Family Support Centre, The University of Queensland. Sanders, M. R., Markie-Dadds, C., Tully, L. A., & Bor, W. (2000). The triple P-positive parenting program: a comparison of enhanced, standard, and self-directed behavioral family 35 Doctorate in Educational and Child Psychology Pamela Hamilton intervention for parents of children with early onset conduct problems. Journal of Consulting and Clinical Psychology, 68(4), 624. Sanders, M.R. (2000). Family observation schedule, QLD, Australia: Parenting and family support centre, The University of Queensland. Sanders, M. R. (1999). Triple p-positive parenting program: towards an empirically validated multilevel parenting and family support strategy for the prevention of behavior and emotional problems in children. Clinical Child and Family Psychology Review, 2(2), 71-90. Sofronoff, K., Jahnel, D., & Sanders, M. (2011). Stepping Stones Triple P seminars for parents of a child with a disability: a randomized controlled trial. Research in Developmental Disabilities, 32(6), 2253-2262. Spijkers, W., Jansen, D.E.M.C, de Meer, G. & Reijneveld, S.A. (2010) Effectiveness of a parenting programme in a public health setting: a randomised controlled trial of the positive parenting programme (Triple P) level 3 versus care as usual provided by the preventive child healthcare (PCH). BMC Public Health, 10(1), 131-136. Webster-Stratton, C., & Hammond, M. (1990). Predictors of treatment outcome in parent training for families with conduct problem children. Behavior Therapy, 21(3), 319-337. The Triple P website provides information of the breadth of the Triple P programme (http://www.triplep.net/glo-en/the-triple-p-system-at-work/). 36 Doctorate in Educational and Child Psychology Pamela Hamilton Appendix A. Inclusion and Exclusion Criteria Inclusion Criteria Exclusion Criteria 1. Publication a) The study only examines the effects a) The study does not only examine the of Triple P, as a parenting programme effects of Triple P, as an parenting for children with conduct problems, programme for children with conduct who are at risk of developing conduct problems or who are at risk of problems or parents who believe developing conduct problems. their child would benefit from the Partial Triple P or combined or non programme. Therefore, the sole Triple P intervention effect sizes will effectiveness of Triple P on conduct not demonstrate the effectiveness of problems can be ascertained. the Triple P programme on reducing conduct problems. b) The article was published in a peer reviewed journal, which would have peer reviewed journal and would not been evaluated and critiqued by have been evaluated and critiqued by researchers within the same field to researchers within the same field to meet quality standards meet quality standards c) The article was written in the English language, as translation would not be 37 b) The article was not published in a c) The article was not written in English, as resources for translation were not Doctorate in Educational and Child Psychology required d) The article was published between Pamela Hamilton readily available d) The article was published before 2010 and 2013, and had therefore 2010 as the articles may have been not been previously reviewed. reviewed in the previous meta analyses. 2. Participants a) The participants must be between 2- a) The participants are not within the 2- 10 years of age in order to look at the 10 years age range and would relationships between Triple P and therefore include adolescent onset childhood onset conduct problems. conduct problems. b) Participants were not specifically targeted because they had been b) Participants were specifically premature babies. This enables the targeted because they had been findings to be generalised and not premature babies. focus on a specific population. 3. Intervention a) The intervention targets both 38 a) The intervention targets employed or employed and unemployed parents. unemployed parents. Such studies This enabled the results to be findings would not be generalisable generalised to a wider social over a wider social demographic. Doctorate in Educational and Child Psychology Pamela Hamilton demographic. b) The intervention does specifically b) The intervention does not specifically target a specific event or time of day. target a specific event or time of day. Therefore, the effectiveness of triple The effectiveness of Triple P P throughout the day will not be throughout the day will provide a ascertained. holistic view of the effectiveness of the intervention. c) The intervention did not have over c) The intervention had over 60 children. Previous meta-analysis have 60 children and would therefore be seen as a low sample size. highlighted low sample sizes influencing effectiveness analysis. Analysing the effectiveness of studies with larger sample sizes will provide more robust evidence. 4. Outcomes a) The study is a group experimental design and therefore allows for the experimental design and intervention effect of the intervention to be effects cannot be isolated. isolated. 39 a) The study is not a group Doctorate in Educational and Child Psychology Pamela Hamilton b) Effectiveness of Triple P was not b) Effectiveness of Triple P was assessed assessed using a questionnaire for by using a questionnaire for parents parents to evaluate their child’s to evaluate their child’s behaviour at behaviour. least. c) The study does not provide sufficient c) Sufficient empirical data was 40 empirical data for the calculation of reported for the calculation of the the standard effect size to be standard effect size post intervention calculated post intervention. Doctorate in Educational and Child Psychology Pamela Hamilton Appendix B. List of Excluded Studies Excluded Study Reason for Exclusion 1. Ferrari, A.J., Whittingham, K., Boyd, R. Sanders, M & Colditz, P. (2011). Prem baby triple P a new parenting intervention for parents of infants born very preterm: Acceptability and barriers. Infant Behavior & Development, 34(4), 602-609. 2.a)The article looked at Prem Baby P which involved children below the age of two. 2.Hodgetts, S., Savage, A. & McConnell, D. (2013)Experience and outcomes of stepping stones triple P for families of children with autism. Research in Developmental Disabilities, 34 (9), 2572–2585 4.b)Effectiveness of Triple P was not assessed using a questionnaire for parents to evaluate their child’s behaviour. 3.Petra, M. & Kohl, P. (2010). Pathways Triple P and the child welfare system: A promising fit. Children and youth services review, 34(2), 611-618. 4.c) Sufficient empirical data were reported for the calculation of standard effect size 4. Sanders, M.R., Stallman, H. & McHale, M. (2011) Workplace Triple P: A controlled evaluation of a parenting intervention for working parents. Journal of Family Psychology, 25(4), 581-590. 3.c)The intervention targets employed or unemployed parents. 5. Spijkers, W., Jansen, D.E.M.C, de Meer, G. & Reijneveld, S.A. (2010) Effectiveness of a parenting programme in a public health setting: a randomised controlled trial of the positive parenting programme (Triple P) level 3 versus care as usual provided by the preventive child healthcare (PCH). BMC Public Health, 10(1), 131-136 1.a) The study does not examine the effects of Triple P, as an intensive parenting programme for children with conduct disorder/problems or who are at risk of developing conduct disorder/problems 41 Doctorate in Educational and Child Psychology Pamela Hamilton Appendix C Article summaries Study and Aims Sample Sofronoff, Jahnel & Sanders (2011) To evaluate the efficacy of the Stepping Stones Triple P seminars, a brief group intervention for parents of a child with a disability. Control group size: 35 (Level 4) Standard Triple P Seminar format Intervention Stepping Stones Triple P There were two Intervention group seminars that size: 35 presented parenting Parental gender: strategies to 92.5% female and improve both child 7.5% male behavior and parenting variables Child gender: Male implicated in the 71.7% and female development and 28.3% maintenance of child problem Child age range: 2- behavior. 10 years of age. Design The study was carried out as an RCT with a 2 (group) 3 (time) design. Assessments of dependent variables took place prior to, six weeks and three months after the workshop. Measures Eyberg Child Behavior Inventory (ECBI) The inventory measures perceptions of disruptive behaviour in children aged 2- 16 years of age. The inventory comprises of a problem scale and intensity scale. Outcomes ECBI Problem scale There were significant intervention effects for the Problem scale, F(1,49) = 12.19, p = .001, Three month follow up results indicated a significant multivariate time effect for reported child behavior problems, F(2,20) = 4.980, p = .018, with significant univariate effects for both the ECBI Problem scale Intensity scale 42 Generalisability and limitations The study had a small number of participants which may have influenced a finding of significance for the ECBI intensity scale. As none of the participants were blinded by their intervention status they may have been influenced by the placebo effect. Data could have been obtained from other sources to determine changes in child behaviour. There was also a very short follow Doctorate in Educational and Child Psychology Pamela Hamilton No significant multivariate intervention effects for the Intensity scale, F(1,49) = 3.845, p = .056. up interval of 3 months. Therefore, caution is advised when generalising this studies findings to other studies Three month follow up results indicated a significant multivariate time effect for reported intensity of behaviours F(1,21) = 7.801, p = .011. Fujiwara, Kato & Sanders (2011) Intervention group Group Triple P Size: 91 comprises eight sessions over an To investigate the Control group eight-week period. effectiveness of Size: 24 Group Triple P with The eight sessions families in Japan, Parent Gender: All are as follows: one with the focus on female group session child behaviour, geared toward parenting style and providing an parental Control group overview of the adjustment child gender: 56% program and 43 Quasi experimental design Strengths and Difficulties Questionnaire (SDQ) (parents) Control-group Measures parents participants were perceptions of pro recruited along social and difficult with those of the behaviours in their intervention group, child. during the same *conduct scale health check-ups. Therefore Child behaviour: Significant intervention effect on the SDQ conduct problems subscale (F (2, 108) = 6.11, p = 0.003), with only the intervention group showing a reduction, of 1.01 The information obtained from this study was from a Japanese population. Therefore, it cannot be generalised to a UK population. In addition, the parents were all Doctorate in Educational and Child Psychology (Level 4) Standard Triple P Group format female and 44% male establishing rapport within the group (2 h each), Intervention group three group child gender: 67 sessions in which male and 33% parent training is female conducted (2 h each), Control group three follow-up child age range: 2- consultations by 3 years old. telephone (15–30 min each), and a final Intervention group group session. age range: 1-8 years old. Recruitment: Recruited from mothers visiting health clinics for mandatory health check-ups for their children across three suburban residential area near Tokyo. In addition flyering and public health 44 Pamela Hamilton participants could not be randomized. points (t = -5.89, p\0.001). women. Therefore, the results cannot be generalised to programmes where both mothers and fathers attend. There was also no follow up undertaken so the overall generalizability of effectiveness cannot be established The above placebo influence and counter measures are also applicable to this study. Doctorate in Educational and Child Psychology Pamela Hamilton nurses who observed behavioral problems in children were asked to recommend to those children’s mothers that they should participate in the study. Sanders, Baker & Turner (2012) This study examined the consumer outcomes, programme acceptability and usage of Triple P Online for parents of children with early-onset disruptive behaviour problems in Australia. 45 Intervention group Parents assigned to Size: 60 the intervention condition were Control group emailed Size: 56 individual log in details and were Child age range: 2- prompted to 9 years old. complete the program within 3 months. Child gender: 67% Access to the male and 33% program continued female until completion of the 6-month Parental gender: follow-up 91% female 9% assessment male This study was a randomized, controlled trial employing a two group (group: TPOL vs internet-asusual control) _ Time (time: pre-intervention, post-intervention, 6-month followup) repeated measures design. On completion of the study the control group was offered access to Eyberg Child Behavior Inventory SDQ (parents) *conduct scale Eyberg Child Behavior Inventory Significant condition effect, F (2,111) ¼ 15.297, p Observation of ¼ .000. child disruptive Univariate tests behaviour – Family showed a Observation significant effect Schedule for both scales, An observation of the child’s Problem scale disruptive TPOL parents behaviour was reporting undertaken. The significantly lower The results could be generalised to some populations within the UK due to possible overlapping between Australian and UK cultural background and parenting styles. Furthermore, the number of children in the control and intervention groups were relatively equally Doctorate in Educational and Child Psychology (Level 4) Standard Triple P Online format Parental age range: 25-50 years of age Recruitment: Through community outreach in mass media, online parenting forums, schools and childcare settings in Brisbane, Australia. Parents self-identified as having difficulties with their child’s disruptive behaviour.. Pamela Hamilton TPOL. primary caregiver and target child were observed during a 40-min videotaped clinic observation including different parenting tasksjoint free-play; and clean-up compliance task; rate (Problem scale) at post intervention than controls, with large effect sizes evident. Intensity scale TPOL parents reported significantly lower frequency (Intensity scale) of behavior problems at post intervention than controls, with large effect sizes evident. SDQ (parents) Significant condition effect, F(5,105) ¼ 3.690,p ¼ .004, with significant univariate effects for the Conduct and Emotion 46 matched However, there was a low representation of families with a low SES. 76% of participants had a lower than average SES. There was a reduced observational sample size due to recording difficulties. Therefore, it would be too hasty to generalise such limited investigations. Doctorate in Educational and Child Psychology Pamela Hamilton scales. Again, parents in the intervention condition reported significantly reduced behavioral and emotional problems at post-intervention in comparison to controls, with medium effect sizes. There were no significant univariate effects for prosocial Behavior. Observation of child disruptive behaviour – Family Observation Schedule Analysis of observed child disruptive behavior revealed no significant condition effects, F(1,38) ¼ .983, p ¼ 47 Doctorate in Educational and Child Psychology Pamela Hamilton .328. Follow up (6 months) Eyberg Child Behavior Inventory MANCOVA revealed a significant effect for ECBI Problem and Intensity scores, F(2,111) ¼ 10.619, p ¼ .000. SDQ (parents) The short-term effects on the SDQ were not maintained over the 6-month period, F(5,105) ¼ 1.415, p ¼ .225. Observation of child disruptive behaviour – Family 48 Doctorate in Educational and Child Psychology Heinrichs, Kliem & Hahlweg (2013) To evaluate the 4year efficacy of the group-based Triple P as a prevention program administered universally. (Level 4) Standard Triple P Group format 49 Intervention group: 186 Control group: 94 Child age range: 2.6 – 6 years Child gender: 51% boys, 49% girls Pamela Hamilton Triple P was implemented in four group sessions that lasted 2 h each. After the completion of group sessions, parents were offered four weekly individual telephone contacts (15–20 min) to discuss progress, questions, and difficulties with a Triple P facilitator. Seventeen preschools were randomly assigned to Triple P (n=11 preschools, 186 families) or a no parenting intervention control group (n=6 preschools, 94 families). Child Behavior Checklist (CBCL 1 1/2–5 and CBCL 4– 18) The German versions of the Child Behavior Checklist (CBCL) for children aged 1 1/2–5 years and 4–18 years (Arbeitsgruppe Deutsche Child Behavior Checklist 1998; 2000) ask parents to rate presence and Observation Schedule Analysis at followup revealed a significant condition effect for observed child disruptive behavior, F(1,23) ¼ 4.814, p ¼ .039,with a small effect size. Child Behavior Checklist Maternal child behavior scores decreased more in the intervention than in the control group. No further main effects of group, child gender, or single parenthood were found either from pre- to post-, or during the followup period. Fathers did This study had a four year efficacy which provides good evidence for the maintenance effect of Triple P. There was also a equal split of boys and girls in the intervention which assists the data in being more generalizable. Doctorate in Educational and Child Psychology Pamela Hamilton frequency of child problem behaviors and emotional disturbances on 100 and 113 items, respectively. Laung, Fan & Sanders (2013) The study examined the effectiveness of Group Triple P with Chinese parents who had a preschool aged child with a developmental disability. (Level 4) 50 Intervention group: 42 The Triple P Level 4 group version consisted of eight Wait list control sessions, with six group: 39 2-h group sessions and two telephone Control group contact parental gender: follow-up sessions. 35 mothers, 4 All the participants fathers had to attend the group sessions and Intervention group complete parental gender: homework in 31 mothers, 4 between the group The study Eyberg Child consisted of a Behaviour randomized Inventory control trial design. A set of questionnaires were to be completed by all participants twice, before and after the intervention. Furthermore, Intervention group not report significant change in child behavior at any time. The difference in pre-4-year followup ES revealed a small advantage for the intervention group for the total score of the CBCL completed by mothers (ES=0.19). ECBI - Intensity the results were significant for ECBIIntensity, F(1, 78) = 20.23–33.45, p < .001, partial h2 = .21–.30, d = 0.33, 95% CI [_0.11, 0.76] and ECBI-Problem The results were significant for ECBIproblem F(1, 78) = There was no follow up for the wait list control group. The study does not list the genders of the parents who did not complete pre and post measures. Doctorate in Educational and Child Psychology Standard Triple P Group format 51 fathers sessions. The group sessions Intervention group consisted of minichild gender: 30 lectures, females, 12 males discussions, role play and exercises. Wait list control The Triple P groups group child were conducted by gender: 27 male, the social workers 12 female of SAHK who were accredited Triple P Recruitment: Level 4 group Promotion posters providers. and application forms were distributed to all parents of children who were receiving services in the Preschool Centres of SAHK, including two Early Education and Training Centres, three Special Child Care Centres, four Special Child Care cum Early Pamela Hamilton participants were requested to complete these questionnaires again, at six months after program completion 12.23–21.03, p < .001, partial h2 = .14–.21, d = 0.22, 95% CI [_0.22, 0.65]. Doctorate in Educational and Child Psychology Education and Training Centres, and two Parents’ Resources Centres. 52 Pamela Hamilton Doctorate in Educational and Child Psychology Pamela Hamilton Appendix D Procedure for weighting studies Studies were weighted according to ‘General design characteristics’, ‘Measures’, ‘Comparison group’ and ‘Analysis’ based on the Kratochwill (2003) coding protocol. Each study was rated as being high, medium or low for each area. The Weight of Evidence scores are based on: A: Quality of methodology (Each question is awarded 0.5 point each for answer of yes, 0 for answer of No= total of 4.5. 2 points or less=low score, 2.5-3=medium score, 3.5+=high score) 1. Was the study a randomised control trial in order to increase methodological rigour? 2. Was data was collected from at least 2 sources in order to analyse child behavioural change? 3. Was data collected from at least 2 assessment methods? 4. Were the measures valid for the specific target audience? 5. Was enough information provided in order for effect size(s) to be calculated? 6. Did the sample size have 51 or more participants? 7. Were follow-up analysis on both control and intervention groups conducted? 8. Was there low attrition? 9. Was group equivalence established? 53 Doctorate in Educational and Child Psychology Pamela Hamilton B: Relevance of Methodology To inspect if the methodological design of the studies were suitable for evaluating the effectiveness of Triple P on the reduction/prevention of behavioural difficulties and conduct problems the following was compiled. (4 points are awarded for a high score) To receive a high weighting for methodological relevance the study must have used a random assignment of participants to the control or intervention group. They study must have adjusted for confounding variables. Pre, post and follow-up measures must have been obtained for all groups involved with the means and standard deviations reported. (3 points are awarded for a medium score) To receive a medium weighting for methodological relevance the study must have used a control group. Pre, post and follow up measures must have been obtained for the intervention and control group. (2 points are awarded for a low score) To receive a low weighting, pre and post-test measure must have been obtained for all groups involved. C: Relevance of evidence to the review question This table investigates if the studies contribute towards answering the review question. 54 Doctorate in Educational and Child Psychology Pamela Hamilton (4 points are awarded for a high score) High- The study will have used a Triple P intervention with participants aged between 2-11 years. Participants would have been identified as having elevated levels of child behavioural problems through the analysis of multiple and established sources of information (e.g. ECBI and observations) or parents requesting to undertake Triple P training. Pre, post and follow up measures (6 months or more) would have been undertaken. The means and standard deviations for the child behaviour subscales of the child behaviour questionnaires would have been reported. (3 points are awarded for a medium score) Medium- The study will have used a Triple P intervention with participants aged between 2-11 years. Participants would have been identified as having elevated levels of child behavioural problems through the analysis of established child behaviour measure(s) or parents requesting to undertake Triple P training. Pre, post and follow up measures (3 months or more) would have been undertaken. (2 points are awarded for a low score) Low- The study will have used a Triple P intervention with participants aged between 2-11 years. Participants would have been identified as having elevated levels of child behavioural problems through the analysis of established child behaviour measure(s). Pre and post measures would have been undertaken. D: Overall Weight of evidence 55 Doctorate in Educational and Child Psychology Pamela Hamilton The overall weight of evidence was calculated by combining the scores for WOE A, B and C. Studies with 10 or more point were given a High Scores. Studies with 8-9.5 points were given a Medium score Studies with 7.5 points or below were given a low score. 56 Doctorate in Educational and Child Psychology Pamela Hamilton Appendix E Coding protocol for reviewed studies Coding Protocol Name of Coder: Date:30/01/14 Full Study Reference in proper format: Fujiwara, T., Kato, N., & Sanders, M. R. (2011). Effectiveness of group positive parenting program (triple p) in changing child behavior, parenting style, and parental adjustment: An intervention study in Japan. Journal of Child and Family Studies, 20(6), 804-813. Intervention Name (description of study): Level 4 Triple P Study ID Number:__________________ Type of Publication: Book/Monograph Journal Article Book Chapter Other (specify): 1.General Characteristics A. General Design Characteristics A1. Random assignment designs (if random assignment design, select one of the following) Completely randomized design Randomized block design (between participants, e.g., matched classrooms) Randomized block design (within participants) Randomized hierarchical design (nested treatments A2. Nonrandomized designs (if non-random assignment design, select one of the following) Nonrandomized design Nonrandomized block design (between participants) Nonrandomized block design (within participants) Nonrandomized hierarchical design Optional coding for Quasi-experimental designs A3. Overall confidence of judgment on how participants were assigned (select on of the following) Very low (little basis) 57 Doctorate in Educational and Child Psychology Pamela Hamilton Low (guess) Moderate (weak inference) High (strong inference) Very high (explicitly stated) N/A Unknown/unable to code B Participants Total size of sample (start of study): 115 Intervention group sample size:91 Control group sample size:24 C. Type of Program Universal prevention program Selective prevention program Targeted prevention program Intervention/Treatment Unknown D. Stage of Program Model/demonstration programs Early stage programs Established/institutionalized programs Unknown E. Concurrent or Historical Intervention Exposure Current exposure Prior exposure Unknown Section 2 Key Features for Coding Studies and Rating Level of Evidence/Support A Measurement (Estimating the quality of the measures used to establish effects) (Rating Scale: 3= Strong Evidence, 2=Promising Evidence, 1=Weak Evidence, 0=No Evidence) A1 The use of the outcome measures produce reliable scores for the majority of the primary outcomes (see following table for a detailed breakdown on the outcomes) Yes 58 Doctorate in Educational and Child Psychology Pamela Hamilton No Unknown/unable to code A2 Multi-method (at least two assessment methods used) Yes No N/A Unknown/unable to code A3 Multi-source (at least two sources used self-reports, teachers etc.) Yes No N/A Unknown/unable to code A4 Validity of measures reported (well-known or standardized or norm-referenced are considered good, consider any cultural considerations) Yes validated with specific target group In part, validated for general population only No Unknown/unable to code Overall Rating of Measurement: 3 2 1 0 B Comparison Group B1 Type of Comparison group Typical intervention Attention placebo Intervention element placebo Alternative intervention Pharmacotherapy No intervention Wait list/delayed intervention Minimal contact Unable to identify type of comparison B2 Overall rating of judgment of type of comparison group Very low Low Moderate High Very high Unable to identify comparison group 59 Doctorate in Educational and Child Psychology Pamela Hamilton B3 Counterbalancing of change agent (participants who receive intervention from a single therapist/teacher etc were counter-balanced across intervention) By change agent Statistical (analyse includes a test for intervention) Other Not reported/None B4 Group equivalence established Random assignment Posthoc matched set Statistical matching Post hoc test for group equivalence B5 Equivalent mortality Low attrition (less than 20 % for post) Low attrition (less than 30% for follow-up) Intent to intervene analysis carried out? Findings: Attrition rate not stated Overall Level of Evidence 1 3= Strong Evidence 2=Promising Evidence 1=Weak Evidence 0=No Evidence C Appropriate Statistical Analysis Analysis 1 Strengths and Difficulties Questionnaire (parents) Appropriate unit of analysis Familywise/expermenter wise error rate controlled when applicable Sufficiently large N Analysis 2____________________________________________________________________ Appropriate unit of analysis Familywise/expermenter wise error rate controlled when applicable Sufficiently large N Analysis 3____________________________________________________________________ Appropriate unit of analysis Familywise/expermenter wise error rate controlled when applicable Sufficiently large N Overall Rating of Analysis: 60 3 2 1 0 Doctorate in Educational and Child Psychology Name of Coder: Pamela Hamilton Date:02/02/2014 Full Study Reference in proper format: Heinrichs, N., Kliem, S., & Hahlweg, K. (2013). Four-year follow-up of a randomized controlled trial of triple p group for parent and child outcomes. Prevention Science, 1-13. Intervention Name (description of study): Group Triple P (level 4) Study ID Number:__________________ Type of Publication: Book/Monograph Journal Article Book Chapter Other (specify): 1.General Characteristics A. General Design Characteristics A1. Random assignment designs (if random assignment design, select one of the following) Completely randomized design Randomized block design (between participants, e.g., matched classrooms) Randomized block design (within participants) Randomized hierarchical design (nested treatments A2. Nonrandomized designs (if non-random assignment design, select one of the following) Nonrandomized design Nonrandomized block design (between participants) Nonrandomized block design (within participants) Nonrandomized hierarchical design Optional coding for Quasi-experimental designs A3. Overall confidence of judgment on how participants were assigned (select on of the following) Very low (little basis) Low (guess) Moderate (weak inference) High (strong inference) Very high (explicitly stated) N/A Unknown/unable to code B Participants Total size of sample (start of study):280 Intervention group sample size:186 Control group sample size:94 61 Doctorate in Educational and Child Psychology Pamela Hamilton C. Type of Program Universal prevention program Selective prevention program Targeted prevention program Intervention/Treatment Unknown D. Stage of Program Model/demonstration programs Early stage programs Established/institutionalized programs Unknown E. Concurrent or Historical Intervention Exposure Current exposure Prior exposure Unknown Section 2 Key Features for Coding Studies and Rating Level of Evidence/Support A Measurement (Estimating the quality of the measures used to establish effects) (Rating Scale: 3= Strong Evidence, 2=Promising Evidence, 1=Weak Evidence, 0=No Evidence) A1 The use of the outcome measures produce reliable scores for the majority of the primary outcomes (see following table for a detailed breakdown on the outcomes) Yes No Unknown/unable to code A2 Multi-method (at least two assessment methods used) Yes No N/A Unknown/unable to code A3 Multi-source (at least two sources used self-reports, teachers etc.) Yes No N/A Unknown/unable to code A4 Validity of measures reported (well-known or standardized or norm-referenced are considered good, consider any cultural considerations) Yes validated with specific target group In part, validated for general population only No Unknown/unable to code Overall Rating of Measurement: 62 3 2 1 0 Doctorate in Educational and Child Psychology Pamela Hamilton B Comparison Group B1 Type of Comparison group Typical intervention Attention placebo Intervention element placebo Alternative intervention Pharmacotherapy No intervention Wait list/delayed intervention Minimal contact Unable to identify type of comparison B2 Overall rating of judgment of type of comparison group Very low Low Moderate High Very high Unable to identify comparison group B3 Counterbalancing of change agent (participants who receive intervention from a single therapist/teacher etc were counter-balanced across intervention) By change agent Statistical (analyse includes a test for intervention) Other Not reported/None B4 Group equivalence established Random assignment Posthoc matched set Statistical matching Post hoc test for group equivalence B5 Equivalent mortality Low attrition (less than 20 % for post) Low attrition (less than 30% for follow-up) Intent to intervene analysis carried out? Findings: 10% attrition over the 4 year follow up period. Post intervention attrition data not stated Overall Level of Evidence 2 3= Strong Evidence 2=Promising Evidence 1=Weak Evidence 0=No Evidence C Appropriate Statistical Analysis Analysis 1____________________________________________________________________________ Appropriate unit of analysis Familywise/expermenter wise error rate controlled when applicable Sufficiently large N Analysis 2____________________________________________________________________________ Appropriate unit of analysis Familywise/expermenter wise error rate controlled when applicable Sufficiently large N 63 Doctorate in Educational and Child Psychology Pamela Hamilton Analysis 3____________________________________________________________________________ Appropriate unit of analysis Familywise/expermenter wise error rate controlled when applicable Sufficiently large N Overall Rating of Analysis: 64 3 2 1 0 Doctorate in Educational and Child Psychology Name of Coder: Pamela Hamilton Date:02/02/14 Leung, C., Fan, A. & Sanders, M.R. (2013). The effectiveness of a group triple p with chinese parents who have a child with developmental disability: a randomized controlled trial. Research in Developmental Disabilities, 34, 976-984. Intervention Name (description of study):Triple P (Level 4) Study ID Number:__________________ Type of Publication: Book/Monograph Journal Article Book Chapter Other (specify): 1.General Characteristics A. General Design Characteristics A1. Random assignment designs (if random assignment design, select one of the following) Completely randomized design Randomized block design (between participants, e.g., matched classrooms) Randomized block design (within participants) Randomized hierarchical design (nested treatments A2. Nonrandomized designs (if non-random assignment design, select one of the following) Nonrandomized design Nonrandomized block design (between participants) Nonrandomized block design (within participants) Nonrandomized hierarchical design Optional coding for Quasi-experimental designs A3. Overall confidence of judgment on how participants were assigned (select on of the following) Very low (little basis) Low (guess) Moderate (weak inference) High (strong inference) Very high (explicitly stated) N/A Unknown/unable to code B Participants Total size of sample (start of study): 81 Intervention group sample size:39 Control group sample size:35 65 Doctorate in Educational and Child Psychology Pamela Hamilton C. Type of Program Universal prevention program Selective prevention program Targeted prevention program Intervention/Treatment Unknown D. Stage of Program Model/demonstration programs Early stage programs Established/institutionalized programs Unknown E. Concurrent or Historical Intervention Exposure Current exposure Prior exposure Unknown Section 2 Key Features for Coding Studies and Rating Level of Evidence/Support A Measurement (Estimating the quality of the measures used to establish effects) (Rating Scale: 3= Strong Evidence, 2=Promising Evidence, 1=Weak Evidence, 0=No Evidence) A1 The use of the outcome measures produce reliable scores for the majority of the primary outcomes (see following table for a detailed breakdown on the outcomes) Yes No Unknown/unable to code A2 Multi-method (at least two assessment methods used) Yes No N/A Unknown/unable to code A3 Multi-source (at least two sources used self-reports, teachers etc.) Yes No N/A Unknown/unable to code A4 Validity of measures reported (well-known or standardized or norm-referenced are considered good, consider any cultural considerations) Yes validated with specific target group In part, validated for general population only No Unknown/unable to code Overall Rating of Measurement: 66 3 2 1 0 Doctorate in Educational and Child Psychology Pamela Hamilton B Comparison Group B1 Type of Comparison group Typical intervention Attention placebo Intervention element placebo Alternative intervention Pharmacotherapy No intervention Wait list/delayed intervention Minimal contact Unable to identify type of comparison B2 Overall rating of judgment of type of comparison group Very low Low Moderate High Very high Unable to identify comparison group B3 Counterbalancing of change agent (participants who receive intervention from a single therapist/teacher etc were counter-balanced across intervention) By change agent Statistical (analyse includes a test for intervention) Other Not reported/None B4 Group equivalence established Random assignment Posthoc matched set Statistical matching Post hoc test for group equivalence B5 Equivalent mortality Low attrition (less than 20 % for post) Low attrition (less than 30% for follow-up) Intent to intervene analysis carried out? Findings: Post intervention attrition rate of 9% and follow-up attrition rate of 14 % (control group only) Overall Level of Evidence 2 3= Strong Evidence 2=Promising Evidence 67 1=Weak Evidence 0=No Evidence Doctorate in Educational and Child Psychology Pamela Hamilton C Appropriate Statistical Analysis Analysis 1Eyberg Child Behaviour Inventory Appropriate unit of analysis Familywise/expermenter wise error rate controlled when applicable Sufficiently large N Analysis 2____________________________________________________________________________ Appropriate unit of analysis Familywise/expermenter wise error rate controlled when applicable Sufficiently large N Analysis 3____________________________________________________________________________ Appropriate unit of analysis Familywise/expermenter wise error rate controlled when applicable Sufficiently large N Overall Rating of Analysis: 68 3 2 1 0 Doctorate in Educational and Child Psychology Name of Coder: Pamela Hamilton Date:30/01/14 Full Study Reference in proper format: Sanders, M.R., Baker, S. & Turner, K.M.T. (2012). A randomized controlled trial evaluating the efficacy of triple p online with parents of children with early onset conduct problems. Behaviour Research and Therapy, 50, 675-684 Intervention Name (description of study): Triple P Online (Level 4) Study ID Number:__________________ Type of Publication: Book/Monograph Journal Article Book Chapter Other (specify): 1.General Characteristics A. General Design Characteristics A1. Random assignment designs (if random assignment design, select one of the following) Completely randomized design Randomized block design (between participants, e.g., matched classrooms) Randomized block design (within participants) Randomized hierarchical design (nested treatments A2. Nonrandomized designs (if non-random assignment design, select one of the following) Nonrandomized design Nonrandomized block design (between participants) Nonrandomized block design (within participants) Nonrandomized hierarchical design Optional coding for Quasi-experimental designs A3. Overall confidence of judgment on how participants were assigned (select on of the following) Very low (little basis) Low (guess) Moderate (weak inference) High (strong inference) Very high (explicitly stated) N/A Unknown/unable to code B Participants Total size of sample (start of study): 116 Intervention group sample size:60 Control group sample size:56 C. Type of Program Universal prevention program Selective prevention program Targeted prevention program 69 Doctorate in Educational and Child Psychology Pamela Hamilton Intervention/Treatment Unknown D. Stage of Program Model/demonstration programs Early stage programs Established/institutionalized programs Unknown E. Concurrent or Historical Intervention Exposure Current exposure Prior exposure Unknown – Parent/child current exposure to professional emotional/behavioural services was an exclusion criteria Section 2 Key Features for Coding Studies and Rating Level of Evidence/Support A Measurement (Estimating the quality of the measures used to establish effects) (Rating Scale: 3= Strong Evidence, 2=Promising Evidence, 1=Weak Evidence, 0=No Evidence) A1 The use of the outcome measures produce reliable scores for the majority of the primary outcomes (see following table for a detailed breakdown on the outcomes) Yes No Unknown/unable to code A2 Multi-method (at least two assessment methods used) Yes No N/A Unknown/unable to code A3 Multi-source (at least two sources used self-reports, teachers etc.) Yes No N/A Unknown/unable to code A4 Validity of measures reported (well-known or standardized or norm-referenced are considered good, consider any cultural considerations) Yes validated with specific target group In part, validated for general population only No Unknown/unable to code Overall Rating of Measurement: 70 3 2 1 0 Doctorate in Educational and Child Psychology Pamela Hamilton B Comparison Group B1 Type of Comparison group Typical intervention Attention placebo Intervention element placebo Alternative intervention Pharmacotherapy No intervention Wait list/delayed intervention Minimal contact Unable to identify type of comparison B2 Overall rating of judgment of type of comparison group Very low Low Moderate High Very high Unable to identify comparison group B3 Counterbalancing of change agent (participants who receive intervention from a single therapist/teacher etc were counter-balanced across intervention) By change agent Statistical (analyse includes a test for intervention) Other Not reported/None B4 Group equivalence established Random assignment Posthoc matched set Statistical matching Post hoc test for group equivalence B5 Equivalent mortality Low attrition (less than 20 % for post) Low attrition (less than 30% for follow-up) Intent to intervene analysis carried out? Findings: 8 % attrition rate post intervention and 14 % attrition rate at follow-up Overall Level of Evidence 3 3= Strong Evidence 2=Promising Evidence 71 1=Weak Evidence 0=No Evidence Doctorate in Educational and Child Psychology Name of Coder: Pamela Hamilton Date:30/01/14 Full Study Reference in proper format: Sanders, M.R., Baker, S. & Turner, K.M.T. (2012). A randomized controlled trial evaluating the efficacy of triple p online with parents of children with early onset conduct problems. Behaviour Research and Therapy, 50, 675-684 Intervention Name (description of study): Triple P Online (Level 4) Study ID Number:__________________ Type of Publication: Book/Monograph Journal Article Book Chapter Other (specify): 1.General Characteristics A. General Design Characteristics A1. Random assignment designs (if random assignment design, select one of the following) Completely randomized design Randomized block design (between participants, e.g., matched classrooms) Randomized block design (within participants) Randomized hierarchical design (nested treatments A2. Nonrandomized designs (if non-random assignment design, select one of the following) Nonrandomized design Nonrandomized block design (between participants) Nonrandomized block design (within participants) Nonrandomized hierarchical design Optional coding for Quasi-experimental designs A3. Overall confidence of judgment on how participants were assigned (select on of the following) Very low (little basis) Low (guess) Moderate (weak inference) High (strong inference) Very high (explicitly stated) N/A Unknown/unable to code B Participants Total size of sample (start of study): 116 Intervention group sample size:60 Control group sample size:56 C. Type of Program Universal prevention program Selective prevention program Targeted prevention program 72 Doctorate in Educational and Child Psychology Pamela Hamilton Intervention/Treatment Unknown D. Stage of Program Model/demonstration programs Early stage programs Established/institutionalized programs Unknown E. Concurrent or Historical Intervention Exposure Current exposure Prior exposure Unknown – Parent/child current exposure to professional emotional/behavioural services was an exclusion criteria Section 2 Key Features for Coding Studies and Rating Level of Evidence/Support A Measurement (Estimating the quality of the measures used to establish effects) (Rating Scale: 3= Strong Evidence, 2=Promising Evidence, 1=Weak Evidence, 0=No Evidence) A1 The use of the outcome measures produce reliable scores for the majority of the primary outcomes (see following table for a detailed breakdown on the outcomes) Yes No Unknown/unable to code A2 Multi-method (at least two assessment methods used) Yes No N/A Unknown/unable to code A3 Multi-source (at least two sources used self-reports, teachers etc.) Yes No N/A Unknown/unable to code A4 Validity of measures reported (well-known or standardized or norm-referenced are considered good, consider any cultural considerations) Yes validated with specific target group In part, validated for general population only No Unknown/unable to code Overall Rating of Measurement: 73 3 2 1 0 Doctorate in Educational and Child Psychology Pamela Hamilton B Comparison Group B1 Type of Comparison group Typical intervention Attention placebo Intervention element placebo Alternative intervention Pharmacotherapy No intervention Wait list/delayed intervention Minimal contact Unable to identify type of comparison B2 Overall rating of judgment of type of comparison group Very low Low Moderate High Very high Unable to identify comparison group B3 Counterbalancing of change agent (participants who receive intervention from a single therapist/teacher etc were counter-balanced across intervention) By change agent Statistical (analyse includes a test for intervention) Other Not reported/None B4 Group equivalence established Random assignment Posthoc matched set Statistical matching Post hoc test for group equivalence B5 Equivalent mortality Low attrition (less than 20 % for post) Low attrition (less than 30% for follow-up) Intent to intervene analysis carried out? Findings: 8 % attrition rate post intervention and 14 % attrition rate at follow-up Overall Level of Evidence 3 3= Strong Evidence 2=Promising Evidence 1=Weak Evidence 0=No Evidence C Appropriate Statistical Analysis Analysis 1Eyberg Child Behaviour Inventory Appropriate unit of analysis Familywise/experimenter wise error rate controlled when applicable Sufficiently large N Analysis 2 Strengths and Difficulties Questionnaire (parent) Appropriate unit of analysis Familywise/experimenter wise error rate controlled when applicable Sufficiently large N Analysis 3 Observation of child disruptive behaviour – Family Observation Schedule 74 Doctorate in Educational and Child Psychology Appropriate unit of analysis Familywise/experimenter wise error rate controlled when applicable Sufficiently large N Overall Rating of Analysis: 75 3 2 1 0 Pamela Hamilton Doctorate in Educational and Child Psychology Name of Coder: Pamela Hamilton Date:04/02/14 Full Study Reference in proper format:_ Sofronoff, K., Jahnel, D., & Sanders, M. (2011). Stepping Stones Triple P seminars for parents of a child with a disability: a randomized controlled trial. Research in Developmental Disabilities, 32(6), 2253-2262. Intervention Name (description of study):Stepping Stones Triple P Study ID Number:__________________ Type of Publication: Book/Monograph Journal Article Book Chapter Other (specify): 1.General Characteristics A. General Design Characteristics A1. Random assignment designs (if random assignment design, select one of the following) Completely randomized design Randomized block design (between participants, e.g., matched classrooms) Randomized block design (within participants) Randomized hierarchical design (nested treatments A2. Nonrandomized designs (if non-random assignment design, select one of the following) Nonrandomized design Nonrandomized block design (between participants) Nonrandomized block design (within participants) Nonrandomized hierarchical design Optional coding for Quasi-experimental designs A3. Overall confidence of judgment on how participants were assigned (select on of the following) Very low (little basis) Low (guess) Moderate (weak inference) High (strong inference) Very high (explicitly stated) N/A Unknown/unable to code B Participants Total size of sample (start of study): 70 Intervention group sample size:35 Control group sample size:35 76 Doctorate in Educational and Child Psychology Pamela Hamilton C. Type of Program Universal prevention program Selective prevention program Targeted prevention program Intervention/Treatment Unknown D. Stage of Program Model/demonstration programs Early stage programs Established/institutionalized programs Unknown E. Concurrent or Historical Intervention Exposure Current exposure -38% of parents reported that they were currently seeking professional help for their child’s EBD problems. Prior exposure Unknown Section 2 Key Features for Coding Studies and Rating Level of Evidence/Support A Measurement (Estimating the quality of the measures used to establish effects) (Rating Scale: 3= Strong Evidence, 2=Promising Evidence, 1=Weak Evidence, 0=No Evidence) A1 The use of the outcome measures produce reliable scores for the majority of the primary outcomes (see following table for a detailed breakdown on the outcomes) Yes No Unknown/unable to code A2 Multi-method (at least two assessment methods used) Yes No N/A Unknown/unable to code A3 Multi-source (at least two sources used self-reports, teachers etc.) Yes No N/A Unknown/unable to code A4 Validity of measures reported (well-known or standardized or norm-referenced are considered good, consider any cultural considerations) Yes validated with specific target group In part, validated for general population only No Unknown/unable to code Overall Rating of Measurement: 77 3 2 1 0 Doctorate in Educational and Child Psychology Pamela Hamilton B Comparison Group B1 Type of Comparison group Typical intervention Attention placebo Intervention element placebo Alternative intervention Pharmacotherapy No intervention Wait list/delayed intervention Minimal contact Unable to identify type of comparison B2 Overall rating of judgment of type of comparison group Very low Low Moderate High Very high Unable to identify comparison group B3 Counterbalancing of change agent (participants who receive intervention from a single therapist/teacher etc were counter-balanced across intervention) By change agent Statistical (analyse includes a test for intervention) Other Not reported/None B4 Group equivalence established Random assignment Posthoc matched set Statistical matching Post hoc test for group equivalence B5 Equivalent mortality Low attrition (less than 20 % for post) Low attrition (less than 30% for follow-up) Intent to intervene analysis carried out? Findings: 23% attrition rate post intervention and 37% attrition rate at follow-up (control group only) Overall Level of Evidence 1 3= Strong Evidence 2=Promising Evidence 78 1=Weak Evidence 0=No Evidence Doctorate in Educational and Child Psychology Pamela Hamilton C Appropriate Statistical Analysis Analysis 1 Eyberg Child Behavior Inventory (ECBI) Appropriate unit of analysis Familywise/experimenter wise error rate controlled when applicable Sufficiently large N Analysis 2____________________________________________________________________________ Appropriate unit of analysis Familywise/experimenter wise error rate controlled when applicable Sufficiently large N Analysis 3____________________________________________________________________________ Appropriate unit of analysis Familywise/experimenter wise error rate controlled when applicable Sufficiently large N Overall Rating of Analysis: 79 3 2 1 0