advertisement

(Summaries written by Laura Collett, Leeds Institute of Clinical Trials Research, University of Leeds) Trials with partial nesting, presentation by Chris Roberts Some interventions require comparisons between two interventions or treatments given in different ways, where one arm includes clustering, and the other does not; for example, group therapy versus a drug. This leads the clustering structure to be incomplete, which has implications for the analysis. Problems highlighted in the analysis of trials with partial nested designs included between treatment heteroscedasticity of continuous outcome measures, and consistency and small sample bias with binary outcome measures. Parameterisation of linear mixed effect models, including random intercept, random coefficient and heteroscedastic models, and their not nested and nested total variance were summarised including the estimate of the ICC. Two trials using partially nested designs were discussed including estimates of effect size, standard error and ICC for different models. Simulation study comparing random effects models, GEEs and Sattherthwaite’s ttest was carried out with the cluster arm as 8 groups of size 6, and the individual arm as 48 individuals. The ratio of individual treatment variance to group variance was set between 0.5 and 2.5, where 7300 simulations were used to estimate test size with precision of 0.005. Graphical summaries of the difference in ICC as the variance ratio (individual/nested) changed between the random intercept, random coefficient, heteroscadastic and GEE models were shown. The same models, with the addition of the Sattherthwaite’s t-test, were compared for empirical size of a 5% level test as the variance ratio increased, showing that GEE models had an empirical level at around 0.075 and random intercept model level decreasing as the variance ratio increased. The conclusions from partial nesting and heteroscedasticity in continuous outcomes is that where total variance of the un-clustered arm is greater the ICC will be increased in random intercept models and decreased in random coefficient and GEE models, an issue that is analogous to the choice between a t-test and the Sattherthwaite t-test when comparing two means with unequal variances, and where there is reason to believe the variances are unequal, the Sattherthwaite t-test should be used. For partially nested binary data, parameterisation of logistic models, including logistic GEEs with robust standard errors, logistic random intercept and logistic random coefficient were summarised including adjusted test for proportions including the design effect. The estimates of the marginal proportions were summarised for each model in the clustered and non-clustered arms. Graphical representation of the log OR when the proportions for the clustered and non-clustered arms were equal and as that proportion increased were shown for different values of the ICC based on different cluster sizes and number of simulations. Graphical representation of the small sample bias for the null treatment effect on the log odds as the proportion increased for different simulation sample sizes were discussed. Models based on simulations comparing 10 clusters of size 10 with 100 individuals, were also compared for their empirical test size and it showed that most models shown were biased for smaller and larger proportions for the lower and upper 2.5% tail respectively, where the LRT performed the best out the models fitted. Stata commands such as ‘sampsi’ and ‘clsampsi’ for estimating power and sample size for clustered trial designs were described and a comparison of ‘clsampsi’ with empirical power between different models with different methods of calculation were made in terms of bias. The use of unequal randomisation in favour of the clustered arm to reduce bias was discussed and a ratio to incur maximum power was put forward, including graphical representations of empirical test sizes between methods with different ratios of unequal randomisation to describe their performance in terms of bias. Note that unequal randomisation can have a detrimental effect on power in individually randomised trial so should be avoided. Design and implementation considerations for a Cluster Randomised Trial in general practice using balanced incomplete blocks, presentation by Amanda Farrin Once research has been conducted, implementation into general practice is required. Research can sometimes produce a lot of waste, be it waste from addressing and incorporating inappropriate questions and designs, lack of accessibility of results, or inherent bias gained during interpretation. The aim of some types of research is to minimise the gap between research and practice, as dissemination of quality research is essential but not sufficient to ensure correct and applicable implementation of interventions. This is especially evident in primary care settings, as it is unfeasible to invent implementation strategies for every new guideline, so adaptable and hence generalisable strategies are needed. ASPIRE (Action to Support Practices Implementing Research Evidence) aims to develop and evaluate an adaptable intervention package to target implementation of ‘high impact’ clinical practice recommendations in general practice; and to focus on recommendations where a measurable change in clinical practice can lead to significant patient benefit. ASPIRE is a cluster randomised controlled trial set in general practices in West Yorkshire to evaluate an adaptable intervention package for two high impact recommendations. The trial will use GP routine data in order to identify whether and how interventions improve patient care, outcomes and cost-effectiveness. The trial is a randomised controlled trial, as this is the optimum design when evaluating behaviour change. It also uses a cluster design, as the inclusion of patient and clinician levels are necessary in implementing clinical practice strategies, and thus to avoid contamination from individual clinician behaviour; randomisation is at the level of the clinician and outcome is at the level of the patient. The trial also incorporates a balanced 2x2 incomplete block design where half of the 60 practices are randomised to an intervention of recommendation 1 and a control of recommendation 2, and the other half are randomised to a control of recommendation 1 and an intervention of recommendation 2. This design has been deemed appropriate for the setting to simultaneously minimise the Hawthorne effect (the non-specific effects arising through trial participation) and maximise power and efficiency. Considering the use of two recommendations, sample size is based on two embedded RCTs, requiring 12,000 records from 60 practices, with a minimum 100 recorded per practice, per recommendation. This provides 90% power to detect 15% difference in adherence rates using 2.5% significance level (to adjust for the use of two outcome comparisons), assuming control adherence at 55% and an ICC of 0.1 (thought to be relatively conservative). A median effect size of 9% has been observed in other guideline implementation studies when implementing recommendations to practices with greater scope for improvement. Design challenges that were highlighted were the assumptions used in calculating sample size, and the choice of recommendations to evaluate. Issues such as cluster size, sample size adjustment for unequal clusters, coefficient of variation, and the design effect, were raised. In addition, assumptions based on increase in adherence were highlighted due to high levels of adherence and variation in adherence apparent for some recommendations. In terms of choice of recommendations it was discussed how to identify ‘high impact’ recommendations based on various criteria. In addition, for the use of 2x2 incomplete block design using 2 recommendations, independence of recommendations must be assumed, as some recommendations intrinsically overlap. Another discussion point highlighted the way interventions were implemented and whether opt-in or opt-out consent processes were preferable given the mechanism of the recommendation; opt-out deemed to be the more pragmatic choice following group discussion. Conclusions were based around the need to keep it simple; minimise bias and maximise generalisability, and to test assumptions as thoroughly as possible using real data. Incomplete block designs can overcome some challenges in implementing research in general practice. Statistical considerations for the design and analysis of non-inferiority trials, presentation by David Gillespie Standard superiority trials are set up in order to detect whether or not a treatment is different from a control in terms of a pre-specified endpoint, but if there is insufficient evidence to conclude a difference, this does not imply that the treatment and the control are the same, as you cannot ‘accept’ the null hypothesis of no difference. Despite this, you may wish to obtain evidence that a treatment is ‘no worse’ than the control, in terms of one pre-specified endpoint, given favourable evidence that other factors are better for that treatment, such as toxicity or cost. Given these points, you may wish to conduct a non-inferiority trial, in order to test whether a new treatment is ‘no worse’ according to a pre-specified margin. The choice of non-inferiority margin to use is usually a combination of statistical and clinical input, given considerations such as the trade-off between the margin of acceptability in efficacy and other benefits such as fewer side-effects, and given that the margin should always be smaller than the difference observed when comparing the treatment with a placebo, including the risk of bio-creep (the process of one treatment being found to be non-inferior, subsequently becoming the standard, until another treatment is again found to be non-inferior, until efficacy of the ‘standard treatment’ is reduced). As the non-inferiority margin is almost always smaller than the corresponding clinically important difference used in superiority trials, this requires the sample size required in order to detect this difference to be larger – a four-fold increase in sample size is required when the non-inferiority margin is half the size of the clinically important difference, assuming a mean difference of zero. When analysing a non-inferiority trial, the current recommendation of analysis set is to perform both intention-to-treat and per-protocol analyses, and if both give evidence of non-inferiority then non-inferiority can be concluded. Other methods of analysis that can be included are randomisation based efficacy, complier average causal effect, and the use of one-sided or two-sided confidence intervals. In addition, if the original design was testing for non-inferiority, but during analysis there is evidence for superiority; then superiority can be concluded based on this evidence without concerns over multiplicity and power, due to the larger sample size available due to the larger clinically important difference. The reverse however is not usually appropriate, unless considered a priori and sample size is enlarged accordingly. Discussion points included the input of a statistician in defining a non-inferiority margin, the choice of confidence interval including the appropriate level and one/twosided nature, and the choice of analysis set.