Pilot RCTs – a valuable way of testing the waters
Cook JA, Kolehmainen N.
Health Services Research Unit, University of Aberdeen, Health Sciences Building, Aberdeen, AB25 2ZD,
UK.
Conducting RCT of therapeutic interventions is challenging and hard work. This is often most painfully
apparent in the low rate of trial recruitment. This difficult is not wholly addressed by substantial study
funding nor by involving experienced researchers.[ref 1] Studies often fail to meet the original participant
target even with an extended recruitment period. Disappointing recruitment is usually driven by a legion of
factors, many outwith the control of the researchers conducting the study. Given the rather ubiquitous
nature of such problems, our understanding of the issues and our ability to successfully anticipate and
address them is remarkably limited.[ref 2]
In general, trials which run smoothly are rare. This appears especially true when evaluating a nonpharmacological intervention as the complex/multi-faceted nature of the intervention raises additional
issues.[ref 3, 4] When evaluating a complex therapeutic intervention, key questions which require careful
consideration include: what are the components of the interventions and how are these hypothesised to
result in the outcome; who delivers the intervention (are there training and expertise requirements?);
when and where is the intervention delivered (e.g. a single session or over a period of time? a hospital
clinic and/or at home?); are others required to co-deliver the intervention (e.g. is the patient required to
carry out daily exercises?); and, how and when the interventions will be evaluated (e.g. patient reported
outcomes at 12 months?). Given the complexity of these issues, pilot and feasibility studies have become
the recommended means[ref 3] to test the waters and identify potential problems and solutions prior to
conducting the main study.[ref 5]
Elsewhere in this issue, [add the authors’ names] describe a pilot RCT investigating the effectiveness of
joint protection advice for patients with osteoarthritis of hand [ref 6]. The authors are to be commended for
seeking to rigorously evaluate effectiveness of additional joint protection advice and related practice and
feedback (when compared to usual practice of prescribing exercises only). As they state, due to the very
small sample size, this pilot study does not to provide evidence of the relative benefit of the interventions.
However, it does highlight some of the challenges faced in conducting research in a health care system
where research is constrained by established care pathways and processes. It also re-iterates some of
the specific challenges related to evaluating complex interventions where at least some aspects of the
intervention need to be tailored to the individual patients’ circumstances. One common challenge is that
while it is often relatively clear what the “new” intervention to be evaluated consists of, the intervention
which to compare it against (i.e. the “control”) can be difficult to determine. The default “usual care” can
be a rather loose and ill-defined term varying between health professionals and centres. As [name the
authors] et al[ref 6] found, use of this control without clear specification can make it difficult to distinguish
between the “intervention” and the “control” and to establish what is actually being assessed.
There is a general agreement that rehabilitation interventions require to be rigorously evaluated to ensure
optimal therapy provision and ultimately improved health outcomes for patients. Where there is genuine
uncertainty about critical aspects of the planned RCT, conducting a pilot or feasibility study can be a
powerful way of uncovering unforeseen challenges and identifying possible solutions.
References
1. McDonald A, Knight R, Campbell MK, Entwistle V, Grant A, Cook JA, et al. What influences
recruitment to randomised controlled trials? A review of trials funded by two UK funding agencies.
Trials, 7:9, 2006.
2. Treweek S, Mitchell E, Pitkethly M, Cook J, Kjeldstrøm M, Taskilla T, Johansen M, Sullivan F,
Wilson S, Jackson C, Jones R. Strategies to improve recruitment to research studies. Cochrane
Methodology Review. Cochrane Library 2010 Issue 1.
3. Craig P, Dieppe P, Macintyre S, Michie S, Nazareth I, Petticrew M. Developing and evaluating
complex interventions: the new Medical Research Council guidance. BMJ 2008;337:a1655.
4. Cook J. The design, conduct and analysis of surgical randomised trials. Trials 2009 10:9.
5. Lancaster G, Dodd S, Williamson PR. Design and analysis of pilot studies: recommendations for
good practice. Journal of Evaluation in Clinical Practice, 10, 2, 307–312.
6. Ref paper