Journal of Hospital Infection 76 (2010) 292e295 Available online at www.sciencedirect.com Journal of Hospital Infection journal homepage: www.elsevierhealth.com/journals/jhin Opinion Randomisation and sample size for clinical audit on infection control I. Fournel*, M. Tiv, C. Hua, M. Soulias, K. Astruc, L.S. Aho Hospital Hygiene and Epidemiology Unit, Hôpital du Bocage, Dijon, France a r t i c l e i n f o s u m m a r y Article history: Received 17 November 2009 Accepted 28 May 2010 Available online 6 August 2010 Clinical audit is both a part of clinical governance and an essential component of infection prevention and control. It is frequently performed on a proportion of the target population. The sample should represent the source population and be sufficient for statistical analysis. In a hospital, infection control practices are likely to be quite similar within the same clinical area (cluster effect). This must be taken into consideration when calculating the necessary number of patients. Sample size is determined by the desired level of precision for estimating the compliance rate, or by the difference between observed and expected rates, or on the difference before and after implementation of interventions. To estimate the hospital-wide compliance rate without additional costs we suggest focusing the audit on a large number of wards, even if fewer observations within each ward are obtained, rather than auditing a large number of practices on a restricted number of wards. Ó 2010 The Hospital Infection Society. Published by Elsevier Ltd. All rights reserved. Keywords Clinical audit Cluster randomisation Infection control Representativeness Sample size Quality in healthcare Introduction Clinical audit, defined as ‘a quality improvement process that seeks to improve patient care and outcomes through systematic review of care against explicit criteria and the implementation of change’, is a part of clinical governance.1 Evaluating professional practices is also an essential component of infection prevention and control.2e4 Indeed, infection control audits present an opportunity to promote infection prevention as well as to suggest care improvement strategies in partnership with an organisation’s multidisciplinary teams.5 Usually these audits imply direct observation of hygiene practices. An external auditor (infection control nurse and/or practitioner) observes hygiene practices in various settings and for different healthcare professionals. Although several guidelines on clinical audit are available, the specific modalities of infection control audits have not been defined as far as we know.1,6 Indeed, the main feature of hygiene audits is that practices tend to look similar within the same wards but differ between departments (cluster effect), which should be taken into account when conducting an audit. * Corresponding author. Address: Hospital Hygiene and Epidemiology Unit, Hôpital du Bocage, BP 77908, 21079 Dijon Cedex, France. Tel.:þ33 380293394; fax: þ33 380293497. E-mail address: isabelle.fournel@chu-dijon.fr (I. Fournel). The purpose of this article is to review the different modalities in terms of sampling and of required number of subjects when auditing hygiene at hospital level. Sample representativeness The comprehensive assessment of professional practices in all heathcare units during a given period is both labour-intensive and difficult to implement. Thus, in practice, to limit the number of observations and resources needed to carry out the audit, we try to select a sample of observations which is representative of the source population.1 The target population needs to be defined first. This depends upon the context, which induced an audit of infection control practices. Audit can be carried out on request from health authorities (national audit for instance) or on request from a specific department such as, for instance, evaluating infection control practices in case of an outbreak. In the first case, the practices will be assessed in a large number of hospital departments, whereas in the second case, the audit will be restricted to a specific ward. Randomisation is required to obtain a representative population sample, from which the results can be extrapolated to the target population. If the audit covers the entire hospital or several wards, healthcare practices tend to be much alike within the same clinical area (cluster effect) due to health professionals’ working habits, ward specialty, service policy, and due to standardised protocols sometimes set up in specific services. This cluster effect must 0195-6701/$ e see front matter Ó 2010 The Hospital Infection Society. Published by Elsevier Ltd. All rights reserved. doi:10.1016/j.jhin.2010.05.025 I. Fournel et al. / Journal of Hospital Infection 76 (2010) 292e295 therefore be taken into account when sampling. Table I summarises various sampling approaches. Number of observations In addition to its representativeness the sample must be of a sufficient size to achieve statistical significance. Sample size calculation is required prior to any study. It depends on the desired level of precision for descriptive studies. In case of comparison with another population or a reference value, the required sample size depends on the power and precision of the statistical test and on the size of the difference. To demonstrate a small difference in compliance rates requires a larger sample number than showing a large difference. Power and precision increase with sample size. For clinical audits evaluating the compliance of the observed practices with the reference standard the required number of subjects will be calculated by the desired precision level of the expected compliance rate. In this case, the calculation of the number of required subjects depends on the expected compliance rate and the desired precision of the estimate with an error a set (that is for a degree of confidence 1a).7 For estimation of a single proportion: n ¼ z2 pq i2 293 of an attribute that is present in the population (here compliance expected before the implementation of corrective actions). Example 2: An audit on hand hygiene is performed after corrective interventions. The compliance was 50% before and the expected compliance is 70%. The sample size required to detect such a difference with 80% power and 5% a risk is 93 per group. Which statistical unit? The sample size or the required number of subjects described above refers to statistical units. In clinical audit this number can correspond either to the number of observations or to the number of different healthcare professionals. When the statistical unit is the number of observations, the sample size refers to the number of observations to be made. When the statistical unit is the number of subjects, the sample size refers to the number of health professionals to be observed. In this case, the same subject can be observed several times. If healthcare professionals have been sampled from several wards, there will be two levels of clustering: professionals on the same ward and observations on the same professionals (Figure 1). A suitable statistical analysis is needed to take this two-level clustering into account, using a random effects model, for example. (1) Cluster effect where n is sample size, z2 is the abscissa of the normal curve that cuts off an area at the tails (1 is equal to the desired confidence level, e.g. 95%; the value for z is found in software or in statistical tables which contain the area under the normal curve); i is the desired level of precision (e.g. i ¼ 0.05); p is the estimated proportion of an attribute that is present in the population (here compliance expected); q is 1p. Example 1: in the case of an audit dealing with hand hygiene, in which z ¼ 1.96, i ¼ 0.05 and expected compliance ¼ 50%, the required sample size is 384 (1.962 0.5 0.5/0.052). As mentioned above, the assessment may also focus on comparing the observed with the expected standard rate, or, in the case of repeated audits, comparing the rate before and after corrective actions.1,8 A simple formula for comparing two proportions (p1 and p0) is: 2 za=2 z1b ðk þ 1Þ n1 ¼ pffiffiffiffiffi pffiffiffiffiffi 2 and n0 ¼ kn1 k 2jArcsin p1 Arcsin p0 j (2) where n1 is the sample size in group 1 and n0 the sample size in group 2; p1 is the estimated proportion of an attribute that is present in the population (here compliance expected after the implementation of corrective actions); and p0 is the estimated proportion It is necessary to take into account the fact that practices appear to be more similar within than between departments. This cluster effect decreases power because individuals in the same cluster are more homogeneous.9,10 To maintain the same power, we must correct the number of subjects calculated with formula 1 previously given by the design effect (DE), also called coefficient of inflation:11 ncorrected ¼ z2 pq DE i2 (3) When the clusters are of equal size, the design effect is calculated as follows: DE ¼ 1 þ rðm 1Þ (4) where m is the number of individuals in a cluster. The intracluster correlation coefficient r is calculated as follows: 12 r¼ s2b s2b (5) þ s2w where s2b is the variance between clusters, and s2w the variance within clusters. Table I Sampling procedures used in audits Scope Sampling methods Benefits Drawbacks Only one hospital department All patients or all events during an entire audit period within one department e Suitable for individual problems or resulting from a targeted request e Completeness Does not permit a comprehensive hospital analysis Several departments Cluster randomisation: randomisation of patients or events during the entire period of audit e To obtain data on practices in various departments Sample bias caused by choice of departments All departments at hospital-wide level All departments are audited. For each department, the audit period is randomised. e Easier to organise e A deficiency in a service could be demonstrated Period effect may occur, e.g. service auditing during an influenza outbreak Patients or care activities chosen individually at hospital-wide level Randomisation of patients or events individually at entire hospital level e Optimum power at an equivalent sample size Difficult to implement: e An exhaustive list of patients or events is required e Services with low activity may never be audited e Mobilise many staff members simultaneously 294 I. Fournel et al. / Journal of Hospital Infection 76 (2010) 292e295 Ward 1 Ward 2 Professional A Professional B Professional C Professional A Professional B Professional C Observation 1 Observation 2 Observation 3 Observation 1 Observation 2 Observation 3 Observation 1 Observation 2 Observation 3 Observation 1 Observation 2 Observation 3 Observation 1 Observation 2 Observation 3 Observation 1 Observation 2 Observation 3 Figure 1. Levels of clustering in hygiene audit. The intracluster correlation coefficient measures the intensity of the link between observations in the same cluster by comparing intra- and intercluster variances.12 It can vary from 0 to 1. The closer it is to 1, the more similar are hygiene practices within the same departments and the more they differ from those in other departments. This method of calculation is appropriate when the clusters are of the same size. When cluster sizes are unbalanced, the design effect is underestimated, which thus leads to a decrease in study power.13 Some authors have proposed calculating the design effect weighting by cluster size.13 For a fixed total number of subjects the number of clusters increases power.14,15 In order to obtain the best snapshot of infection control compliance at hospital level, we suggest auditing a large number of wards rather than a high number of care practices in a restricted number of wards, even if we obtain fewer observations within each. Table II summarises sample size for different compliance rates including a correction for cluster effect. When an audit is carried out on multiple wards, it is important that the data are analysed in a way that takes this clustering into account. Otherwise, the standard error of the final estimate will tend to be too small and there will be a tendency to overstate the precision of the findings. Discussion Voluntary participation in audit may lead to a selection bias because volunteers are probably more compliant with good hygiene practices. Invitation to participate in hospital audit may lead to greater involvement of the various services. The ideal would be to embed the audit into as many different wards as possible. The calculation of the required number of subjects is rarely reported in published audits and seems to be more related to practicability than to a-priori sample size calculation. The French National Authority for Health considers that a number of observations between 30 and 50 observations is a good compromise.16 Other authors state that 20e50 observations should be sufficient for single audits.8 To our knowledge, there is no justification for these numbers, except that numbers of observations exceeding 30 maximise the conditions for the application of parametric statistical tests. Indeed, the distribution of a continuous variable can sometimes be approximated by the normal distribution.17 For the British National Audit Office, a sample of fewer than 30 subjects is a case series, for which it is not possible to extrapolate the results to the whole target population.18 The French group for evaluation of hospital hygiene practices (GREPHH) website stipulates 30 as the minimum number of observations for analysis. However, the cluster effect is rarely taken into account in this type of study. Estimating the variability between clusters is the main difficulty. Table II demonstrates that the sample size for a compliance rate of 90% can range between 138 and 1383, if 10 subjects or events are observed in each cluster (e.g. ward). The development of national hygiene audits could facilitate obtaining the data necessary for calculating the intraclass correlation coefficient and thus the required number of subjects could become predictable. When the statistical unit is the number of subjects and when this number per cluster is small, Kerry estimates that the design effect can be ignored.11 Nevertheless the threshold from which he estimated that the number of subjects per cluster is small has not yet been defined. In conclusion, to obtain the best snapshot of overall compliance rates hospital-wide at constant means, we recommend that future audits on infection control should focus on a large number of wards even if the observations within each ward are few, rather than auditing a high number of care activities in a small number of services. Randomisation is required to ensure a good representation of results. The choice of wards and departments should be adjusted to meet the purpose of the audit as well as for the nature of corrective interventions. Table II Sample size corrected for intracluster correlation coefficient (ICC) for different compliance rates at 95% confidence level and 10 subjects per cluster Compliance (%) 10 20 30 40 50 60 70 80 90 ICC 0 0.01 0.10 0.20 0.30 0.40 0.50 0.60 0.70 0.80 0.90 1 138 246 323 369 384 369 323 246 138 151 268 352 402 419 402 352 268 151 263 467 613 701 730 701 613 467 263 387 688 904 1033 1076 1033 904 688 387 512 910 1194 1365 1421 1365 1194 910 512 636 1131 1484 1696 1767 1696 1484 1131 636 761 1352 1775 2028 2113 2028 1775 1352 761 885 1574 2065 2360 2459 2360 2065 1574 885 1010 1795 2356 2692 2804 2692 2356 1795 1010 1134 2016 2646 3024 3150 3024 2646 2016 1134 1259 2237 2937 3356 3496 3356 2937 2237 1259 1383 2459 3227 3688 3842 3688 3227 2459 1383 The closer ICC (Formula 5) is to 1, the more similar are hygiene practices within the same departments and the more they differ from those in other departments. I. Fournel et al. / Journal of Hospital Infection 76 (2010) 292e295 Acknowledgements We thank M. Fournel for assistance in editing. Conflict of interest statement None declared. Funding sources None. References 1. National Institute for Clinical Excellence. Principles for best practice in clinical audit. Oxford: Radcliffe Medical Press; 2002. 2. Scottish Office Department of Health Advisory Group on Infection. Scottish infection manual. Edinburgh: SODH; 1998. 3. Hajjar J. Healthcare associated infection control in France: 2005e2008 national program. J Hosp Infect 2008;70:17e21. 4. Millward S, Barnett J, Thomlinson D. Evaluation of the objectivity of an infection control audit tool. J Hosp Infect 1995;31:229e233. 5. Bryce EA, Scharf S, Walker M, Walsh A. The infection control audit: the standardized audit as a tool for change. Am J Infect Control 2007;35:271e283. 6. Copeland G. A practical handbook for clinical audit. London: National Clinical Governance Support Team; 2005. 295 7. United Bristol Healthcare NHS Trust Clinical Audit Central Office. How to select an audit sample. UK: UBHT Clinical Audit Central Office; 2005. 8. Cochran WG. Sampling techniques. 2nd ed. New York: Wiley; 1963. 9. Cosby RH, Howard M, Kaczorowski J, Willan AR, Sellors JW. Randomizing patients by family practice: sample size estimation, intracluster correlation and data analysis. Fam Pract 2003;20:77e82. 10. Donner A. An empirical study of cluster randomization. Int J Epidemiol 1982;11: 283e286. 11. Kerry SM, Bland JM. The intracluster correlation coefficient in cluster randomisation. BMJ 1998;316:1455. 12. Killip S, Mahfoud Z, Pearce K. What is an intracluster correlation coefficient? Crucial concepts for primary care researchers. Ann Fam Med 2004;2:204e208. 13. Eldridge SM, Ashby D, Kerry S. Sample size for cluster randomised trials: effect of coefficient of variation of cluster size and analysis method. Int J Epidemiol 2006;32:1292e1300. 14. Guittet L, Ravaud P, Giraudeau B. Planning a cluster randomised trial with unequal cluster sizes: practical issues involving continuous outcomes. BMC Med Res Methodol 2006;6:17. 15. Guittet L, Giraudeau B, Ravaud P. A priori postulated and real power in cluster randomised trials: mind the gap. BMC Med Res Methodol 2005;5:25. 16. Agence Nationale pour le Développement de l’Evaluation Médicale. L’évaluation des pratiques professionnelles dans les établissements de santé. L’audit clinique. Paris: ANDEM; 1994. 17. Jekel JF, Katz DL, Elmore JG. Epidemiology, biostatistics and preventive medicine. Philadelphia: WB Saunders; 2001. 18. National Audit Office. Statistical and technical team. A practical guide to sampling. London: National Audit Office; 1998.