Articles Early palliative care for patients with advanced cancer: a cluster-randomised controlled trial Camilla Zimmermann, Nadia Swami, Monika Krzyzanowska, Breffni Hannon, Natasha Leighl, Amit Oza, Malcolm Moore, Anne Rydall, Gary Rodin, Ian Tannock, Allan Donner, Christopher Lo Summary Background Patients with advanced cancer have reduced quality of life, which tends to worsen towards the end of life. We assessed the effect of early palliative care in patients with advanced cancer on several aspects of quality of life. Methods The study took place at the Princess Margaret Cancer Centre (Toronto, ON, Canada), between Dec 1, 2006, and Feb 28, 2011. 24 medical oncology clinics were cluster randomised (in a 1:1 ratio, using a computer-generated sequence, stratified by clinic size and tumour site [ four lung, eight gastrointestinal, four genitourinary, six breast, two gynaecological]), to consultation and follow-up (at least monthly) by a palliative care team or to standard cancer care. Complete masking of interventions was not possible; however, patients provided written informed consent to participate in their own study group, without being informed of the existence of another group. Eligible patients had advanced cancer, European Cooperative Oncology Group performance status of 0–2, and a clinical prognosis of 6–24 months. Quality of life (Functional Assessment of Chronic Illness Therapy—Spiritual Well-Being [FACIT-Sp] scale and Quality of Life at the End of Life [QUAL-E] scale), symptom severity (Edmonton Symptom Assessment System [ESAS]), satisfaction with care (FAMCARE-P16), and problems with medical interactions (Cancer Rehabilitation Evaluation System Medical Interaction Subscale [CARES-MIS]) were measured at baseline and monthly for 4 months. The primary outcome was change score for FACIT-Sp at 3 months. Secondary endpoints included change score for FACIT-Sp at 4 months and change scores for other scales at 3 and 4 months. This trial is registered with ClinicalTrials.gov, number NCT01248624. Findings 461 patients completed baseline measures (228 intervention, 233 control); 393 completed at least one followup assessment. At 3-months, there was a non-significant difference in change score for FACIT-Sp between intervention and control groups (3·56 points [95% CI –0·27 to 7·40], p=0·07), a significant difference in QUAL-E (2·25 [0·01 to 4·49], p=0·05) and FAMCARE-P16 (3·79 [1·74 to 5·85], p=0·0003), and no difference in ESAS (–1·70 [–5·26 to 1·87], p=0·33) or CARES-MIS (–0·66 [–2·25 to 0·94], p=0·40). At 4 months, there were significant differences in change scores for all outcomes except CARES-MIS. All differences favoured the intervention group. Interpretation Although the difference in quality of life was non-significant at the primary endpoint, this trial shows promising findings that support early palliative care for patients with advanced cancer. Funding Canadian Cancer Society, Ontario Ministry of Health and Long Term Care. Introduction The complex needs of patients with advanced cancer and their caregivers arise many months before the patient’s death.1 Correspondingly, WHO defines palliative care as “an approach that improves the quality of life of patients and their families…by means of early identification and impeccable assessment and treatment of pain and other problems, physical, psychosocial and spiritual.”2 Specialised palliative care teams enact this approach, through the holistic care of patients dying from cancer or other terminal illnesses, and their presence is increasing worldwide.3 However, most of these teams provide terminal care at home or to inpatients rather than in outpatient settings,4–6 and referral to palliative care teams for most patients occurs in the last 2 months of life or not at all.7,8 We did a systematic review9 of randomised controlled trials from 1984 to 2007 assessing the effectiveness of specialised palliative care. Not all studies assessed a palliative care team, with interventions including a coordinating service, a nursing intervention, or counselling. Only four of 13 studies assessing quality of life had significant findings. However, most lacked statistical power and were done late in the disease process, resulting in difficulties with recruitment, attrition, and co-intervention. None specifically assessed an early palliative care intervention in patients with cancer. Since publication of this review,9 results have been reported from two randomised controlled trials assessing early palliative care interventions in patients with advanced cancer. The first was a study10 of 322 participants with advanced cancer and a prognosis of about 1 year; patients were randomised to routine care or to a palliative care problem-solving intervention through telephone contact from advanced practice nurses. In this study, and in the second, a trial11 of 151 patients with advanced nonsmall-cell lung cancer, patients randomised to the early palliative care group had better quality of life and mood. www.thelancet.com Published online February 19, 2014 http://dx.doi.org/10.1016/S0140-6736(13)62416-2 Published Online February 19, 2014 http://dx.doi.org/10.1016/ S0140-6736(13)62416-2 See Online/Comment http://dx.doi.org/10.1016/ S0140-6736(13)62676-8 See Online for video Division of Medical Oncology and Haematology, Department of Medicine (C Zimmermann MD, M Krzyzanowska MD, B Hannon MBChB, N Leighl MD, Prof A Oza MD [Lon], Prof M Moore MD, Prof I Tannock MD), and Department of Psychiatry (C Zimmermann, Prof G Rodin MD, C Lo PhD), University of Toronto, Toronto, ON, Canada; Department of Psychosocial Oncology and Palliative Care (C Zimmermann, N Swami BSc, B Hannon, A Rydall MSc, Prof G Rodin, C Lo), Department of Medical Oncology (M Krzyzanowska, N Leighl, Prof A Oza, Prof M Moore, Prof I Tannock), and Campbell Family Cancer Research Institute (C Zimmermann, Prof G Rodin, Prof I Tannock), Princess Margaret Cancer Centre, University Health Network, Toronto, ON, Canada; and Western University, London, ON, Canada (Prof A Donner PhD) Correspondence to: Dr Camilla Zimmermann, Department of Psychosocial Oncology and Palliative Care, Princess Margaret Cancer Centre, Toronto, ON, Canada camilla.zimmermann@uhn.ca 1 Articles However, the former study10 did not assess intervention by a palliative care team and the latter11 included only patients with non-small-cell lung cancer. We report a cluster-randomised controlled trial of early palliative care in patients with advanced cancer. We opted for cluster rather than individual randomisation (randomising clinics, rather than individual patients) on the basis of evidence from the health-services literature9,12,13 and advice from oncologists that it is difficult to recruit patients to be individually randomised (or not) to an intervention such as palliative care, in view of strong preconceived preferences among patients and their oncologists. The design implications of cluster-randomised trials were thoroughly taken into account in our trial.14 We postulated that, compared with standard cancer care, early intervention (clinical prognosis of 6–24 months) by a palliative care team would be associated with improved patient quality of life, symptom control, and satisfaction with care, and less difficulty with clinician– patient interactions. Methods Study design and participants The study took place at Princess Margaret Cancer Centre, a comprehensive cancer centre and part of the University Health Network in Toronto, ON, Canada, between Dec 1, 2006, and Feb 28, 2011. Recruitment involved daily screening of participating oncology clinics by research personnel to establish eligibility. Eligible patients were aged 18 years or older, had stage IV cancer (for breast or prostate cancer, refractory to hormonal therapy was an additional criterion; patients with stage III cancer and poor clinical prognosis were included at the discretion of the oncologist); an estimated survival of 6–24 months (assessed by their main oncologist);15 and Eastern Cooperative Oncology Group (ECOG) performance Early palliative care intervention Standard care Outpatient clinics Staff Palliative care physician and nurse Oncologist and oncology nurses Visits Routine once monthly; more often if necessary Ad hoc; mainly based on chemotherapy or radiation schedule Symptom assessment in clinic Routine, structured assessment during every visit by palliative care nurse and physician No structured assessment Psychosocial assessment in clinic Routine assessment and discussion of goals of care, of patient and family support needs, and of patient and family coping and psychological distress; discussion of advance care planning according to patient and family readiness No routine assessment Telephone follow-up Routine by palliative care nurse after each visit; more often as As needed by oncology nurse; rare access to oncologist needed by palliative care nurse and physician On-call service 24-h on-call service explained during first visit; provided by specialised palliative care physicians Access to 24-h on-call service (oncology resident or clinical associate) Inpatient care Direct access to palliative care unit for symptom management No access to palliative care unit; admission to oncology ward or medical ward (via emergency department for urgent care) Inpatient staff Primary care by trained palliative care nurses and physicians Primary care by oncology nurses and clinical associates Palliative care training for nurses Formal 10-day training at opening of palliative care unit, and continued education by palliative care unit advance practice nurse No formal palliative care training Palliative care inpatient follow-up Follow-up by palliative care team when admitted to No follow-up by palliative care team non-palliative-care-unit service at University Health Network Hospital service Home care Community care access centre services* Explained and offered during first visit; reassessed at each visit Ad hoc; generally no home care referral until referral to palliative care team Communication with family physician and community care access centre Routine Rarely; ad hoc Home palliative care physician† Explained during first visit; offered when ECOG performance status ≥3 or when patient requests None Multidisciplinary, addressing physical, psychological, social and spiritual needs Ad hoc, mainly addressing physical needs Approach to care All care providers ECOG=Eastern Cooperative Oncology Group. *Community care access centre services include personnel such as nursing, personal support, physical therapy, occupational therapy, and equipment such as hospital bed, walker, wheelchair. †Home palliative care physicians provide either back-up support to family physicians doing house calls or direct care if (as is the case for most patients) the family physician does not provide house calls. Table 1: Comparison of early palliative care intervention and standard care 2 www.thelancet.com Published online February 19, 2014 http://dx.doi.org/10.1016/S0140-6736(13)62416-2 Articles status of 0, 1, or 2 (assessed by their main oncologist).16 Consenting patients completed baseline measures to be eligible. Exclusion criteria were insufficient English literacy to complete questionnaires and inability to pass the cognitive screening test (Short Orientation-MemoryConcentration Test score <20 or >10 errors).17 Patients provided written, informed consent to participate and the study was approved by the Research Ethics Board of the University Health Network. Randomisation and masking In this cluster-randomised controlled trial, oncology clinics were the units of randomisation and patients the units of inference.14 A clinic was defined as a unique, consistent time and location for outpatient oncology care by a medical oncologist. Written consent was obtained from all 16 medical oncologists in the five largest site groups (lung, gastrointestinal, genitourinary, breast, and gynaecology) to randomise their 24 clinics either to immediate consultation and follow-up by a palliative care team, or to standard care. Randomisation was done by the statistical team at Western University (London, ON, Canada) using a computer-generated sequence, was in a 1:1 ratio, and was stratified by clinic size and tumour site: lung (four clinics), gastrointestinal (eight clinics), genitourinary (four clinics), breast (six clinics), and gynaecological (two clinics). Eight oncologists had clinics in two tumour sites, and were therefore randomised to two clusters; five oncologists had their two clinics randomised within the same trial group and three to different groups. Although complete masking of interventions was not possible, patients provided written informed consent to participate in their own study group, without being informed of the existence of another group. This form of masking is common in cluster randomised trials,18 and avoids potential bias from patients in the control group requesting the intervention or otherwise altering their behaviour. Oncologists and investigators were aware of assignment. depending on the status of the patient, included: arrangement of home nursing care services; transfer of care to a home palliative care physician (when the patient’s ECOG performance status was 3 or worse, or when requested); and admission to the Princess Margaret Cancer Centre palliative care unit for urgent symptom control or terminal care. This model was assessed in our pilot study,20 which showed improvement in symptom control and satisfaction of patients. The study duration was 4 months; on completion of the study, patients in the intervention group were offered continued follow-up in the oncology palliative care clinic. The control group received no formal intervention, but palliative care referral was not denied, if requested. Participants in the control group referred to the palliative care service received the same care as patients in the intervention group, but did not have the same standardised monthly follow-up. On completion of the 4-month measures, which represented the end of the trial, patients in the control group continued with standard care and were referred (or not) to the palliative care team when this referral would normally occur. Study measures consisted of several scales. The Functional Assessment of Chronic Illness Therapy— 24 clusters (clinics) randomised 12 clinics allocated to control 1367 patients screened for eligibility 1134 excluded 944 ineligible to participate 99 declined participation 57 not interested 23 time required 9 too ill 10 other reason 91 did not complete baseline assessment 12 clinics allocated to intervention 1926 patients screened for eligibility 1698 excluded 1357 ineligible to participate 251 declined participation 87 not interested 75 time required 54 no symptoms 14 too ill 7 palliative wording 14 other reason 90 did not complete baseline assessment Procedures Table 1 shows how the intervention differs from standard care. The Princess Margaret Cancer Centre palliative care service consists of an outpatient oncology palliative care clinic, a 12-bed palliative care unit, and an inpatient consultant team.19 The core intervention was consultation and follow-up in the oncology palliative care clinic by a palliative care physician and nurse, consisting of: (1) comprehensive, multidisciplinary assessment of symptoms, psychological distress, social support, and home services, within 1 month of recruitment (60–90 min duration); (2) routine telephone contact from a palliative care nurse 1 week after the first consultation, and thereafter as needed; (3) monthly outpatient palliative care follow-up (20–50 min); and (4) a 24-h on-call service for telephone management of urgent issues. Ancillary interventions, 233 assigned to control 17 died 24 withdrew 192 completed at least one follow-up assessment 9 died 28 withdrew 155 completed study 228 assigned to intervention 15 died 12 withdrew 201 completed at least one follow-up assessment 29 died 41 withdrew 131 completed study Figure 1: Trial profile www.thelancet.com Published online February 19, 2014 http://dx.doi.org/10.1016/S0140-6736(13)62416-2 3 Articles Intervention (n=228) Age (years) Control (n=233) 61·2 (12·0) 60·2 (11·3) Women 136 (59·6%) 125 (53·6%) Married or common law partner 156 (68·4%) 167 (71·7%) 43 (18·9%) 42 (18·0%) Living alone Employment status Retired 104 (45·6%) 101 (43·3%) Employed 45 (19·7%) 59 (25·3%) Unemployed 29 (12·7%) 24 (10·3%) Disability 50 (21·9%) 49 (21·0%) Below high school 18 (8·0%) 24 (10·3%) High school 56 (24·8%) 57 (24·6%) 152 (67·3%) 151 (65·1%) Lung 55 (24·1%) 46 (19·7%) Gastrointestinal 74 (32·5%) 65 (27·9%) Genitourinary 27 (11·8%) 51 (21·9%) Breast 41 (18·0%) 31 (13·3%) Gynaecological 31 (13·6%) 40 (17·2%) Education* College, university, or other Tumour site Active chemotherapy No 29 (12·7%) 25 (10·7%) Awaiting chemotherapy 25 (11·0%) 26 (11·2%) 174 (76·3%) 182 (78·1%) Radiation treatment Yes 16 (7·0%) 13 (5·6%) CCI total score >0† 75 (32·9%) 71 (30·5%) ECOG performance status‡ 0 61 (26·8%) 76 (32·6%) 1 149 (65·4%) 143 (61·4%) 2 18 (7·9%) 14 (6·0%) FACIT-Sp§ (n=443) 101 (20·3) 105 (18·8) QUAL-E¶ (n=436) 73 (11·1) 74 (11·5) ESAS|| (n=461) 28 (15·5) 23 (15·7) FAMCARE-P16** (n=449) 64 (9·7) 68 (9·7) Score CARES-MIS†† (n=448) 4 4·7 (5·6) Spiritual Well-Being (FACIT-Sp) scale measures quality of life, including physical, social and family, emotional, functional, and spiritual domains (score range 0–156, higher scores are better).21,22 The Quality of Life at the End of Life (QUAL-E) scale measures quality of life in domains of life completion, effect of symptoms, relationship with health provider, and preparation for end of life (range 21–105, higher scores are better).23 The Edmonton Symptom Assessment System (ESAS) consists of nine numerical scales with anchors of 0 (best) and 10 (worst) for pain, fatigue, drowsiness, nausea, anxiety, depression, appetite, dyspnoea, and wellbeing; individual symptom scores are summed for the ESAS Distress Score, ranging from 0–90.24 The FAMCARE-P16 scale measures satisfaction with information-giving, availability of care, psychological care, and physical care in patients with advanced cancer (range 16–80, higher scores are better).25 The Cancer Rehabilitation Evaluation System Medical Interaction Subscale (CARES-MIS) assesses specific problems of patients in their interactions with nurses and doctors, including those related to information seeking, communication, and control of the medical team (range 0–44, higher scores are worse).26 Follow-up measures were distributed in person or by mail. Patients who did not respond within 2 weeks received a reminder telephone call and an offer for research staff to help with completion of forms. Patients who could not be reached or declined help were deemed lost to follow-up. Medical records for each patient were reviewed at recruitment and at each follow-up visit. All measures were completed by the patient monthly for 4 months after enrolment. The 4-month study interval was chosen on the basis of two considerations: that there should be sufficient time for improvement in quality of life (or deterioration in patients in the control group, which tends to occur in all patients with advancing 3·9 (5·4) Intervention group (n=228) Control group (n=233) Data are mean (SD) or n (%). CCI=Charlson Comorbidity Index; ECOG=Eastern Cooperative Oncology Group. FACIT-Sp=Functional Assessment of Chronic Illness Therapy—Spiritual Well-Being scale. QUAL-E=Quality of Life at the End of Life scale. ESAS=Edmonton Symptom Assessment System. FAMCARE-P16=FAMCARE patient satisfaction with care measure. CARES-MIS=Cancer Rehabilitation Evaluation System Medical Interaction Subscale. *Data missing for two patients in intervention group and one patient in control group. †The CCI is a measure of comorbidity for patients with cancer; it generates a weighted score on the basis of the presence of various medical disorders, each disorder is assigned a score of 1, 2, 3, or 6 based on the CCI scoring index, scores are summed to provide a total score for each patient.28 ‡An ECOG score of 0 indicates fully active at predisease performance; 1 ambulatory but restricted in physically strenuous activity; 2 not fully ambulatory but lying or sitting <50% of the day.16 §The FACIT-Sp scale ranges from 0 to 156, with higher numbers representing better quality of life. ¶The QUAL-E scale ranges from 21 to 105, with higher numbers representing better quality of life. ||The ESAS scale ranges from 0 to 90, with higher numbers representing worse symptom control. **The FAMCARE-P16 scale ranges from 16 to 80, with higher numbers representing better patient satisfaction with care. ††The CARES-MIS subscale ranges from 0 to 44, with higher numbers representing greater problems with medical interactions. Data are n (%). *Numbers for admissions, consultations, and referrals are not exclusive (ie, one patient might have a consultation and a palliative care unit admission). Table 2: Baseline characteristics of trial participants Table 3: Palliative care intensity for intervention and control groups Palliative care clinic visits None 0 213 (91·4%) 1 23 (10·1%) 9 (3·9%) 2 30 (13·2%) 5 (2·1%) 3 28 (12·3%) 3 (1·3%) 4 68 (29·8%) 0 ≥5 79 (34·6%) 3 (1·3%) 17 (7·5%) 0 Inpatient palliative care consultations* 18 (7·9%) 2 (0·9%) Palliative home nursing referrals* 39 (17·1%) 7 (3·0%) Home palliative care physician referrals* 18 (7·9%) 7 (3·0%) Palliative care unit admissions* www.thelancet.com Published online February 19, 2014 http://dx.doi.org/10.1016/S0140-6736(13)62416-2 Articles illness), and that there should not be excessive dropout and therefore reduced power. On further consideration and before trial initiation, 3 months was designated as the primary endpoint to further ensure sufficient power for analyses; 4 months (trial end) was designated a secondary endpoint, because it would show whether or not an effect was sustained. Change score for FACIT-Sp from baseline to 3 months was the primary outcome. Change score for FACIT-Sp at 4 months, and change scores for all other scales at 3 months and 4 months, were secondary outcomes. As an exploratory substudy, we also collected data from caregivers; these data will be reported separately, as will data regarding economic cost and qualitative findings. Statistical analysis The intracluster correlation coefficient accounts for the greater similarity of responses of patients within clusters compared with between clusters.14 Our initial sample size Intervention Control n n Mean observed change from baseline (SD) estimation showed that 380 patients (190 per group), would provide 80% power at the two-sided 5% level of significance to detect a between-group difference in FACIT-Sp of 0·45 SD (medium effect size) by the primary endpoint of 3 months. Sample size was recalculated in 2008, on the basis of the observed SD (from aggregated baseline data of 245 patients), intracluster correlation coefficient, attrition, and adherence. The revised sample size was 450 patients (225 per group). Patients were analysed according to their original randomisation. To account for clustering,14 we used mixed effects models, implemented with PROC MIXED or GLIMMIX using SAS software (version 9.3). We controlled for baseline variables of tumour site, age, chemotherapy treatment, and ECOG score, on the basis of their association with quality of life in a previous (unpublished) analysis of our baseline data. To limit multiple testing, available cases analyses were focused on changes observed at the primary (3-month) and secondary Available cases analysis* Mean observed change from baseline (SD) Adjusted difference between change scores (95% CI) p value Effect size† ICC FACIT-Sp 1 month 154 1·86 (11·99) 168 –1·34 (10·12) ·· ·· ·· 2 months 138 0·58 (13·09) 151 –2·71 (12·92) ·· ·· ·· ·· 3 months 140 1·60 (14·46) 141 –2·00 (13·56) 3·56 (–0·27 to 7·40) 0·07 0·26 0·035 4 months 122 2·46 (15·47) 149 –3·95 (14·21) 6·44 (2·13 to 10·76) 0·006 0·44 0·024 1 month 154 1·09 (6·79) 162 –1·19 (7·22) ·· ·· ·· 2 months 137 1·38 (7·49) 151 –0·61 (8·13) ·· ·· ·· 3 months 139 2·33 (8·27) 139 0·06 (8·29) 2·25 (0·01 to 4·49) 0·05 0·28 0·036 4 months 121 3·04 (8·33) 148 –0·51 (7·62) 3·51 (1·33 to 5·68) 0·003 0·45 0·015 1 month 180 –0·72 (13·01) 172 1·13 (10·79) ·· ·· ·· 2 months 158 0·89 (14·83) 160 1·45 (14·08) ·· ·· ·· 3 months 151 0·14 (16·93) 149 2·12 (13·88) –1·70 (–5·26 to 1·87) 0·33 –0·13 0·067 4 months 131 –1·34 (15·98) 155 3·23 (13·93) –4·41 (–8·76 to –0·06) 0·05 –0·31 0·034 1 month 160 1·77 (8·14) 169 –2·64 (7·96) ·· ·· ·· 2 months 140 1·95 (9·12) 157 –2·26 (7·36) ·· ·· ·· ·· 3 months 142 2·33 (9·10) 145 –1·75 (8·21) 3·79 (1·74 to 5·85) 0·0003 0·47 <–0·0001 4 months 121 3·70 (8·58) 153 –2·42 (8·33) 6·00 (3·94 to 8·05) <0·0001 0·73 –0·018 ·· ·· QUAL-E ·· ·· ESAS ·· ·· FAMCARE-P16 ·· CARES-MIS 1 month 157 –0·45 (4·33) 170 0·88 (3·32) ·· ·· ·· 2 months 144 –0·28 (4·57) 156 0·86 (3·73) ·· ·· ·· 3 months 139 –0·16 (5·50) 147 0·85 (4·06) –0·66 (–2·25 to 0·94) 0·40 –0·21 0·018 4 months 123 –0·35 (4·38) 154 0·61 (3·60) –0·84 (–1·91 to 0·22) 0·11 –0·24 0·023 ·· FACIT-Sp=Functional Assessment of Chronic Illness Therapy—Spiritual Well-Being scale. QUAL-E=Quality of Life at the End of Life scale. ESAS=Edmonton Symptom Assessment System. FAMCARE-P16=FAMCARE patient satisfaction with care measure. CARES-MIS=Cancer Rehabilitation Evaluation System Medical Interaction Subscale. ICC=intracluster correlation coefficient. *Differences in change scores between groups and associated tests of effect were estimated by regression, adjusting for clustering and baseline covariates; negative ICCs were assumed to arise by chance and therefore treated as zero; regression analyses were done only at the 3-month and 4-month follow-up intervals to limit multiple testing. †Effect sizes are Cohen’s d: an effect size of 0·20 is small, 0·50 is moderate, and 0·80 is large.29 Table 4: Change scores relative to baseline and the regression-estimated differences in change scores between groups www.thelancet.com Published online February 19, 2014 http://dx.doi.org/10.1016/S0140-6736(13)62416-2 5 Articles (4-month) endpoints. Sensitivity analyses were done using three different methods for handling missing data: last value carried forward, complete case analysis, and multiple imputation. For multiple imputation we used the Markov Chain Monte Carlo method to generate 30 imputations, in line with evidence that this number would minimise power loss with 35% missing data at the 3-month primary endpoint.27 This trial is registered with ClinicalTrials.gov, number NCT01248624. Mean change 8 6 4 2 0 –2 –4 –6 –8 –10 Mean change FACIT-Sp 6 5 4 3 2 1 0 –1 –2 –3 QUAL-E Role of the funding source The study sponsors provided operating funds specifically for the trial after a competitive scientific peer-review process (Canadian Cancer Society), or indirectly via programme funding (Ontario Ministry of Health and Long-Term care), but had no role in study design, data collection, data analysis, data interpretation, or writing of the report. The Data Safety Monitoring Board at Princess Margaret Cancer Centre oversaw trial integrity and safety and had full access to the study data, as did study investigators (CZ, NS, AD, CL). CZ had final responsibility for the decision to submit for publication. ESAS 8 Mean change 6 4 Results 2 0 –2 –4 –6 FAMCARE-P16 6 Mean change 4 2 0 –2 –4 –6 CARES-MIS 2·5 Mean change 2·0 1·5 1·0 0·5 0 –0·5 –1·0 –1·5 Control 3 months Intervention Control Study group 4 months Intervention Figure 2: Adjusted mean changes in quality of life, symptom distress, satisfaction with care, and problems with medical interactions at 3 months and 4 months in the control and intervention groups Change scores were estimated by regression, adjusting for clustering and baseline covariates. Bars are 95% CIs. For FACIT-Sp, QUAL-E, and FAMCARE-P16, positive changes are improvements; for ESAS and CARES-MIS, negative changes are improvements. For FACIT-Sp=Functional Assessment of Chronic Illness Therapy-Spiritual Well-Being scale. QUAL-E=Quality of Life at the End of Life scale. ESAS=Edmonton Symptom Assessment System. FAMCAREP16=FAMCARE patient satisfaction with care measure. CARES-MIS=Cancer Rehabilitation Evaluation System Medical Interaction Subscale. 6 992 patients were eligible and 461 completed baseline measures: 223 patients were in the control group (mean number of patients per cluster 19·4 [SD 12·1]) and 228 were in the intervention group (mean number of patients per cluster 19·0 [12·9]). Figure 1 shows reasons for declining participation. Although presence of symptoms was not an entry criterion, 54 patients eligible for the intervention group declined to participate on the basis of lack of symptoms. 301 patients completed measures at 3 months (152 intervention; 149 control) and 286 patients completed the 4-month trial (131 intervention; 155 control). There were no substantive differences between groups at baseline for demographic or medical data, except for a larger number of patients in the control group with genitourinary cancers than in the intervention group (table 2). Outcome measure scores tended to be worse in the intervention group at baseline (table 2). However, during the trial, there was no difference between groups in the number of patients receiving chemotherapy (86% [195 of 228] in the intervention group vs 89% [208 of 233] in the control group, p=0·36) or radiation (21% [47 of 228] vs 15% [34 of 233], p=0·14). All participants in the intervention group had at least one visit to the oncology palliative care clinic (median four; range one to nine). The intensity of core and ancillary interventions is shown in table 3, as is contact of the control group with the intervention. The primary outcome of change scores for FACIT-Sp at 3 months did not differ significantly between groups (mean change score in intervention group +1·60 [SD 14·46] vs control group –2·00 [13·56], p=0·07). For www.thelancet.com Published online February 19, 2014 http://dx.doi.org/10.1016/S0140-6736(13)62416-2 Articles Last value forward Complete cases Multiple imputation Adjusted difference between change scores (95% CI) p value 0·02 3·20 (–0·07 to 6·47) 0·06 0·005 4·07 (0·64 to 7·51) 0·02 3·76 (0·81 to 6·71) 0·02 2·23 (0·26 to 4·19) 0·03 4·65 (1·85 to 7·44) 0·003 2·76 (0·84 to 4·67) 0·005 Adjusted difference between change scores (95% CI) p value Adjusted difference between change scores (95% CI) 3 months 3·89 (–0·02 to 8·01) 0·06 7·19 (1·57 to 12·8) 4 months 4·34 (0·70 to 7·98) 0·02 7·70 (2·64 to 12·8) 3 months 2·35 (0·17 to 4·54) 0·04 4 months 2·75 (0·56 to 4·95) 0·02 p value FACIT-Sp QUAL-E ESAS 3 months –0·95 (–4·54 to 2·64) 0·59 –3·92 (–7·15 to –0·69) 0·02 –2·39 (–6·18 to 1·41) 0·22 4 months –2·07 (–6·75 to 2·62) 0·37 –4·05 (–8·40 to 0·30) 0·07 –3·85 (–8·13 to 0·42) 0·08 3 months 4·95 (3·09 to 6·81) <0·0001 4·70 (2·38 to 7·03) <0·0001 4·70 (2·87 to 6·54) <0·0001 4 months 5·59 (3·65 to 7·52) <0·0001 5·63 (3·15 to 8·12) <0·0001 6·41 (4·44 to 8·37) <0·0001 3 months –1·34 (–2·60 to –0·08) 0·04 –0·88 (–2·71 to 0·95) 0·32 –0·80 (–2·01 to 0·40) 0·19 4 months –1·38 (–2·19 to –0·58) 0·002 –1·25 (–2·37 to –0·13) 0·03 –1·01 (–1·80 to –0·22) 0·01 FAMCARE-P16 CARES-MIS Differences in change scores between groups and associated tests of effect were estimated by regression, adjusting for clustering and baseline covariates. Negative intracluster correlation coefficient were assumed to arise by chance and therefore treated as zero. FACIT-Sp=Functional Assessment of Chronic Illness Therapy—Spiritual WellBeing scale. QUAL-E=Quality of Life at the End of Life scale. ESAS=Edmonton Symptom Assessment System. FAMCARE-P16=FAMCARE patient satisfaction with care measure. CARES-MIS=Cancer Rehabilitation Evaluation System Medical Interaction Subscale. Table 5: Sensitivity analyses using three methods for handling missing data secondary quality of life and symptom distress outcomes at 3 months, the difference between groups in change scores for QUAL-E was significant (+2·33 [8·27] vs +0·06 [8·29], p=0·05), but the difference between groups in change score for ESAS was not significant (+0·14 [16·93] vs +2·12 [13·88], p=0·33). At the secondary 4-month endpoint, the differences in change scores were significant for FACIT-Sp (+2·46 [15·47] vs –3·95 [14·21], p=0·006), QUAL-E (+3·04 [8·33] vs –0·51 [7·62], p=0·003), and ESAS (–1·34 [15·98] vs +3·23 [13·93], p=0·05; table 4, figure 2). There were significant differences in the secondary outcome of change scores for satisfaction with their care at the 3-month endpoint (mean change score for FAMCARE-P16 +2·33 [SD 9·10] in the intervention group vs –1·75 [8·21] in the control group, p=0·0003) and at the 4-month endpoint (+3·70 [8·58] vs –2·42 [8·33], p<0·0001; table 4, figure 2). There was no significant difference in change scores for problems with medical interactions at the 3-month endpoint (mean change score for CARES-MIS, –0·16 [SD 5·50] in the intervention group vs 0·85 [4·06] in the control group, p=0·40), or at the 4-month endpoint (–0·35 [4·38] vs 0·61 [3·60], p=0·11). Table 5 shows results of the sensitivity analyses. At the 3-month endpoint, the non-significant difference in FACIT-Sp between control and intervention groups was consistent with last value carried forward and multiple imputation (p=0·06 for both); the complete cases analysis showed a significant difference (p=0·02). Differences for the QUAL-E (p≤0·04 for all approaches) and FAMCARE-P16 scales (p<0·0001 for all approaches) were consistent with the main analyses. At the 4-month endpoint, the significant differences for FACIT-Sp, QUAL-E, and FAMCARE-P16 were robust across analyses, whereas the significant difference for the ESAS was not; CARESMIS change scores showed significant differences between groups across sensitivity analyses at 4 months, although the main available cases analysis was not significant. Discussion In this trial of 461 patients with advanced cancer, early referral to a palliative care team did not significantly improve quality of life (measured by the FACIT-Sp scale) at 3 months compared with usual cancer care. However, quality of life according to the QUAL-E scale was significantly improved in the intervention group compared with the control group at 3 months, as was satisfaction with their care (FAMCARE-P16). Changes in symptom severity (ESAS) and problems with medical interactions (CARES-MIS) were not significantly different between groups at 3 months. By 4 months, differences between groups were significant for all scales (including FACITSp) except CARES-MIS, favouring the early palliative care intervention. WHO has recommended that all countries implement comprehensive palliative care programmes to improve quality of life for patients with cancer or other lifethreatening illness,30 and a growing majority of countries worldwide have such programmes.3 However, palliative www.thelancet.com Published online February 19, 2014 http://dx.doi.org/10.1016/S0140-6736(13)62416-2 7 Articles care services are fully integrated within the health-care system of only a small proportion of these countries,3 and referrals are usually late in the disease process5,6 despite guidelines recommending early involvement.31 This cluster randomised trial provides promising findings that support the early involvement of specialised palliative care in the treatment of patients with a wide range of advanced cancers (panel). Quality of life was chosen as the primary outcome for this trial because it is a central focus of palliative care.2,32 We used as our primary outcome a cancer-specific measure encompassing physical, functional, social, and psychological domains, and the spiritual domain, which is of particular relevance in patients with advanced cancer.33 Additionally, we used a secondary outcome measure that was developed to assess the effectiveness of interventions targeting improved care at the end of life.23 Quality of life in patients with cancer tends to decrease gradually in the last year of life, with a steep decline in the last 2–3 months.34,35 Panel: Research in context Systematic review We did a systematic review of randomised controlled trials to assess the effectiveness of specialised palliative care,9 using the keyword groupings: “palliative”, “terminal”, or “hospice” and “quality of life”, “quality of care”, “satisfaction”, “wellbeing”, “economic” or “cost”. We searched Medline, Ovid Healthstar, CINAHL, Embase, and the Cochrane Register of Controlled Trials, from inception until Jan 31, 2008. Of 22 studies identified, not all assessed a clearly defined palliative care team, with other interventions including coordinating services, nursing interventions, or counselling services. Only four of 13 studies assessing quality of life had significant findings. However, most had insufficient statistical power, and took place late in the disease process, resulting in difficulties with recruitment, loss to follow-up, and co-intervention in patients in the control groups. Only five studies were done in outpatient clinics, with most done in patients’ homes; none specifically assessed an early palliative care intervention in patients with cancer. We updated our search on Aug 30, 2013, using the same keyword groupings. Since our 2008 review,9 two other randomised controlled trials were reported assessing early palliative care interventions in patients with advanced cancer. Bakitas and colleagues10 randomised 322 participants with advanced cancer and a prognosis of about 1 year to routine care or to a palliative care problem-solving intervention provided by advanced practice nurses through telephone contact. Temel and colleagues11 randomised 151 patients with advanced non-small-cell lung cancer to an early palliative care team intervention or to oncology care alone. In both studies, early palliative care improved quality of life and mood. Interpretation The findings of our cluster-randomised trial of 461 outpatients with a wide range of advanced cancer diagnoses suggest that early palliative care might be beneficial for patients with a wide range of advanced solid tumour malignancies. Such a collaborative approach between oncology and palliative care differs from the traditional view of palliative care services being used only at the very end of life, and is increasingly endorsed in oncology guidelines.31 Our findings are consistent with those of previous studies that have shown a benefit of early palliative care for improving quality of life,10,11 and also show that this approach improves satisfaction of patients with their care. Taken together, these studies show the benefits of integrated palliative care services, and provide support for early involvement of specialised palliative care in tandem with standard oncology care. 8 Despite the worse status of the intervention group at baseline, which might have placed it at a steeper point in this decline, the present study showed a non-significant improvement in quality of life at the primary 3-month endpoint, and a significant benefit at 4 months. This finding extends evidence from a previous trial of early involvement by a palliative care team, which showed improvement in quality of life for patients with non-smallcell lung cancer.11 The effect sizes for quality of life (FACITSp and QUAL-E) in our study were 0·26–0·28 at 3 months and 0·44–0·45 at 4 months (table 4), compared with those of 0·41–0·52 identified after early palliative care for patients with metastatic non-small-cell lung cancer.11 Satisfaction of patients with their care improved significantly at both endpoints in the early palliative care intervention group, whereas it deteriorated in the control group. The measure used for the present trial was validated specifically for outpatients with advanced cancer25 and included recommended domains of accessibility and coordination of care, symptom management, communication and education, support and personalisation of care, and support of decisionmaking.36 This patient-centred outcome is indicative of quality of care, which is particularly important for patients with advanced illness, who spend a large proportion of their time interacting with medical teams.36,37 In addition to measuring overall satisfaction of patients using the FAMCARE-P16 measure, we explored specific problems with clinician–patient interactions using CARES-MIS. Although the two measures contain similar items, in CARES-MIS they are negatively worded, with emphasis on the patient rather than on the clinician, (eg, “Doctor’s attention to your description of symptoms” in FAMCARE-P16 vs “I have difficulty telling my doctor about new symptoms” in CARES-MIS). This negative phrasing and emphasis on the patient might have contributed to the highly positively skewed baseline scores for the CARES-MIS, which showed little variability or room to improve. The small number of patients acknowledging such difficulties also meant that power was affected by any data loss. Much the same as in other studies,9,10 the present trial showed only a marginal difference in symptom intensity, and only at the secondary 4-month endpoint. This finding contrasts with our pilot study, in which symptom scores improved substantially after the same intervention.20 A possible reason is that for the present trial, the mean baseline ESAS distress score was only 28 (of a possible 90); thus, similar to CARES-MIS, there was little room for improvement. Inclusion of a minimum level of symptom distress as an entry criterion should be considered for future studies, as was done in a successful trial of a nurse-led intervention for cancer-related fatigue.38 The difference in effect sizes for quality of life and symptom severity (table 4) affirms that quality of life is a broader construct, which can improve despite a relative lack of change in symptoms. www.thelancet.com Published online February 19, 2014 http://dx.doi.org/10.1016/S0140-6736(13)62416-2 Articles In complex interventions it is important to define the core intervention, required for all patients randomised to the intervention group, and additional interventions, which are an extension of the core, but are conditional upon the situation of the individual patient.39,40 In our study, the core intervention consisted of outpatient consultation and follow-up by a palliative care physician and nurse. Although there are differences in how palliative care is practised worldwide, physicians and nurses are usually at the core of hospital palliative care services, which are mainly directed at inpatients.5,6 Most palliative care clinics, including ours, began as a half-day clinic per week, which need not be a large drain on existing inpatient palliative care resources.4 We did not gather data for the level of proficiency in palliative care of participating oncologists. However, in a survey of Canadian oncologists, 50% (285 of 568) had completed a rotation in palliative care during their training; and 43% (244 of 568) reported they felt comfortable providing palliative care.6 Those who had completed a palliative care rotation during their training were more likely to refer early to palliative care, suggesting that this experience might have increased their awareness of possible benefits of this practice. Limitations of this trial include that it was done at one centre and interventions were not masked. Availability of a well-functioning palliative care team provided a clinically relevant intervention, but exposure of controls to the intervention was unavoidable because of oncologists’ familiarity with this team, and because three oncologists were randomised to both trial groups; this exposure might have reduced power. Although we cannot exclude that the benefits of the intervention were attributable to increased attention in general, rather than specifically from a palliative care team, attention to concerns of patients is itself an important aspect of palliative care.32 There was also selection bias, which is common in cluster-randomised studies because of randomisation of clusters before consent of individuals.18 A larger number of patients declined participation in the intervention group, including because of lack of symptoms. Aware of this potential limitation, we opted for cluster randomisation to maximise recruitment, and were able to attain our planned sample size. Early involvement of specialised palliative care is becoming a quality standard for patients with cancer31 but it is currently underutilised.5–8 This study suggests that early palliative care might improve quality of life and increase satisfaction with their care for patients with a large range of advanced solid tumour malignancies. These results provide support for earlier involvement of palliative care teams for patients with advanced cancer. Further studies are needed to establish which patients are most likely to benefit; a cost analysis is underway, and will establish the economic implications of implementing this model. Contributors CZ contributed to conception and design, analysis and interpretation of data, drafting of the report, critical revision of the report, and obtaining funding. NS contributed to data collection, drafting of the report, analysis and interpretation of data, critical revision of the report, and administrative support. MK and IT contributed to study conception and design, data collection, analysis and interpretation of data and critical revision of the report. BH contributed to analysis and interpretation of data, drafting of the report, and critical revision of the report. NL, AO, and MM contributed to study conception and design, data collection, and critical revision of the report. AR contributed to study conception and design, critical revision of the report, and administrative support. GR contributed to study conception and design, analysis and interpretation of data and critical revision of the report. AD contributed to study conception and design, analysis and interpretation of data, statistical analysis and critical revision of the report. CL contributed to analysis and interpretation of data, statistical analysis, drafting of the report, and critical revision of the report. Declaration of interests We declare that we have no competing interests. Acknowledgments Results of this study were presented, in part, at the American Society of Clinical Oncology (ASCO) Annual Meeting; Chicago, IL, USA; June 4, 2012; and at Best of ASCO, 2012. This study was funded by grants 017257 and 020509 from the Canadian Cancer Society (CZ) and by the Ontario Ministry of Health and Long Term Care. CZ is also supported by the Rose Family Chair in Supportive Care, Faculty of Medicine, University of Toronto. The views expressed in the study do not necessarily represent those of the sponsors. We thank the patients who participated in this study, and their caregivers. We thank the medical oncologists who referred patients to this study, and the clinical and administrative staff of the palliative care team at Princess Margaret Cancer Centre. We thank Debika Burman, Nanor Kevork, and Ashley Pope (Department of Psychosocial Oncology and Palliative Care, Princess Margaret Cancer Centre) for their assistance with preparation of study materials, recruitment of patients, and data entry and preparation. References 1 Harrison JD, Young JM, Price MA, Butow PN, Solomon MJ. What are the unmet supportive care needs of people with cancer? A systematic review. Support Care Cancer 2009; 17: 1117–28. 2 WHO. WHO definition of palliative care. www.who.int/cancer/ palliative/definition/en (accessed Nov 13, 2013). 3 Lynch T, Clark D, Connor SR. Mapping levels of palliative care development: a global update 2011. London: Worldwide Palliative Care Alliance, 2011. 4 Meier DE, Beresford L. Outpatient clinics are a new frontier for palliative care. J Palliat Med 2008; 11: 823–28. 5 Hui D, Elsayem A, De la Cruz M, et al. Availability and integration of palliative care at US cancer centers. JAMA 2010; 303: 1054–61. 6 Wentlandt K, Krzyzanowska MK, Swami N, Rodin GM, Le LW, Zimmermann C. Referral practices of oncologists to specialized palliative care. J Clin Oncol 2012; 30: 4380–86. 7 Lamont EB, Christakis NA. Physician factors in the timing of cancer patient referral to hospice palliative care. Cancer 2002; 94: 2733–37. 8 Osta BE, Palmer JL, Paraskevopoulos T, et al. Interval between first palliative care consult and death in patients diagnosed with advanced cancer at a comprehensive cancer center. J Palliat Med 2008; 11: 51–57. 9 Zimmermann C, Riechelmann R, Krzyzanowska M, Rodin G, Tannock I. Effectiveness of specialized palliative care: a systematic review. JAMA 2008; 299: 1698–709. 10 Bakitas M, Lyons KD, Hegel MT, et al. Effects of a palliative care intervention on clinical outcomes in patients with advanced cancer: the Project ENABLE II randomized controlled trial. JAMA 2009; 302: 741–49. 11 Temel JS, Greer JA, Muzikansky A, et al. Early palliative care for patients with metastatic non-small-cell lung cancer. N Engl J Med 2010; 363: 733–42. 12 Fairhurst K, Dowrick C. Problems with recruitment in a randomized controlled trial of counselling in general practice: causes and implications. J Health Serv Res Policy 1996; 1: 77–80. www.thelancet.com Published online February 19, 2014 http://dx.doi.org/10.1016/S0140-6736(13)62416-2 9 Articles 13 14 15 16 17 18 19 20 21 22 23 24 25 26 10 Gilbody S, Whitty P. Improving the delivery and organisation of mental health services: beyond the conventional randomised controlled trial. Br J Psychiatry 2002; 180: 13–18. Donner A, Klar N. Design and analysis of cluster randomization trials in health research. London: Arnold, 2000. Glare P, Virik K, Jones M, et al. A systematic review of physicians’ survival predictions in terminally ill cancer patients. BMJ 2003; 327: 195. Oken MM, Creech RH, Tormey DC, et al. Toxicity and response criteria of the Eastern Cooperative Oncology Group. Am J Clin Oncol 1982; 5: 649–55. Katzman R, Brown T, Fuld P, Peck A, Schechter R, Schimmel H. Validation of a short Orientation-Memory-Concentration Test of cognitive impairment. Am J Psychiatry 1983; 140: 734–39. Campbell MK, Piaggio G, Elbourne DR, Altman DG. Consort 2010 statement: extension to cluster randomised trials. BMJ 2012; 345: e5661. Zimmermann C, Seccareccia D, Clarke A, Warr D, Rodin G. Bringing palliative care to a Canadian cancer center: the palliative care program at Princess Margaret Hospital. Support Care Cancer 2006; 14: 982–87. Follwell M, Burman D, Le LW, et al. Phase II study of an outpatient palliative care intervention in patients with metastatic cancer. J Clin Oncol 2009; 27: 206–13. Webster K, Cella D, Yost K. The Functional Assessment of Chronic Illness Therapy (FACIT) measurement system: properties, applications, and interpretation. Health Qual Life Outcomes 2003; 1: 79. Peterman AH, Fitchett G, Brady MJ, Hernandez L, Cella D. Measuring spiritual well-being in people with cancer: the functional assessment of chronic illness therapy—Spiritual Well-being Scale (FACIT-Sp). Ann Behav Med 2002; 24: 49–58. Steinhauser KE, Clipp EC, Bosworth HB, et al. Measuring quality of life at the end of life: validation of the QUAL-E. Palliat Support Care 2004; 2: 3–14. Bruera E, Kuehn N, Miller MJ, Selmser P, Macmillan K. The Edmonton Symptom Assessment System (ESAS): a simple method for the assessment of palliative care patients. J Palliat Care 1991; 7: 6–9. Lo C, Burman D, Rodin G, Zimmermann C. Measuring patient satisfaction in oncology palliative care: psychometric properties of the FAMCARE-patient scale. Qual Life Res 2009; 18: 747–52. Schag CA, Heinrich RL, Aadland RL, Ganz PA. Assessing problems of cancer patients: psychometric properties of the cancer inventory of problem situations. Health Psychol 1990; 9: 83–102. 27 28 29 30 31 32 33 34 35 36 37 38 39 40 Graham JW, Olchowski AE, Gilreath TD. How many imputations are really needed? Some practical clarifications of multiple imputation theory. Prev Sci 2007; 8: 206–13. Charlson ME, Pompei P, Ales KL, MacKenzie CR. A new method of classifying prognostic comorbidity in longitudinal studies: development and validation. J Chronic Dis 1987; 40: 373–83. Cohen J. Statistical power analysis for the behavioral sciences, 2nd edn. New York: Taylor & Francis Group, 1988. Sepulveda C, Marlin A, Yoshida T, Ullrich A. Palliative care: the World Health Organization’s global perspective. J Pain Symptom Manage 2002; 24: 91–96. Smith TJ, Temin S, Alesi ER, et al. American Society of Clinical Oncology provisional clinical opinion: the integration of palliative care into standard oncology care. J Clin Oncol 2012; 30: 880–87. Morrison RS, Meier DE. Clinical practice. Palliative care. N Engl J Med 2004; 350: 2582–90. Whitford HS, Olver IN, Peterson MJ. Spirituality as a core domain in the assessment of quality of life in oncology. Psychooncology 2008; 17: 1121–28. Giesinger JM, Wintner LM, Oberguggenberger AS, et al. Quality of life trajectory in patients with advanced cancer during the last year of life. J Palliat Med 2011; 14: 904–12. Hwang SS, Chang VT, Fairclough DL, Cogswell J, Kasimis B. Longitudinal quality of life in advanced cancer patients: pilot study results from a VA medical cancer center. J Pain Symptom Manage 2003; 25: 225–35. Dy SM, Shugarman LR, Lorenz KA, Mularski RA, Lynn J. A systematic review of satisfaction with care at the end of life. J Am Geriatr Soc 2008; 56: 124–29. Morrison RS, Siu AL, Leipzig RM, Cassel CK, Meier DE. The hard task of improving the quality of care at the end of life. Arch Intern Med 2000; 160: 743–47. de Raaf PJ, de Klerk C, Timman R, Busschbach JJ, Oldenmenger WH, van der Rijt CC. Systematic monitoring and treatment of physical symptoms to alleviate fatigue in patients with advanced cancer: a randomized controlled trial. J Clin Oncol 2013; 31: 716–23. Craig P, Dieppe P, Macintyre S, Michie S, Nazareth I, Petticrew M. Developing and evaluating complex interventions: the new Medical Research Council guidance. BMJ 2008; 337: a1655. Rinck GC, van den Bos GA, Kleijnen J, de Haes HJ, Schade E, Veenhof CH. Methodologic issues in effectiveness research on palliative cancer care: a systematic review. J Clin Oncol 1997; 15: 1697–707. www.thelancet.com Published online February 19, 2014 http://dx.doi.org/10.1016/S0140-6736(13)62416-2