Linking Clinical Cohorts and Basic/Clinical
Science in Developing Countries
Challenges and pitfalls
Eugène Kroon, MD
-SEARCH, Thai Red Cross AIDS Research Center
-USAMC-AFRIMS
Bangkok, Thailand
eugene.kroon@afrims.org
www.ias2013.org
Kuala Lumpur, Malaysia , 30 June - 3 July 2013
Cohorts: challenges and pitfalls
Prospective cohort studies are considered to yield the most
reliable results in observational epidemiology. They enable a
wide range of exposure-disease associations to be studied.
Many excellent examples in the field of HIV medicine:
treatment outcomes in different populations, ART safety,
longer term disease complications, etc.
but:
Cohort studies may be expensive to conduct, are sensitive
to attrition and often take a long follow-up time to generate
useful data.
www.ias2013.org
Kuala Lumpur, Malaysia , 30 June - 3 July 2013
Cohorts: challenges and pitfalls
In addition:
Randomized controlled trials (RTC) are generally
considered a superior methodology in the hierarchy
of evidence in at least therapy.
RTC limit the potential for any biases by randomly
assigning one patient pool to an intervention and
another patient pool to non-intervention or placebo.
This minimizes the chance that the incidence
of (unknown) confounding variables will differ
between the two groups.
www.ias2013.org
Kuala Lumpur, Malaysia , 30 June - 3 July 2013
Cohorts: challenges and pitfalls
Today touching upon:
1. Internal versus external validity
2. Attrition and missing data
3. Representativeness and generalizability
4. Cohort effect and the need to ‘refresh cohorts’
5. Reality check when setting up a prospective
cohort: data collection, timing and looking in the
right population
www.ias2013.org
Kuala Lumpur, Malaysia , 30 June - 3 July 2013
Internal versus external validity
Associations found in cohorts may hold up in
cohorts, but may not be replicable outside the cohort
setting due to interference of many other factors,
that in a clinical trial can be randomized.
Example:
the complex issue of nevirapine vs.efavirenz in
terms of virological failure
www.ias2013.org
Kuala Lumpur, Malaysia , 30 June - 3 July 2013
Internal versus external validity
In the early 2000s four articles were published by multiple, reputable
and sizeable cohorts, both in ART naïve and ART experienced
patients, all suggesting a greater or earlier risk of virological failure on
nevirapine (NVP) vs. efavirenz (EFV)
Keiser et al.HIV Clin Trials. 2002;3:296-303. Shorter time to treatment
failure on NVP (307 days vs. 589 days; p <.001).
Cozzi-Lepri et al.J Infect Dis. 2002;185:1062-1069. Adjusted relative
hazard of virological failure on NVP 2.08 (95% CI, 1.37–
3.15; P=0.0006).
Matthews et al .AIDS. 2002;16:53-61. Relative hazard for achieving
undetectable viral load on NVP 0.77 (95% CI, 0.61-0.96, P=0.02)
Phillips et al. AIDS. 2001;15:2385-2395. Relative hazard of virological
failure on EFV 0.57 (95% CI, 0.47-0.69;P < 0.0001).
www.ias2013.org
Kuala Lumpur, Malaysia , 30 June - 3 July 2013
Internal versus external validity
Two of those four publications mentioned the possibility of
bias explicitly and state that replication of study findings in
randomized trials and other cohort studies is required.
RTC:
2004, the Lancet (vol. 363, page 1253-1263):
Comparison of first-line antiretroviral therapy with regimens
including nevirapine, efavirenz, or both drugs, plus stavudine
and lamivudine: a randomised open-label trial, the 2NN Study.
F van Leth, P Phanuphak, K Ruxrungtham, et al.
www.ias2013.org
Kuala Lumpur, Malaysia , 30 June - 3 July 2013
Internal versus external validity
2NN study:
multicentre, open-label, randomised trial
1216 antiretroviral-therapy-naive patients
48 weeks duration
nevirapine 400 mg once daily
nevirapine 200 mg twice daily
efavirenz 600 mg once daily
nevirapine (400 mg) and efavirenz (800 mg) once daily
all arms plus stavudine and lamivudine
www.ias2013.org
Kuala Lumpur, Malaysia , 30 June - 3 July 2013
Internal versus external validity
2NN at 48 weeks:
Treatment failure difference NVP twice daily and EFV
5·9% (43·7% NVP vs 37·8% EFV; 95% CI -0·9 to 12·8)
Differences in plasma HIV-1 RNA below 50 copies per mL
at week 48 not significant between any of the arms
(p=0·193)
www.ias2013.org
Kuala Lumpur, Malaysia , 30 June - 3 July 2013
Internal versus external validity
The authors state in their discussion:
“The disparities between results from cohort studies and the 2NN Study
might be partly explained by differences in study design. Since there
is no randomisation in cohort studies, selection bias could have been
introduced if the choice between nevirapine and efavirenz were
influenced by patients and physicians. ……Although these biases can
be kept to a minimum by the use of multivariate models, the possibility of
residual confounding in cohort studies remains.”
However, Dr Andrew Carr noted in his commentary: “2NN found that
nevirapine twice daily was not equivalent to efavirenz at week 48.” as noninferiority had not been shown.
www.ias2013.org
Kuala Lumpur, Malaysia , 30 June - 3 July 2013
Internal versus external validity
The story continues:
The HIV-CAUSAL Collaboration. AIDS 2012;26:1691-1705.
The effect of efavirenz versus nevirapine-containing regimens on
immunologic, virologic and clinical outcomes in a prospective
observational study
Results:
A total of 15 ,336 individuals initiated an efavirenz regimen and 8129
individuals initiated a nevirapine regimen. Relative hazard for virological
failure on NVP 1.5 (95% CI, 1.3–1.8).
(no reference to 2NN study)
www.ias2013.org
Kuala Lumpur, Malaysia , 30 June - 3 July 2013
Attrition and missing data
Cohorts usually reflect real life field situations with, for example, no
transport covered for subjects or staff salary supplementation.
Irrespective of these factors, attrition is a reality.
This can be especially troubling in cohorts looking at clinical outcomes
and/or death.
Methods have been developed to try and account for this incompleteness,
including modeling approaches that project missing data. These
approaches can be validated, but they do depend on the robustness of
that model.
As attrition is a reality, such methods are necessary to put to use the data
we do gather. ‘Higer level’ expert input is needed and ongoing research
into the assumptions used in the modeling.
www.ias2013.org
Kuala Lumpur, Malaysia , 30 June - 3 July 2013
Attrition and missing data
Geng et al. Am J Epidemiol. 2012;175(10):1080–1087.
‘A Causal Framework for Understanding the Effect of Losses to
Follow-up on Epidemiologic Analyses in Clinic-based Cohorts:
The Case of HIV-infected Patients on Antiretroviral Therapy in Africa’
The authors evaluated adult patients starting ART at an HIV/AIDS clinic in
Uganda, where 29% of patients were lost to follow-up after 2 years:
‘In a clinic-based cohort in Africa, unweighted and inverse probability
censored weighting approaches which rely on the ‘‘missing at random’’
assumption (=assuming that the fact that data are missing is unrelated to
the actual values of the missing data)-yielded biased estimates. A
sampling-based approach (using supplemental data from a sample of lost
patients subsequently tracked in the community) can in general
strengthen epidemiologic analyses conducted in many clinic-based
cohorts.’
www.ias2013.org
Kuala Lumpur, Malaysia , 30 June - 3 July 2013
Representativeness and generalizability
Unless a cohort is drawn as a random sample,
descriptive findings and bi-variate and multivariate
associations cannot be generalized.
Sometimes representativeness can matter:
E.g. cohort studies into HIV incidence in
preparation for HIV vaccine studies (studies
RV121, RV126, and RV217)
And sometimes it may not matter…
www.ias2013.org
Kuala Lumpur, Malaysia , 30 June - 3 July 2013
TAHOD - IeDEA
• 2410 patients,
• 12 countries
• Median CD4
170 cells/mm3
before 2007
to
136 cells/mm3
after 2009
‘The data reflect trends in leading clinical HIV centers with the highest level of resources for HIV
testing, care, treatment, lab monitoring in their countries. We take this as in indication that we
are not doing well enough in the places where HIV is concentrated (urban centers like Bangkok,
Phnom Penh and Kuala Lumpur).’
Kiertiburanakul S, CROI 2013, Abstract 1089.
www.ias2013.org
Kuala Lumpur, Malaysia , 30 June - 3 July 2013
Cohort effect and the need to ‘refresh cohorts’
This refers to the dwindling incidence of disease
because of declining probability of acquisition.
Example:
A long existing urban MSM cohort may have low HIV
incidence (say 1%) with several hundreds of men
left in the cohort, while a VCT clinic in the same are
reports a much higher incidence (for example 7 per
100 PY) in the same population.
This means it is time to refresh the cohort.
www.ias2013.org
Kuala Lumpur, Malaysia , 30 June - 3 July 2013
Reality check when setting up a prospective cohort: data
collection, timing and looking in the right population
Collect only the data needed
Thailand Progress Adult Study (HIV disease
progression) – 3 sites following 6000+ pts yearly
to evaluate treatment outcome and morbidities.
First 3 years too many details were collected
(demographics, social situation, symptoms, AEs)
and most data were missing.
Data collection was streamlined and data
collection limited to ensure that endpoints are
being collected.
Tools: staff training, data manual and electronic
transfer of lab data.
www.ias2013.org
Kuala Lumpur, Malaysia , 30 June - 3 July 2013
Reality check when setting up a prospective cohort: data
collection, timing and looking in the right population
Timing:
HIV incidence studies for vaccine studies may be
affected by change in genotype (IDU, Bangkok 1998) or
change in incidence. The latter may be affected by
foreseeable circumstances, such as PrEP.
TNT 003 at TRC is a cohort of HIV positives and HIV
negatives aged > 30 yrs to evaluate long term
morbidities of HIV and ARV (bone, liver, kidney, brain).
After two years and 500+ patients, very few events are
seen in both groups. The protocol will be modified to
follow these pts every 5 years.
www.ias2013.org
Kuala Lumpur, Malaysia , 30 June - 3 July 2013
Reality check when setting up a prospective cohort: data
collection, timing and looking in the right population
Looking in the right population:
Study RV217; HIV incidence in MARP in Pattaya, Thailand.
Dropped FSWs from protocol target population after finding
the following incidence (M. Robb, AIDS Vaccine, August
2012, Boston):
TGW
MSM
FSW
6.3%
5.85 %
0.94%
N = 146
N = 181
N = 141
Study RV254/SEARCH010 has tested samples from nearly 70,000
subjects to identify 136 AHI. Majority are in MSM. If costs become a
barrier, one could consider looking in MSM only.
www.ias2013.org
Kuala Lumpur, Malaysia , 30 June - 3 July 2013
Acknowledgements
Frits van Griensven
HIV-NAT, Thai Red Cross AIDS Research Center
Steve Kerr
HIV-NAT, Thai Red Cross AIDS Research Center
Kirby Institute UNSW
Annette Sohn
amfAR
TREAT Asia
Jintanat Ananworanich
SEARCH, Thai Red Cross AIDS Research Center
HIV-NAT, Thai Red Cross AIDS Research Center
Mike Benenson
Retired USAMC-AFRIMS
www.ias2013.org
Kuala Lumpur, Malaysia , 30 June - 3 July 2013