Kyla Hayford
Delta Omega Proposal
Advisor: Orin Levine
Scholarship application. Details below.
First Name:
Kyla
Last Name:
Hayford
Middle Initial:
Department:
IH
Advisor:
Orin Levine
Degree:
PhD
Project Title:
Using Oral Fluid Samples to Validate Immunization Coverage of Infants in Bangladesh: A Non-Invasive
Approach to Identifying Measles Susceptible Children Missed by Conventional Surveillance
Category:
measurement
Itemized Budget Plan For Up To $1000 (1 page or less):
Oral fluid swabs for ½ of participants (Oracol oral specimen collection devices), 330 x $1.50 $495 Plastic
covers for vaccination cards (gift for participating in study), 660 x $0.35 $231 Translation and back
translation of survey and informed consent forms in Bangla $270 _____ Total: $996 *Note: Additional grants
will be used to share costs for assays, oral fluid collection devices, staff and transportation costs.
1
Kyla Hayford
Delta Omega Proposal
Advisor: Orin Levine
Using Oral Fluid Samples to Validate Immunization Coverage of Infants:
A Non-Invasive Approach to Identifying Measles Susceptible Children Missed
by Conventional Surveillance
Introduction
The provision of basic childhood vaccinations is a cornerstone of an effective health system. Despite
impressive gains in vaccination coverage since the 1980s, many low-income countries have failed to sustain the
80% coverage goal set out by the Universal Childhood Immunization initiative(1). Recent outbreaks of measles,
pertussis and polio in low-income countries indicate that conventional surveillance methods are not sufficiently
identifying susceptible children. Approximately 1.4 million children died of vaccine-preventable deaths in 2002,
which could have been largely prevented via improved surveillance and targeting(2).
Measuring immunization coverage in a low-income country is difficult and costly. In the absence of a
feasible gold standard of population immunity, several proxy indicators of immunization coverage are used but
correlation between indicators is often poor due to differences in the methods used to collect the data and calculate
coverage(3). The conventional approaches for estimating immunization coverage are based on administrative
records from health centers, vaccination card records or parental recall – all of which suffer from social desirability
bias, recall error and other measurement errors (4-7). Studies from Bolivia and Kenya found that parental report
data underestimated the number of susceptible individuals, especially in populations with relatively high coverage (8,
9). A second concern is that reported vaccination does not account for incorrect administration of vaccines or
breaches in the cold chain, leading to an overestimation of actual population immunity. Finally, because
immunization coverage is used widely as an indicator of health systems functioning, progress toward the MDGs and
improvements in coverage are often tied to donor funding, there is a significant risk of over-reporting by
governments. A recent study by Lim and others found that governments receiving performance bonuses for
improving immunization access were overestimating true coverage by 33-100% (3). Given the potential for bias and
error, new tools are needed that go beyond reported vaccination data to establish alternative estimates of coverage
and to explore reasons for discrepancies across sources (4, 5, 7, 10-12).
Oral fluid biomarkers of immunity can be used to validate measles immunization coverage estimates. A
common approach to validate immunization coverage estimates is to estimate the prevalence of disease-specific
antibody from blood samples(13). Its use has ben limited in low-income countries due to the costs, invasiveness
and feasibility of collecting blood, but oral fluid collection has emerged as a realistic, minimally invasive and lowcost alternative (14-20). Despite earlier concerns about the accuracy of oral fluid assays, the new disease-specific
IgG enzyme-linked immunosorbent assays (ELISAs) for measles have demonstrated high sensitivities and
specificities vis-à-vis blood-based assays (9, 18, 19, 21-24).1 Because oral fluid assays for measles are superior to
other oral fluid assays (e.g. vaccine-preventable disease alternatives, this study proposes to explore the validity of
immunization coverage estimates by comparing conventional methods for estimating measles coverage to noninvasive oral fluid sampling methods.
Goal and Specific Aims
The goal of the proposed study is to use non-invasive oral fluid biomarkers of measles immunity to assess the
validity of conventional measles immunization coverage estimates based on administrative records and parental
recall. The specific aims are to:
 Estimate prevalence of measles immunity using oral fluid assays among 12-23 month olds and compare it
to immunization coverage estimates from administrative data and parental recall in the same population.
 Determine the sensitivities and specificities of parental report and ‘card plus report’2 of measles
immunization compared to oral fluid-confirmed immunity.
 Develop a prediction model of measles immunity in the population based on individual-level and
community-level predictors for misreport of a child’s immunization status.
 Test if there is children receiving vaccinations from a measles campaign are more likely to confer
immunity than from routine immunization.
Two studies and manufacturer guidelines estimate the sensitivity of the oral fluid ELISA for detectiong measles-specific IgG
ranged from 93% to 98%. Specificity ranged from 87% to 98%
2 ‘Card + report’ is the standard method used by DHS and WHO coverage surveys in which maternal report data is used only
if mothers do not have vaccination cards with recorded vaccination events.
1
2
Kyla Hayford
Delta Omega Proposal
Advisor: Orin Levine
Methods
Sampling and Eligibility. A population-based representative survey with oral fluid collection will be conducted
with 660 randomly selected eligible parent-child pairs in one district of Dhaka, which has one of the worst
immunization coverage rates in the country. Eligible children include all 12-23 month olds living in the randomly
selected household with a parent/guardian present at time of interview and who consent to the study protocol and
oral fluid sampling. Based on the 30-cluster WHO Immunization Coverage Survey design, each cluster will need
22 children to estimate prevalence within ±5%, based on a two-tailed alpha=5% and power = 80%(25). To
account for intra-cluster correlation, sample size calculations include a design effect (deff) of 1.3, which is slightly
higher than deffs used by the 2004 Bangladesh DHS for measles vaccination questions. Reliability re-tests on a
10% subsample will be conducted.
Data Collection. A survey on vaccination history and an oral fluid sample will be collected from every enrolled
child. Prior to requesting vaccination cards, data collectors will ask parents about the child’s vaccination history
and corresponding dates of vaccinations for all routine infant vaccines (BCG, DTP3, OPV3, MCV) using standard
questions from DHS and WHO immunization surveys. After parental recall is completed, vaccination cards will be
requested and dates of all vaccines will be recorded. Health status, history of measles, and additional demographic
and socioeconomic characteristics of the household will also be collected. Oral fluid samples will be obtained with
Oracol oral-specimen collection devices, stored in a cold box until processing in a lab in Dhaka according to the
manufacturer’s protocol. Oral fluid specimens will be tested for measles-specific antibodies with an IgG capture
EIA, which has been validated for oral fluid. Government data will be provided by the Bangladesh EPI team.
Analysis Plan. For the primary analysis, a log binomial regression will be used to estimate ‘coverage’ or
prevalence of immunity and predictors of immunity. A score test for binomial proportions will be used to test if
the prevalence of immunity based on oral fluid samples is significantly different from coverage from government
estimates (one-sample test) or parental recall (two-sample test). Correlation, kappa statistics, sensitivities and
specificities will be calculated for pair-wise comparisons of oral fluid vs. card+report3, and oral fluid vs. parental
report. The second analysis will evaluate the accuracy of parental report compared to oral fluid-confirmed
immunity. Using multilevel logistic regression models, a prediction model will be developed to estimate measles
immunity using individual and cluster-level factors associated with oral-fluid confirmed immunity. By identifying
which factors are associated discordance between reported status and actual immune status, we can correct
coverage estimates from conventional surveillance methods to better reflect the true proportion of measlesimmune individuals. Error models, which have been used to adjust for error due to an imperfect gold standard,
will be explored to ensure oral-fluid based estimates of measles immunity are accurately modeled (9, 15, 24).
Significance
The proposed study uses an innovative, non-invasive technique of oral-fluid collection to assess population
immunity to measles. Only three studies have used oral fluid assays to assess population immunity in low income
settings, but none have directly attempted to validate conventional coverage estimates against this tool (8, 9, 19).
This study moves beyond estimation by developing a model to estimate true immunity based on comparisons
between oral fluid-confirmed immunity and reported immunization. If scaled up to the national level,
governments, UNICEF or donor agencies would be able to ‘correct’ reported immunization coverage estimates by
collecting a small subsample of oral fluid to identify characteristics of households that are likely to over- or underreport their true immune status.
With such broad use of immunization coverage indicators now, it is crucial to develop tools to validate the
accuracy of conventional methods for estimating coverage. Measles immunization lags behind the other basic child
vaccines and therefore is used as one of three indicators to measure progress toward the 4th Millennium
Development Goal (MDG4) of reducing child mortality (26, 27). As we approach the 2015 deadline for the
MDGs, there may be incentives for governments to over-report measles immunization coverage. Thus, exploring
the accuracy of measles immunization coverage is not only relevant to measles policy but will also generate
evidence on how to interpret and validate different sources of coverage data, which is applicable for surveillance of
all childhood immunizations.
‘Card + report’ is the standard method used by DHS and WHO coverage surveys in which data from vaccination cards are
used. If cards are not available or incomplete, maternal recall of child’s vaccinations is substituted.
3
3
Kyla Hayford
Delta Omega Proposal
Advisor: Orin Levine
Itemized Budget *
Oral fluid swabs for ½ of participants (Oracol oral specimen collection devices), 330 x $1.50
Plastic covers for vaccination cards (gift for participating in study), 660 x $0.35
Translation and back translation of survey and informed consent forms in Bangla
*Note: Additional grants will be used to share costs for assays, oral fluid collection devices,
staff and transportation costs.
$495
$231
$270
_____
Total: $996
Sources
1. WHO. Weekly epidemiologic record: Progresss in global measles control and mortality reduction,
2000-2007. World Health Organization. 2008;43(49):441-8.
2. WHO. Immunization Surveillance, Assessment and Monitoring. n.d. [cited February 8, 2009];
Available from: http://www.who.int/immunization_monitoring/diseases/en/
3. Lim SS, Stein DB, Charrow A, Murray CJ. Tracking progress towards universal childhood
immunisation and the impact of global initiatives: a systematic analysis of three-dose diphtheria,
tetanus, and pertussis immunisation coverage. Lancet. 2008 Dec 13;372(9655):2031-46.
4. Bolton P, Holt E, Ross A, Hughart N, Guyer B. Estimating vaccination coverage using parental
recall, vaccination cards, and medical records. Public Health Rep. 1998 Nov-Dec;113(6):521-6.
5. Goldman N, Pebley AR. Health cards, maternal reports and the measurement of immunization
coverage: the example of Guatemala. Soc Sci Med. 1994 Apr;38(8):1075-89.
6. Suarez L, Simpson DM, Smith DR. Errors and correlates in parental recall of child immunizations:
effects on vaccination coverage estimates. Pediatrics. 1997 May;99(5):E3.
7. Valadez JJ, Weld LH. Maternal recall error of child vaccination status in a developing nation. Am J
Public Health. 1992 Jan;82(1):120-2.
8. Cutts FT, Bartoloni A, Guglielmetti P, Gil F, Brown D, Bianchi Bandinelli ML, et al. Prevalence of
measles antibody among children under 15 years of age in Santa Cruz, Bolivia: implications for
vaccination strategies. Trans R Soc Trop Med Hyg. 1995 Jan-Feb;89(1):119-22.
9. Ohuma EO, Okiro EA, Bett A, Abwao J, Were S, Samuel D, et al. Evaluation of a measles vaccine
campaign by oral-fluid surveys in a rural Kenyan district: interpretation of antibody prevalence data
using mixture models. Epidemiol Infect. 2008 Jun 10:1-7.
10. George K, Victor S, Abel R. Reliability of mother as an informant with regard to immunisation.
Indian J Pediatr. 1990 Jul-Aug;57(4):588-90.
11. Langsten R, Hill K. The accuracy of mothers' reports of child vaccination: evidence from rural Egypt.
Soc Sci Med. 1998 May;46(9):1205-12.
12. Murray CJ, Shengelia B, Gupta N, Moussavi S, Tandon A, Thieren M. Validity of reported
vaccination coverage in 45 countries. Lancet. 2003 Sep 27;362(9389):1022-7.
13. McDade TW, Williams S, Snodgrass JJ. What a drop can do: dried blood spots as a minimally invasive
method for integrating biomarkers into population-based research. Demography. 2007
Nov;44(4):899-925.
14. Gay NJ, Vyse AJ, Enquselassie F, Nigatu W, Nokes DJ. Improving sensitivity of oral fluid testing in
IgG prevalence studies: application of mixture models to a rubella antibody survey. Epidemiol Infect.
2003 Apr;130(2):285-91.
15. Vyse AJ, Cohen BJ, Ramsay ME. A comparison of oral fluid collection devices for use in the
surveillance of virus diseases in children. Public Health. 2001 May;115(3):201-7.
16. Nokes DJ, Enquselassie F, Nigatu W, Vyse AJ, Cohen BJ, Brown DW, et al. Has oral fluid the
potential to replace serum for the evaluation of population immunity levels? A study of measles,
rubella and hepatitis B in rural Ethiopia. Bull World Health Organ. 2001;79(7):588-95.
17. Nokes DJ, Enquselassie F, Vyse A, Nigatu W, Cutts FT, Brown DW. An evaluation of oral-fluid
collection devices for the determination of rubella antibody status in a rural Ethiopian community.
Trans R Soc Trop Med Hyg. 1998 Nov-Dec;92(6):679-85.
4
Kyla Hayford
Delta Omega Proposal
Advisor: Orin Levine
18. Nokes DJ, Nigatu W, Abebe A, Messele T, Dejene A, Enquselassie F, et al. A comparison of oral
fluid and serum for the detection of rubella-specific antibodies in a community study in Addis Ababa,
Ethiopia. Trop Med Int Health. 1998 Apr;3(4):258-67.
19. Lowther SA, Curriero FC, Kalish BT, Shields TM, Monze M, Moss WJ. Population immunity to
measles virus and the effect of HIV-1 infection after a mass measles vaccination campaign in Lusaka,
Zambia: a cross-sectional survey. Lancet. 2009 Feb 9.
20. Parry JV, Perry KR, Mortimer PP. Sensitive assays for viral antibodies in saliva: an alternative to tests
on serum. Lancet. 1987 Jul 11;2(8550):72-5.
21. Gay NJ, Hesketh LM, Morgan-Capner P, Miller E. Interpretation of serological surveillance data for
measles using mathematical models: implications for vaccine strategy. Epidemiol Infect. 1995
Aug;115(1):139-56.
22. Kremer JR, Muller CP. Evaluation of commercial assay detecting specific immunoglobulin g in oral
fluid for determining measles immunity in vaccinees. Clin Diagn Lab Immunol. 2005 May;12(5):66870.
23. Nigatu W, Nokes DJ, Enquselassie F, Brown DW, Cohen BJ, Vyse AJ, et al. Detection of measles
specific IgG in oral fluid using an FITC/anti-FITC IgG capture enzyme linked immunosorbent assay
(GACELISA). J Virol Methods. 1999 Dec;83(1-2):135-44.
24. Hesketh L, Charlett A, Farrington P, Miller E, Forsey T, Morgan-Capner P. An evaluation of nine
commercial EIA kits for the detection of measles specific IgG. J Virol Methods. 1997 Jun;66(1):51-9.
25. WHO. WHO Immunization Coverage Cluster Survey: Reference Manual. World Health
Organization. 2005.
26. WHO. Weekly epidemiologic record: Progresss in global measles control and mortality reduction,
2000-2007. World Health Organization. 2009;43(49):441-8.
27. UNDP. United Nations Development Programme: Millennium Development Goals (MDGs). n.d.
[cited February 12, 2009]; Available from: http://www.undp.org/mdg/
5
Kyla Hayford
Delta Omega Proposal
Advisor: Orin Levine
6