Running head: META-ANALYSIS
1
4 points out of 4 points for report. 1 point for peer evaluations. Total 5 points.
Meta-analysis "Pertussis- The Case for Universal Vaccination"
Heather Surcouf, Leader
Laura Martin, Recorder
Lisa Bayhi, Member
Lisa Nicoletti-Wack, Member
Southeastern Louisiana University
Running head: META-ANALYSIS
2
Meta-analysis "Pertussis- The Case for Universal Vaccination"
Introduction
1. Was the rationale for the review described in the context of what is already
known? Yes, this author described the trend of an increasing incidence of pertussis outbreaks
over the last several years in conjunction with the introduction of the Tdap vaccine booster in
2005 by the CDC, specifically for adults and adolescents. Rittle (2010) acknowledged that the
introduction of the vaccine was the motivation for the project as well as to explore the need for
pertussis booster when it had been presumed that receipt of the vaccine or having had the disease
provided lifelong immunity along with the ever-rising incidence in pertussis. Rittle (2010)
stated the purpose of the meta-analysis was to examine the epidemiology of pertussis, the
prevalence of pertussis, the role that the waning effectiveness of the vaccine has had on the
spread of the disease, and how the vaccination of adult and adolescents can help reduce the
spread of the disease. Although the author stated a multi-purpose to this meta-analysis, only the
issue of vaccine effectiveness in children examining both missed and late doses seemed to be
examined. The epidemiology of pertussis, prevalence of pertussis, waning effectiveness of the
vaccine as well as a review of current statistics were presented as a literature review and
background information. Good.
2. Was an explicit statement of questions being addressed with reference to
participants, interventions, comparisons, outcomes, and study design (PICOS)provided?
Although no formal PICO statement is presented, Rittle (2010) did attempt to validate his
purpose of identification of research supporting "universal vaccination of adolescents and adults,
reducing disease incidence in infants and young children" (p. 282) by including detailed
statistical data contained within each case control study evaluated and including PICO “like”
Running head: META-ANALYSIS
3
questions within his introduction including: Why is a pertussis booster needed when having the
disease or being vaccinated should provide lifelong immunity? Why does the literature document
an ever-rising incidence in pertussis if all children are receiving the required five doses of
pertussis vaccine prior to entering school? Is vaccine effectiveness waning? Are primary care
providers doing all they can to ensure children and their families are protected from this disease?
Do providers have the knowledge and understanding of pertussis to educate families and the
community? Do parents and schools understand what they must do to control outbreaks once
pertussis has been documented in the community?
It appears he attempted to make the case that the presence of increased incidence of
pertussis in infants is a direct reflection of lack of boosters in adults and adolescents. Although
this may in fact be true, this does not represent a PICO format, rather a hypothesis that the author
is proposing and gathering support for through the literature. Hypothesis does not have a place
in meta-analysis (Picciano, 2008). Good.
Methods
1. Did the researchers specify study characteristics (ie. PICOS, length of follow-up)
and report characteristics (ie. years considered, language, publication status) used as
criteria for eligibility, with a rationale? The author did present information regarding study
characteristics and specified some components of PICOS. The studies were all case-controlled
and evaluated pertussis in the community in conjunction with the role of vaccine effectiveness as
well as low vaccine coverage. However the years considered in each study was not presented,
only the publication dates of the studies. It is assumed by the reader that these are all published
studies as these were all found in the author’s review of the literature and were appropriately
referenced indicating a publication date for each. The reader is also left to assume that all studies
Running head: META-ANALYSIS
4
were published in English, but is not specifically stated in the article. The rationale for eligibility
for review was not clearly specified. The author reported searching through various search
engines with appropriate key words, but limiting his selection of studies to “four of the studies
that evaluated pertussis in the community and the role of vaccine effectiveness and low vaccine
coverage” (Rittle, 2010, p. 287). Although he noted that these are all case-control studies, the
author did not note using this as inclusion criteria. He did not specify other limiters, such as year
of publication, publication status, or language. He also did not give a rationale for the purpose of
selecting the four studies, other than to say that the four studies met the broad characteristics
quoted above.
2. Were all information sources (ie databases with dates of coverage, contact with
study authors to identify additional studies) in the search described as well as the date last
searched? The author did describe the information sources in the search for studies. The search
was “conducted in MEDLINE, Academic Search Elite, Health Business, Cochrane Central
Register of Controlled Trials, CINAHL Plus, Cochrane Database of Systematic Reviews, and
Biological Abstracts” (Rittle, 2010, p. 286). The dates of the articles found ranged from 1965 to
mid-2009, however the date range of the actual search was not presented. There was no
presentation in this article if the author attempted to contact any of the study authors.
The author noted searching bibliographies from articles that were retrieved and CDC
reports pertaining to pertussis, vaccines and pertussis antigens. He also did not note using any
grey literature for search. True
3. Was full electronic search strategy for at least one database presented, including
limits used, so that it could be repeated? The author did not outline an electronic search
strategy that could be replicated.
Running head: META-ANALYSIS
5
4. Was a PRISMA-type flow chart included to summarize the search strategy and
results? No PRISMA flow chart was contained within the article.
5. Was the method of data extraction from reports described( ie. piloted forms,
independently conducted, done in duplicate or parallel), as well as any processes for
obtaining and confirming data from investigators? The method for data extraction for the
reports was not clearly described. The author described utilizing the statistical analysis package,
MedCalc (version 11.1.1.0;2009), to analyze the results of the studies, cases, and controls. In
this package “the Mantel-Haenzel method for calculating the weighted summary OR and
assessing the homogeneity of the four study populations” (Rittle, 2010, p. 287) was used. A test
for heterogeneity was also conducted (Q = 7.184, p = .066). All four study populations were
found to be homogeneous and validated the fixed effects model for meta-analytic calculations.
There was no description of the author confirming data from the investigators. Good.
6. Were all the variables for which data were sought listed and defined (ie. PICOS,
funding sources) including any assumptions and simplifications made? The variables sought
by the author included cases of pertussis, role of vaccine effectiveness and low vaccine coverage.
No funding source information was provided. Not all variables mentioned in the Introduction
were defined nor was the related data sought.
7. Were the methods used for assessing risk of bias of individual studies described
(including specification of whether this was done at the study or outcome level) and how
this information should be used in any data synthesis? There was no description of any risks
of bias in any of the four studies selected. The author did provide a brief summary of the
limitations, rather than biases, of each study. Yes. These included possible misclassification of
cases in the study due no laboratory confirmation of pertussis; delay in case identification; high
Running head: META-ANALYSIS
mobility of families leading to missed doses; only examining the timing of administration of the
fourth dose of the vaccine (Rittle, 2010). There was no discussion of potential bias at the
outcome level only the limitations which was stated to be a result of the limited number of
studies included in the meta-analysis. Good.
8. Were the principal summary measures described (ie. risk ratio, difference in
means)? The principle summary measures described were odds ratio (OR) and confidence
intervals of each study used in the meta-analysis. A pooled OR from all of the four studies was
calculated with a 95% confidence interval. The author reported using Medcalc for a weighted
summary of the odds ratio. A risk ratio was listed in a table summarizing data of 2008 pertussis
rates compared to the rates of the years 2007 to 2004.
9. Were the methods of handling data and combining results of studies described,
including measures of consistency for each meta-analysis? The author only reported using
Medcalc to assess the weighted OR and the homogeneity of the four populations, and a test for
heterogeneity. The author also discussed the confidence intervals both individually and pooled.
The author did not go into detail about how the data was handled, other than to state Medcalc
was used. Had there been more studies included other effect size indicators could have been
used such as gamma and trimmed d (Hogarty & Kromery, 2000 as cited in Rittle, 2010). These
“are more robust to violations of the assumptions of population normality and homogeneity of
variance and can be more appropriate than conventional parametric indices” (Hogarty &
Kromery, 2000 as cited in Riddle, 2010, p. 287). Rittle (2010) also stated that SAS or another
statistical software package that incorporates more robust effect size indicators could have
modified the results of the analysis. Good.
6
Running head: META-ANALYSIS
7
It was not noted by the author whether a third party was used in handling the data. If the
author himself conducted the analysis, bias could have been an issue. Very true. The fixed effects
model was valid for meta-analytic calculations. The measure of consistency was the odds ratio
with a CI of 95%.
The author provided a poor description of methods of validation of consistency and statistical
significance of data obtained via meta-analysis. As stated previously, Rittle (2010) described the
use of MedCalc for statistical support but, reported the calculation of the weighted summary OR
and the assessment of the homogeneity of the combined study populations. Polit and Beck
(2012) noted that although the OR is widely reported, it is less meaningful than risk ratio in
assessing risk index. It is also noted to be difficult to interpret in systematic reviews. It was not
noted by the author whether a third party was used in handling the data. If the author himself
conducted the analysis, bias could be an issue. The fixed effects model was valid for metaanalytic calculations. The measure of consistency was the odds ratio with a CI of 95%. Repeat
10. Were the methods of additional analyses (ie. sensitivity or subgroup analyses, metaregression) described indicating which ones were pre-specified? The author did not outline
additional analyses of the data evaluated within the studies reviewed.
Results
1. Did the authors give numbers of studies screened, assessed for eligibility and
included in the review with reasons for exclusion at each stage, ideally with a flow
diagram? There was no flow diagram presented in this article. The author reported obtaining
only 27 studies relating to pertussis or whooping cough in the database search. There was no
description for exclusion of articles, only a general description of the four chosen for analysis.
2. The characteristics for each study for which data were extracted were presented (ie.
Running head: META-ANALYSIS
8
study size, PICOS, follow-up period) with all citations? The only characteristics described
for each study was the location of the study, the issue in question, study size or number of cases
of pertussis. These were all presented on the table in the article. A formal PICO was not
presented, rather the issue in question in relation to the number of cases to the number of
controls. The citations were provided for all four of the studies included in this analysis.
However it is not clear if all of the references for each of the studies have been provided.
A very broad description of the characteristics of each study is given by the author. There is no
evidence of any encoding of study participant characteristics.
3. Were all outcomes considered (benefits or harms) presented for each study with a)
simple summary data for each intervention group b) effect estimates and confidence
intervals ideally with a forest plot? There was no consideration of the benefits or harms to a
study group, rather rates of pertussis comparing those missing one or more vaccine doses or late
vaccine doses. This information was presented as a simple summary for each study. The author
did provide a simple summary stating that if children missed any dose or delay in dosing of the
pertussis vaccine, they were twice as likely to contract the disease as those children who were
dosed on time or were fully immunized. The effects estimate was calculated using odds ratio
(OR) which Polit and Beck (2012) describe as providing an estimate of relative risk of an event
occurring given one condition versus the risk given another condition. The condition in the case
of this meta-analysis is on-time administration or the completion of the pertussis vaccination
regimen. Polit and Beck (2012) stated that confidence intervals could be built around the OR.
Both the OR and the 95% confidence interval for each study were presented in a table format and
on a forest plot.
Running head: META-ANALYSIS
9
Rittle (2010) considered outcomes and summarized the findings that “a person who was
missing a dose of pertussis vaccine or received a dose of pertussis vaccine late would be 2.133
times more likely to contract the disease than a person who had all doses of vaccine on time”
(p.287).
4. The results of each meta-analysis done was presented, including confidence intervals
and measures of consistency? Confidence intervals for each study, as well as a pooled value
were given. Information regarding significance and confidence intervals were included; however
there was no information regarding measures of consistency, such as Cronbach’s alpha. The
author noted that some studies “made adjustments for confounding factors” (Rittle, 2010, p. 288)
but nothing specific was presented. Rittle (2010) reported a combined OR of 2.13 with a 95% CI
of 1.65-2.76 (p.299).
Discussion
1. The main findings were summarized, including the strength of evidence for each
main outcome and their relevance to key groups ie. healthcare providers, users, policy
makers? The main finding of the meta-analysis were simply summarized as all the studies
indicated “if children missed any doses of pertussis vaccine or had any delay in any dose of the
pertussis vaccine they were twice as likely to contract the disease as their on-time and fully
immunized peers” (Rittle, 2010, p. 287). The pooled data supported this indicating those with a
missing dose or a late dose would be 2.133 times more likely to contract pertussis as those who
had all doses of vaccine on time (95% CI) (Rittle, 2010). Rittle (2010) presented a summary for
application to nursing practice. Rittle (2010) stated “the best method to ensure a reduction of the
incidence in pertussis is to promote universal vaccination not only for all children but also for all
adolescents and adults” (p. 289). Barriers to timely pertussis vaccination were also presented
Running head: META-ANALYSIS
10
and included lack of insurance coverage, not all emergency departments administering Tdap as
the standard tetanus booster, increased cost of Tdap compared to Td booster vaccines and
variations in school immunization requirements among states (Riddle, 2010).
However, what is
unclear is if the recommendation for administration of the Tdap booster to all adolescents and
adults can be made based on the meta-analysis as provided by the author. As previously stated,
the author offered a hypothesis at the onset of the meta-analysis. Based on the data as the author
presented, the analysis did not objectively and statistically supported the author’s
recommendation for revaccination. Don’t be shy.
2. The limitations at study and outcome level (ie risk of bias) were discussed as well as
at review level (ie. incomplete retrieval of identified research, reporting bias)? The
limitations at the meta-analysis study level were summarized. The author acknowledged if there
were more studies included in the analysis a more definitive conclusion may have resulted. The
author stated that no adjustments were made for confounding variables in the meta-analysis as
these had been made in “some of the studies” (Rittle, 2010, p. 288). However the actual
adjustments to the four studies were not available. Had there been more studies included other
effect size indicators could have been used such as gamma and trimmed d (Hogarty & Kromery,
2000 as cited in Rittle, 2010). These “are more robust to violations of the assumptions of
population normality and homogeneity of variance and can be more appropriate than
conventional parametric indices” (Hogarty & Kromery, 2000 as cited in Riddle, 2010, p. 287).
Rittle (2010) also stated that SAS or another statistical software package that incorporates more
robust effect size indicators could have modified the results of the analysis. No risks of bias
were discussed or presented for any of the included studies. The author did discuss limitations to
each study, albeit briefly. These limitations included only accounting if any dose was late,
Running head: META-ANALYSIS
11
problems with delay of case (pertussis) identification and high mobility of families, no laboratory
confirmation of pertussis and only examining timing of fourth dose of pertussis vaccine (Rittle,
2010). There was a risk of statistical (researcher) bias as the results were reported with no
mention of oversight or third party involvement with the statistical analysis and study selection.
3. A general interpretation of the results in the context of other evidence and
implications for future research was presented? The general interpretation of the results of the
meta-analysis was provided as a person missing a dose of pertussis vaccine or receiving the
vaccine late was 2.133 times more likely to contract pertussis as those receiving the vaccine on
time. Implications for further research was suggested by the author. Rittle (2010) stated that
there was no examination in the studies addressing the handling of the vaccine or storage
practices including if the vaccines were frozen or if they were kept too warm, each of which can
either destroy or leave the vaccine ineffective. He also recommended determining whether the
whole cell or accelular pertussis vaccine were used in any of all of the studies as well as what
were the ages of the patients wen these vaccines were administered (Rittle, 2010). However,
these recommendations do not appear to have been gleaned from the results of the meta-analysis,
rather additional questions that the author had regarding the vaccine handling and administration.
The reader is left to assume future research needs.
Funding
The sources of funding for the meta-analysis and other support (ie. supply of data,
data analysis)were discussed including the role of funders for the meta-analysis? There was
no indication from the author that there was funding of the meta-analysis. Therefore the reader
is left to assume that there was not funding involved. There is an acknowledgement from the
author thanking Dr. Gemma Lundberg and Dr. Nancy Rea for their assistance in preparation of
Running head: META-ANALYSIS
12
the manuscript. A gracious acknowledgement to the author’s family is also provided for their
support and encouragement throughout the process of obtaining the doctoral degree.
References
Polit, D., & Beck, C. (2012). Nursing research: Generating and assessing evidence for nursing
practice (9th ed.). Philadelphia, PA: Wolters| Lippincott Williams & Wilkins.
Picciano, A.G. (2008, August 10). B. Meta-Analysis. Podcast retrieved from
http://www.youtube.com/watch?v=iHvn30Zi4zM.
Rittle, C. (2010). Pertussis-The case for universal vaccination. Journal for Specialists in
Pediatric Nursing, 15(4). 282-291. Doi: 10.1111/j.1744-6155.2010.00249.x