Running head: CRITIQUE OF "A META-ANALYSIS OF INTERVENTIONS TO
1
Critique of "A meta-analysis of interventions to promote mammography among ethnic minority
women"
Brian Foster, Roxanne McMurray, Rhonda Oldham, & April Yearwood
Union University
NUR 710
Fall 2011
CRITIQUE OF "A META-ANALYSIS OF INTERVENTIONS TO
2
Critique of "A meta-analysis of interventions to promote mammography among ethnic minority
women"
Han et al. (2009) identified a lack of research related to the interventions to increase the
mammography screening in the ethnic population. Based on the author’s concerns, they
completed a meta-analysis to increase knowledge of the issues.
The Problem or Justification
Han et al. (2009) clearly stated the objective, purpose and goal of the meta-analysis; but
did not utilize the terms problem, clinical question or hypothesis. However, from the stated
objective, purpose and goal, one can identify the problem and determine a need for the metaanalysis. The authors clearly lay a case for the research by discussing multiple studies
concerning the incidence of breast cancer, the need for preventive intervention and the lack of
evaluation of intervention success where ethnic minority women are concerned.
The objective, “was to determine the effectiveness of the interventions for improving
mammography screening among asymptomatic ethnic minority women” (Han et al., 2009, p.
246). “The purpose of this meta-analysis was to determine the effects of intervention programs
on mammography screening among ethnic minority women” (Han et al., p. 247). The authors
identified that other appropriate research lacked data for minority women and noted their goal
“was to fill this gap by conducting analyses on more recent studies …… that were targeted
specifically to ethnic minority women” (Han et al., p. 247).
The scope of the project was appropriate to address the problem or purpose of the study.
Russo states “meta-analysis provides a standard approach for examining the existing literature on
a specific, possibly controversial, issue to determine whether a conclusion can be reached
regarding the effect of a treatment or exposure” (2007, p. 637). The authors plainly noted
CRITIQUE OF "A META-ANALYSIS OF INTERVENTIONS TO
3
current research, a gap in the research and formulated a meta-analysis study to address the issue.
The gap was the lack of other studies related to ethic minority women, which the purpose of the
study specified.
The topic of the review is important to nursing because it specifically relates to women’s
health issues. With the growing number of minorities in America, nurses must be aware of the
disparities and ways to overcome this issue. The results of this study concerning interventions to
promote mammography use in minorities could allow nurses to re-focus their education efforts in
ways that are more effective. For example, this study found “the intervention effect was bigger
for studies conducted in a healthcare setting than for the community-based studies” (Han et al.,
2009, p. 251). This information facilitates health care providers, including nurses, to strengthen
valuable health care resources in the healthcare setting rather than the community environment.
The meta-analysis did not use specific terms such concepts, variables or phenomena.
Polit and Beck state phenomenon is “the abstract concept under study, often used by qualitative
researchers in lieu of the term variable” (2012, p. 737). The phenomenon defined in this study is
asymptomatic ethnic minority women in relation to interventions for improving mammography.
Search Strategy
The study clearly described reasonable criteria for selecting reference material including
only English based articles from 2000 to 2008. The material included was from studies and
abstracts. Studies were located using electronic and hand searches from a variety of collections
with appropriateness verified by two study team members. The criteria for the material included
in the study were:
(a) The study aimed to increase use of mammography screening among asymptomatic
women, either exclusively or in addition to other health behaviors; (b) the study included
CRITIQUE OF "A META-ANALYSIS OF INTERVENTIONS TO
4
more than 40% of women with ethnic minority background (i.e., Asian American, Black
or Hispanic) in the sample; (c) outcomes were based on a woman’s adherence to
mammography’s screening, documented either by self-report or in a clinical database or
medical record; (d) an experimental or quasi-experimental design was used in the study;
and (e) the study was reported between September 2000 and August 2008. Not included
were international studies because the focus was on intervention strategies to improve
breast cancer screening among ethnic minority women in the United States. (Han et al.,
2009, p. 247)
These criteria were appropriate because they focused the literature review back to the
objective, purpose and goal of the meta-analysis. Criteria (a) limited the review to asymptomatic
women that is required by the objective. Criteria (b) focused the review on ethnic women, which
is central to the overall purpose of the study. Criterion (c) and (d) centered the review on
experimental or quasi-experimental designs to ensure high-level research techniques focused on
the appropriate subject material were included.
The databases used by the researchers included Medline, CINAHL, PsycInfo and Web of
Science. The databases were appropriate because they offer a large number of original research
articles and provided a wealth of information as evidenced from the reference list; but they are
not all inclusive. Han et al. discussed by “excluding unpublished sources (e.g., dissertations) or
government documents that might not be readily available” (2009, p. 251). Additional databases
to consider are Ovid and EBSCO because of their unique search abilities; however these are
subscription and do include a cost for use.
The study included the key word phrases of “Asian, African American, Hispanic or
Latino, breast cancer screening, mammography, experimental studies, interventions, and
CRITIQUE OF "A META-ANALYSIS OF INTERVENTIONS TO
intervention studies (Han et al, 2009, p. 247). The key words seemed to include most of the
necessary items for an extensive review based upon the study’s objective, purpose and goal; but
it was not exhaustive. One of the main words used throughout the meta-analysis was minority,
but the authors did not include it as a key word phrase for the review.
Han et al. (2009) discussed the study methods of electronic and hand searches to locate
749 titles and abstracts. However, the authors did not address any specific measures utilized to
identify unpublished studies. Additional sources of work are identified by expanding the search
to other databases beyond ones that focus on original, published work.
The Sample
The authors utilized studies based on specified inclusion criteria. The inclusion and
exclusion criteria were easily elicited. They selected studies that had outcomes in common.
Among these inclusion criteria were increasing mammography among “asymptomatic women”,
the study must contain at least 40% of women with an “ethnic minority background”, outcome
was dependent upon compliance with screening, “experimental or quasi-experimental design,
and the study must have been reported between “September 2000 and August 2008” (Han et al.,
2009, p. 247). The authors excluded international studies because they would not be
representative of strategies implemented in the United States. The inclusion and exclusion
criteria seem to express coherently the population of interest.
The search strategy employed initially yielded 749 studies. The authors state that, “of
these 749 studies, 607 were excluded” (Han et al., 2009, p. 247). However, the authors provide
no explanation for this initial exclusion. They further exclude from the remaining 142 based on
the absence of characteristics from the inclusion criteria. The authors utilized 23 studies that
meet inclusion criteria (Han et al., 2009, p. 247). The population size available across all
5
CRITIQUE OF "A META-ANALYSIS OF INTERVENTIONS TO
6
included studies is 22,949 (Han et al., 2009, p. 249). The search strategy employed yielded a
large sample of studies but given the exclusion of 607 studies with no description of where they
fell among inclusion and exclusion criteria, it cannot be determined if this was a comprehensive
analysis.
There is no mention of specific strengths or weakness of the sample included in this
analysis. However, the authors did test for bias in the 23 included studies by using statistical
strategies such as the leave-one-out approach. Once they removed a study, they recalculated the
mean weighted effect size. They also used the funnel plot and fail-safe N (Han et al., 2009, p.
249). This type of statistical analysis for bias shows the strength of the study sample.
The authors did not describe a process to resolve issues with excluded studies. There
were no accounts of contacting the original researchers to obtain previously unpublished data.
Those studies with insufficient data were simply excluded. The authors state, “In addition, 13
did not include enough information to calculate the effect size” (Han et al., 2009, p. 249).
Studies were also excluded for reasons other than insufficient data. For example, the
authors did not include 43 studies because they did not include at least “40% ethnic minority
women” (Han et al., 2009, p. 247). They also excluded studies based on lack of a control group,
lack of clearly stated interventions, and absence of the appropriate outcome such as “a woman’s
adherence to mammography screening” (Han et al., 2009, p. 247). The authors also excluded
based on sample populations that were used in other included studies.
Quality Appraisal
The reviewers utilized a numeric scoring system to delineate the quality of the included
studies. This was derived from a review of “relevant literature” (Han et al., 2009, p. 248). There
is no defense of the quality determination method other than the citations provided. The quality
CRITIQUE OF "A META-ANALYSIS OF INTERVENTIONS TO
7
was scored based items such as study design, outcome measure, clarity of outcome definition,
and information on withdrawal. Each item had a possible score of 0 or 1 depending on the
presence or absence of the desired characteristic. The authors state, “For the purpose of this
analysis, studies with scores of 1-2 were considered to be low quality and those with scores of 34 were considered to be high quality” (Han et al., 2009, p. 249). While this method was clear
and easy to follow, the authors made no attempt to defend their use of this quality rating tool.
The authors utilized two raters to code the quality variables independently. In cases of
identified inconsistencies, the team members would resolve the discrepancy by discussion. The
authors did provide a significant қ for coding agreement 0.86 (Han et al., 2009, p. 249).
However, the authors did not describe individual discrepancies or give specifics about their
nature, scope, or significance.
The appraisal information was overall adequately described. The authors allowed the
reader to see the quality appraisal tool and how they used it. The results of the quality rating of
each included study are listed in the supplemental hyperlink provided in the text of the study
(Han et al., 2009, p. 249).
Data Extraction (from individual studies reviewed)
The authors adequately discussed the methods of data extraction. They stated after the
literature was reviewed with limited via key words and English only articles, relevant studies
were screened for inclusion criteria. Then, after being identified by two independent authors, the
“titles and abstracts of all identified studies were reviewed by two study team members” (Han et
al., 2009, p. 247) in which Russo (2007), states that “two or more authors of a meta-analysis
should abstract information from studies independently” (p. 639). However, they did not state
whether they were blinded to the authors. The authors developed a structured abstraction form
CRITIQUE OF "A META-ANALYSIS OF INTERVENTIONS TO
8
they refer to as “Figure 1. Summary of the study selection process” (Han et al., 2009, p. 248) as
well as a written description to define their process and reasoning for exclusion out of 749
studies and the eligibility for inclusion of the 23 studies obtained for systematic review.
The reviewers coded the characteristics of the studies (variables for inclusion) as:
first author, year, study design, setting, sample (percentage of ethnic minority women),
unit of assignment, type of intervention, intervention period, time to outcome measure
(months), method of outcome ascertainment, number of participants in the study groups,
mean age of the study sample, proportion of mammography screening for the treatment
and control groups, theory, control group (no intervention, minimal intervention or usual
care, or other non-breast-cancer intervention), any cultural strategies used, and study
quality. (Han et al., 2009, p. 248)
This information was established beforehand to help create a homogenous population with the
authors giving a rationale for exclusion and inclusion criteria. There is even more information
provided on each study via an online link to a summary of the patient data characteristics for this
meta-analysis. They further go into how they categorized the interventions as listed on page 248
of the article.
The authors created and provided a table that addresses how they determined the study
quality rating and defined each rating as: “studies with scores of 1 – 2 were considered to be low
quality and those with scores of 3 – 4 were considered to be high quality” (Han et al., 2009, p.
248). Russo (2007), states that “a quality score for each study included in a meta-analysis may
be useful to ensure that better studies receive more weight” (p. 639). Additionally, “two raters
independently coded the variable. Every discrepancy was identified and resolved by discussion
among team members” (Han et al., 2009, pp. 248-249). Furthermore, they conducted a
CRITIQUE OF "A META-ANALYSIS OF INTERVENTIONS TO
9
sensitivity analysis on high versus low quality ratings as well as on possible influential studies to
“gauge the impact on the variability of effect sizes” (Han et al., 2009, p. 250). Lastly, additional
analyses were performed to examine publication bias as noted in the data analysis below.
Data Analysis-overall
The reviewers explained their method of pooling the results and how they integrated the
data. They used the meta command in Stata to conduct the data analysis and to determine an
overall mean weighted effect size (MWES) for the 23 studies (Han et al., 2009). The commonly
used DerSimonian-Laird fixed results model was used to summarize the interventions effect
along with using estimated confidence intervals. They used multiple approaches to compare the
overall MWES along with examining publication bias via a funnel plot and fail-safe N
calculations. The analysis of data seems to be thorough and credible within the studies used.
Subgroup analyses and sensitivity analyses were conducted to rid the study of any extreme
outliers that might skew the results. A table on page 250 shows the estimated effect sizes (p <
.001) with a 95% confidence interval (0.043 to 0.113) for each of the 23 studies and is helpful to
show the summary of findings. The authors also referenced an online table “which summarizes
the characteristics of studies included in this meta-analysis, http://links.lww.com/A1244” (Han et
al., 2009, p. 249).
Data Analysis
The meta-analysis performed was justifiable. The implications associated with
the meta-analysis results indicate the “interventions were effective in improving mammography
screening among ethnic minority women” with a 95% confidence interval from 23 studies (Han
et al., 2009, p. 259).
CRITIQUE OF "A META-ANALYSIS OF INTERVENTIONS TO
10
Appropriate procedures were followed for computing effect size for all relevant outcomes.
Han and colleagues discussed in detail how effect size was calculated and what the sensitivity
analysis included. Odds ratio was calculated for the estimated effect sizes to examine publication
bias. However, the odds ratio value was not disclosed. The pooled results calculated effect size by
using the random-effects model to report for significant heterogeneity among interventions, which
was indicated by a significant Q (Q = 92.95, df = 22, p<.001) (Han et al., 2009, p. 249). The
random-effects model was repeated to calculate MWES for various subgroups to further deal with
heterogeneity.
The appropriate subgroups analyses were accounted for in detail, to examine effectiveness
of the different intervention methods to indicate whether effect size results varied by population
subgroups. Publication bias was adequately and clearly dealt with. The authors first examined
publication bias by “plotting the standard error by the natural logarithm of the logged odds ratio for
the estimated effect size” (Han et al., 2009, p. 250). A funnel plot, and fail-safe N was also
completed. The authors indicated that the funnel plot appeared slightly asymmetrical indicating
potential publication bias. However, the authors stated that the fail-safe aided in adding rigor the
study. More stated values, charts and tables would have been helpful to aid in summarizing the
findings.
Usability of Results-Conclusions
The conclusions about the quality, quantity and consistency of evidence, are addressed in the
discussion and are related to the research question. Quality and quantity are discussed in the
limitations section. The results indicate that there is an overall increase of mammography use for
minority women in the intervention group by 7.8% (Han et al., 2009, p. 251). Han and company
acknowledged and gave a rational for why the results could not be compared directly with those of
CRITIQUE OF "A META-ANALYSIS OF INTERVENTIONS TO
11
other meta-analysis, because of different study selection criteria and intervention typology, which
were consistent with two other referenced studies. Inconsistency and low quality findings were
addressed and accounted for in the promotoras intervention. The authors contributed the small
and negative effect size to the study design, non-randomization, and large sample sizes. The
authors indicated that in the future when using the network design such as promotoras, a smaller
community with a well-prepared trainer that had a well-monitored plan, would be more
appropriate. Interventions that were most effective in analyzing effect size were specified, along
with which intervention studies, based on valid theories, were suggested for further studies.
Consistency was present along with previous meta-analysis and found that intervention effect was
bigger in community or healthcare settings.
The significant results versus the non-significant results of the individual studies were
indicated to generalize the outcomes and feasibility. Non-significant data was stated and a rationale
was given in regard to implications in designing mammography-enhancing interventions for ethnic
minority women. The majority of the results were nonsignificant. The access enhancing
intervention, and directed approaches “yielded the biggest increase in mammography use, but the
results could not be computed directly to other meta-analyses due to different study selection
criteria and intervention typology” (Han et al., 2009, p. 251). The tailored, theory based
intervention resulted in a bigger effect size compared to the non-tailored intervention. The authors
could have eliminated some of the low-quality studies to change the results of analyses, but the
study was still indicated as rigorous.
Ethical implications were addressed by assessing that community research with a traditional
randomized controlled trial could put the control group at an unfair advantage, if free
mammograms were not available to all.
CRITIQUE OF "A META-ANALYSIS OF INTERVENTIONS TO
12
The limitations of the review were noted. The limitation of lacking culturally matched
materials or matching interventions deliveries were pointed out and suggested that future metaanalysis is needed, as more experimental data is available. Also, published analysis of intervention
effects for Asian or Pacific Islanders or Hispanic women were lacking. The design limitations were
found in examination of characteristics in the Hispanic subgroup, which was attributed to a large
sample size, less controlled situation, and lack of randomization. Unpublished sources resulted in
other limitations; but were justified as not affecting their inclusion and exclusion data. However,
Polit and Beck, (2012) indicates that the “exclusion of grey literature in a systematic review can
lead to bias, particularly the overestimation of effects” (p. 658). Another study revealed “restricting
studies by language overestimates treatment effect only by 2%” (Russo, 2007, p. 640). The reviews
used only from the United States pose a limitation, but the study question indicated that the study
was specific to the United States and findings may not be generalized to other countries. Another
limitation was the difficultly to tease out the effect of each individual component. A factorial
design would solve this limitation, but was avoided because of expense. This is the only time
expense was mentioned or considered in this study.
The authors declared a limitation in the quality and quantity of their study. Uneven quality
and quantity of meta-analysis studies is common. The authors attempted to address this by assigning
estimates to low and high studies. Four to six studies were labeled as nonsignificant studies, which
the authors stated that the low estimates should have been used as preliminary studies, however the
studies were still used. The Quality of Reporting of Meta-analyses (QUOROM) guideline to
determine validity, of the results, would have been helpful and wise to use.
Implications for healthcare practice would be to increase access-enhancing strategies,
increase the use of a theory-based, tailored approach and increase active community partners to
CRITIQUE OF "A META-ANALYSIS OF INTERVENTIONS TO
13
improve screening outcomes. Through the above intervention, mammography use among ethnic
minority women would improve.
The need for future research was clearly stated. The authors suggested that future studies
needed more tightly controlled trials to improve results when reviewing mammography studies
among minority groups, particularly Hispanic women. They also suggested that alternative
research is warranted to allow for research to benefit all ethnic minority communities.
Additionally, it was suggested that future interventions are needed to promote mammography
screening among ethnic minority women. Finally, the authors suggested that it would be helpful
in future meta-analysis studies if the rate differences of mammography screening were consistent.
The results indicate that there was “an average of 7.8% increase in mammography use in
minority women in the treatment groups receiving a variety of interventions” (Han et al., 2009, p.
251). It was concluded that this “meta-analysis had a estimated intervention effect with a 95% CI
of 0.043 to 0.113 indicating that the interventions were effective in improving mammography
screening among ethnic minority women by using a random-effects model to account for significant
heterogeneity” (Han et al., 2009, p. 249). The study size included 23 studies that dealt with
variance, sensitivity analyses, quality, quantity, bias, and limitations, and the study had a robust
combined result. This meta-analysis is reasonable to address this clinical problem. The results
suggest that interventions are a need for increase use of a theory-based, tailored approach with
enhancement of access, to promote mammography among ethnic minority women. Although it is
difficult to decipher the results as a reader, the study can be a footing for future studies.
With the evidence provided, it should be suggested that a change in practice is
recommended. The study has external validity, and generalization of the results could be applied to
minority women. Through an increase use of a theory-based, tailored approach and enhancement of
CRITIQUE OF "A META-ANALYSIS OF INTERVENTIONS TO
14
access, minority women will improve in mammography screening. This will aid in detection and
diagnosis of breast cancer and increase survival. Anytime survival rate can be increased,
accommodations in practice should be made as it is realistic and possible.
CRITIQUE OF "A META-ANALYSIS OF INTERVENTIONS TO
15
References
Han, H., Lee, J., Kim, J., Hedlin, H. K., Song, H., & Kim, M. T. (2009). A meta-analysis of
interventions to promote mammography among ethnic minority women [Entire issue].
Nursing Research, 58(4).
Polit, D.F., & Beck, C. T. (2012). Evidence-based practice in nursing & healthcare (2nd ed.).
Philadelphia, PA: Wolters Kluwer.
Russo, M. W. (2007, August). How to review a meta-analysis. Gastroenterology & Hepatology,
3(8), 637-642.