To the editor
BMC Public Health
Thank you for your interest in our manuscript and for the opportunity to improve it. You
will find below a reply to the reviewer´s suggestions and criticisms one by one. The
main criticisms are related to our statistical analysis approach. We included in the reply
a detailed justification for the statistical model we used, which was based, according to
good data analysis practices, on the previous exploratory analyses, (not shown in the
manuscript, as common practice).
Besides the changes in the contents, we have also made an extensive language
review, on top of the reviewers´ kind suggestions.
We hope that the manuscript is now suitable for publication.
Yours sincerely,
José Ueleres Braga, on behalf of all co-authors.
Answer to Reviewer
Title: Factors associated with the implementation of the fixed dose combination
RHZE tuberculosis regimen in Brazil: An ecological study
Version: 1 Date: 20 November 2012
Reviewer: Joao Lopes
Thank you for your thorough and detailed review of our manuscript. We appreciate the
opportunity to improve it with your comments. A reply point-by-point to all your
comments follows.
Reviewer's report:
The authors present a study on factors that are associated with the length of the
implementation period of the FDC-RHZE tuberculosis treatment in Brazil. They
have analyzed data at the municipalities level and considered epidemiological,
demographic and health procedure explanatory factors collected from an
available database (SINAN-Tuberculosis). The authors recoded these continuous
variables as binary data in order to calculate association levels between
independent and dependent variables using odds ratio methods. The author
conclude that population size and density, TB incidence, absolute number of TB
patients and use of DOTS and bacteriological confirmation analysis are associated with
a rapid achievement of full-coverage of the implementation of FDC-RHZE.
The choice on the statistical methods, however, seems incorrect. The recoding of
the variables does not seem to have any justification and the result is the lost of
vast amounts of information.
Furthermore the choice on the cut-offs for the recoding seems arbitrary, which is
particularly worrying considering its impact in the final results.
Another criticism is that the authors seem to mislead the reader by overstating
the implication of their analyses. This makes skim reading through the manuscript
particularly confusing.
Finally, I think the manuscript could really benefit from a more careful English
writing.
Major Compulsory Revisions
1) My main criticism is on the choice of the statistical method. Odds ratio is used
to infer associations between binary data, however, none of the variables
considered in the study are binary. In fact, except for "period of initiation of
treatment", all of the considered variables are continuous. Some of these can be
recoded as ordinal variables without much loss (e.g. the ones dealing with
percentages, where the space of results can be easily divided in equally-sized
intervals), but recoding the others as ordinal can be hardly justified. As for
recoding them as binary data, I cannot understand the purpose. I think the
authors should use other types of methods such as t-test, ANOVA, GLM, etc.
R: You are right: we agree that in general, discretization produces loss of information
when analyzing data. However, the choice of data format (continuous, ordinal or binary)
depends on a vast set of methodological issues, besides the way data was collected or
is available. The choice of the statistical model, such as one of the generalized linear
models (GLM), depends on some analytical procedures. The first step is an extensive
exploratory data analysis to evaluate sample distribution characteristics (asymmetry,
kurtosis, etc.) and bivariate relationships, using procedures such as chi-square, t-test
and ANOVA statistical tests. The following step includes the analysis of the relationship
between independent variables. The last stage is modeling the relationships between
dependent and independent variables in a multivariable approach. According to these
good data analysis practices, we followed all these steps. We concluded, based on the
exploratory analysis, that the best variable format for our independent factors and for
the outcome was binary. As a consequence, we chose the final statistical model
(logistic regression, a type of GLM).
Once decision for discretization of continuous variable data is made, the threshold point
should be based on biological, epidemiological and operational criteria, on sample
distribution evaluation and on cut-off points previously used in the literature. Again, we
used exploratory analyses to decide on the cut-off points. We were also careful in
categorizing variables according to cut-offs that make epidemiological and operational
sense. This is now explained in the manuscript.
The exploratory analysis results are not shown in the manuscript, as common practice.
Cut-off points are in Table 3. We added a sentence on the Method and the Result
section to clarify.
2) Another major criticism is the somewhat overstatement presented in the
paper. Particularly, some passages in the text (and even the title of the paper)
are put in a way that leads the reader to think that the authors analyzed factors
influencing the implementation of the FDC-RHZE per se (such as the treatment
efficacy, rate of defaults, etc), instead what the authors present is an analysis on
the factors that influence the length of the implementation period. This should be
clear since the beginning.
R: We used the term “implementation” in the public health context. However, we agree
that it might be misleading. We change the title and the text in order to clarify that we
are not addressing factors that influenced the treatment outcomes or the
implementation per se but the speed of the implementation process.
3) The quality of the written English should be improved. The use of the passive
voice, for instance, does not seem adequate in some occasions, making the
reading rather difficult. There is also a considered heterogeneity in the quality of
the English along the manuscript. The Abstract and the Conclusion are
particularly badly written.
R: All the manuscript was submitted to an extensive language review.
4) There seems to be a bit of confusion on the interpretation of odds ratio in
some parts of the text. Odds ratio measures the strength of associations between
variables not probabilities. These are certainly related, but are not the same.
R: We accepted your suggestion, the sentences were revised.
- Minor Essential Revisions
Title:
The title is misleading. The authors are not studying the implementation of
FDC-RHZE, they are studying the length of the implementation period.
R: The title was changed.
Running title:
Same problem as the title. Consider changing "implementation" for
"implementation period"
R: The running title was changed. We thought that period could still be misleading
because readers could understand we were addressing different moments.
Keywords:
Risk factors as a keyword does not seem adequate.
R: We accepted your suggestion, we deleted this key word.
Abstract:
P2,L24-29 This paragraph should be rewritten as it is quite confusing.
R: The paragraph was revised.
P2,L29 The author did not "evaluated the implementation process", they
establish associations with the length of the implementation period.
R: Accordingly with the title change, we also revised this sentence.
P2,L32 "were eligible" should be "were considered eligible"
R: Suggestion accepted.
P2,L37 "were used as effect measures and their precision" is not proper written
English.
R: The text was revised.
P2,L38 "smaller", "lower" than what? Consider using just small and low.
R: Suggestion accepted.
P2,L39 "were more likely", this cannot be stated just by analyzing odds ratio.
Odds ration does not give probabilities directly nor likelihoods.
R: The text was revised.
P2,L42 "to learn lessons" should be "to learn lessons from"
R: Suggestion accepted.
P3,L50 "thuspreventing" is missing a space
R: Thank you for pointing this out.
P3,L56 The author should give the definition of FDC-RH before using it
R: Done, thank you.
P3,L60 should be "adolescent and adults (over 10 years old)"
R: Done, thank you.
P3,L61 wrong verb tense, "has reduced" should be "reduced"
R: Done, thank you.
P3,L62-65 This sentence is confusing, consider rephrasing it
R: It was rephrased.
P3,L62 Shouldn't "FDC presentation" be "FDC regime"?
R: Done, thank you.
P3,L66 "at that point widely spread" should be something like "which, at that
point, was widely spread"
R: Done, thank you.
P3,L70 I don't understand this: "including availability of new presentations"
R: This expression was deleted.
P3,L72 It seems a odd construction to use "historically" and "until the 90's"
together, consider dropping one of them.
R: Done, thank you.
P3,L74-76 This sentence is confusing
R: The sentence was revised.
P3L76-78 This sentence needs citation. Also, consider breaking it in two to make
it more readable.
R: Citations were added. The sentence was revised.
P3L79 Should be "In Brazil, several public health programs, including DOTS, had
their implementation process investigated."
R: Done, thank you
P3L89 Convoluted sentence construction, consider "The analysis unit of this
analytical ecological study is the Brazilian municipalities"
R: Done, thank you
P3L95 Should be "(e.g. FDC)" or "(i.e. whether FDC was in use)".
R: Done, thank you
P4L107 The authors should mention "period of initiation of treatment", which is
presented in Table 1
R: This is referred to as “i- date of onset of the new regimen implementation”, present
line 110 in the track change document, we changed (both in table and text) to period of
onset of new treatment regimen implementation
P4L108 Table 1 should also present descriptive statistics of "date of onset of the
new regimen implementation"
R: Done, see above, both annotations are now consistent.
P4L110 Sometimes the authors defined "rapid implementation" as up to three
months, other times as less than two months. This should be clear.
R: In our initial exploratory analysis, we divided the distribution in three ranges. In the
following analyses, we understood that a three month cut-off would be more
appropriate. We maintained the three ranges in the descriptive tables and used up to
three months (<=3m) as our outcome cut-off in table 3. We hope this is clarified in the
new version of our manuscript.
P4L113 Authors should be careful on the use of the terminology "distribution", it
is not clear what is referring to (i.e. random variable distributions, mean and st.
deviation calculations).
R: It was rephrased; we hope it is clear now.
P4L116 Which software was used to calculated the crude and adjusted Odds
Rations.
R: This was added to the method session
P4L116-117 What method was used to define CI (e.g. large sample
approximation to the sampling distribution of the log odds ratio)?
R: This was included in the statistical analysis method.
P4L117-119 Which variables were included and which were excluded?
R: They are now cited in the Result method, lines 161-166 of the track change
document.
P4L129-132 This is a repetition of what was mentioned already. Should be
dropped.
R: Thank you for pointing this out. It was deleted.
P4L132-L134 Actually I think the authors should show the values of their
analyses on the onset of the implementation. Also, using methods more suited to
continuous variables should held different results (e.g. t-test, ANOVA, GLM, etc)
R: As explained in the major comments, we explored all possibilities in the analysis
process, with different variable format and statistical tests, including the ones
suggested here. No relevant finding was worth being reported.
P4L135 What do the authors mean when they state that "This outcome can be
considered satisfactory"? What does the all sentence mean?
R: We have rephrased.
P4L143-144 What does it mean that "the levels were intermediate when
compared with the nationwide incidence rate"?
R: This was rephrased.
P5L154-155 Actually, table 2 does not present an association study.
R: You are absolutely right!!! Thank you for pointing this out, it was rephrased.
P5L156 "it is" should be "that take"
R: Done, thank you
P5L162 In truth, your association analysis (Table 3) shows that an higher
coverage of DOTS is associated with faster implementation. Which leads me to
another criticism, how did the authors decide what cut-off to use to code a
continuous variable such as DOTS coverage into a binary variable? Surely, this
cut-off has a big impact on the results of the analysis.
R: We included in the Result section the cut-off for this (DOT) and all other variables.
P5L171-172 This statement has no relation with the study presented here. A
better citation from Habicht et al (1999) would be that they enumerate as
indicators for evaluating public health programs: adequacy, provision, utilization,
coverage and impact.
R: We included the suggested text and tried to clarify in the following sentence that we
studied one of the aspects recommended by the authors.
P5L173-174 Again the authors seem to overstate their study. A more honest
sentence is in order. Something along these lines: "In the present study, we
chose to analyze only the time till full coverage of an implementation due to data
limitations."
R: we added part of the suggested sentence. However, the choice was not based on
data limitation, as stated in the result section. Also, other indicators were not the scope
of the present study. We believe that the reviewer´s concerns are clarified with this new
text.
P5L177-181 Elmahalli et al (2010) study can hardly be related to the study being
presented. The latter was on treatment success, where the former is on time till
full coverage.
R: Elmahalli´s study aims to evaluate an implementation process. We think that the
indicators chosen by the authors, i.e., treatment outcome, are less adequate for this
matter, although they are good indicators of public health program performance. We
have rephrased to clarify our point of view.
P5L181-182 This sentence, in the context of the paragraph it is inserted in, is an
overstatement of the analysis being performed here.
R: We have rephrased the whole paragraph. Thank you.
P5L186-194 These sentences are not relevant to the study and, in my opinion,
should be dropped.
R: We dropped the details on the paragraph, leaving only one sentence to address the
next reviewer´s comment (see below).
P5L195 In truth, the major conclusions of the study are that there are 6 areas
that should be addressed for a successful implementation of a MDR-TB
treatment. Again this studied is not directly related to the analysis presented
here. This is on treatment success, whereas the present study is on time till full
coverage of treatment. This difference should be stated to be more clear to the
reader.
R: We briefly cite other studies to stress the differences between others’ and our
approach.
P6L199 This is an overstatement, "associated with the new regimen
implementation process" should be "associated to the time till full coverage of the
new regimen implementation process"
R: We substituted “implementation process measured as the time to reach full
coverage” for “implementation process”.
P6L206 "higher probability" is not correct, this is most probably true, but it is not
shown anywhere in the results. Remember that, although both odds ratio and
probabilities can be calculated from a contingency table, they are not the same
thing.
R: We rephrased.
P6L244 Maybe the authors can enumerate some of the other factors possibly
associated to rapid implementation of the new regimen.
R: Done.
P7L247-250 This sentence is confusing and should be rewritten.
R: We have rephrased.
P7L255-257 This sentence is an overstatement to say the least. There was no
"evaluation of the implementation process". This would require considering
adequacy, provision, utilization, coverage and impact (Habicht et al, 1999).
R: As in other paragraphs, we have rewritten.
P7L257 "analythical" should be "analytical"
R: corrected, thank you
P7L259 what does "those characteristics" refer to?
R: We specified the characteristics.
P7L261 "as well of other diseases" should be "as well as for other diseases"
R: corrected, thank you
Table 1:
I would break the ordinal categories "More than 4 months" into at least two other.
It will give a better idea of the distribution of times.
R: The exploratory analysis show that the distribution of data does not justify further
ranges.
Table 3:
"up to 3 months" means <3 or <=3?
R: <=3 months, as now specified in the manuscript.
The choice of the cut-offs for each factor seem arbitrary. Can the authors explain
their choice?
R: As explained in major comment 1 reply, we used exploratory analytical procedures
to decide on the cut-off. Our choice of the three month cut-off is explained in the Result
Section.
The absolute number of patient treated is a factor that is only mentioned here. It
should be mentioned in the Methods section. Also, isn't this factor correlated with
the population size and the level of incidence TB?
R: The number of patients treated is indeed related to the population size and incidence
rate, but we explored the independent influence of these factors on the time to
coverage. This was added to the Method section.
Level of interest: An article whose findings are important to those with closely
related research interests
Quality of written English: Needs some language corrections before being
published
Statistical review: No, the manuscript does not need to be seen by a
statistician.
Declaration of competing interests:
I declare that I have no competing interests