Course Introdução à Medicina
School year 2010/2011
BLAND-ALTMAN’S
METHOD:
Teacher Cristina Maria Nogueira da Costa Santos
Class 04










The importance of precise measurement
History of Bland-Altman’s method
What is the Bland-Altman’s Method
Our main question
Aims
Information collection
Methods
Our results
Discussion
Final observations
Bland-Altman’s
method is used to…
What is the importance of
precise measurement
in medicine
?
 In medicine, any error can be fatal;
 The constant development of new measurement methods;
How can one guarantee that the values read in the new
instrument are compatible with the values from the old one?
COULD WE?
WHY CAN’T WE USE THE
CORRELATION COEFFICIENT ?
 The use of correlation is misleading;
 It does not measure the degree of agreement between two
methods but only their correlation;
By how much the new instrument
is likely to differ from the old one?
15 g; 30g; 25g.
+ 15
-5
5 g; 20g; 15g.
+ 15
-5
The
Bland-altman’s
method
history
 In 1980 cardiologists asked for Martin Bland and G.
Altman help in assessing the rate of agreement
between two methods.
 After reviewing the different methods available,
they found out none of them served this need and
decided to create one of their own.
 1983 – first publication: The Institute of Statistics 1
1
Bland J M and Altman D G
Bland JM, Altman DG. This week's citation classic: Comparing methods of clinical
measurement. Current Contents CM20(40) Oct 5, 8, 1992
The
Bland-altman’s
method
history

1986: Lancet magazine published the article:
“Statistical methods for assessing agreement between
two methods of clinical measurement”

Most quoted article from Lancet magazine: > 18 000 citations.
What is the
Bland-altman’s method
2
?
 Alternative method, based on graphical techniques and simple
calculations;
 Created in 1983 by Bland J M & Altman D G;
 Used to assess agreement between two methods of clinical
measurement.
 By how much the new instrument is likely to differ from the old one?
2 Bland
J M & Altman D G. Statistical methods for assessing agreement between two methods
of clinical measurement. Lancet 1:307-10, 1986.
What is the
Bland-altman’s method ?
1st step: build a scatter of the difference between the methods
against their mean and make sure that there is no relation between the
difference and the mean, and that the sd is constant – 1st ASSUMPTION
Example showing relation between averages and differences
Example showing good agreement (no relation)
Example showing not constant sd (standard deviation)
What is the
Bland-altman’s method ?
2nd step: determine the limits of agreement (LA)
LA = [u – 2sd ; u + 2sd]
LA – limits of agreement
u – average of differences
sd - standard deviation of difference
It is expected that most of the differences (95%) lie between the limits of agreement.
Such differences are likely to follow a normal distribution – 2nd ASSUMPTION
Example showing normal distribution
The article that explains the Bland-Altman’s method is the
most quoted article from Lancet magazine,
which means that the method has been much quoted or/and applied.
but…
Verify if the journals with the
greatest impact factor
HAS THE METHOD BEEN
publish articles that apply the method
in a more correct way
CORRECTLY APPLIED ?
Find out what if the
most recent articles
MAIN AIM
apply the method in a more correct way
Selection of the
Population
Selection’s Criteria:
• Documents indexed on ISI quoting
the Bland-Altman’s Method at
19 March 2011, 09:22pm.
18360 documents quoting
the method
• Among all the documents
indexed on ISI quoting the method,
we selected the proceeding papers
and the articles.
Why?
•Proceeding papers and articles are the
ones which can apply the method.
17289 documents as
population
(proceeding papers and articles)
Selection of the
Sample
The Selection:
17289
population
• Random selection of a sample of 70
documents, using the statistical
program SPSS®.
• Random distribution of the sample
between the two classes in an equal
way, using SPSS®.
70
sample
35
Why 70 ?
► With 70 documents we guarantee that all the documents
can be read and analyzed, and at the same time we have an
amount big enough to enable us to draw valid conclusions.
35
Documents’ analysis
And Data collecting
• The 35 documents were distributed to the
class’ students for analysis and data collecting.
• The data were collected
using a checklist developed
for this purpose.
Each document was analyzed twice
by two different students in order
to verify the inter-observer reproducibility
of the checklist.
We built a database using the SPSS®
program and inserted there the data
collected.
A third person
analysed the items
which were differing.
Data
Analysis
•The data analysis was done using the SPSS ® program and
the chi-square statistical test was used to analyze the
results.
•To test the checklist reproducibility a crosstab was done on
SPSS. The two analyses’ answers were tested for their
concordance.
We estimated, with a confidence interval of 95%:
• The proportion of documents that apply the method and the proportion
of documents that only quote it;
• The proportion of documents that apply each of the checklist’s
conditions against the ones that don’t, sorted by the impact factor of the
journals where they were published, and publication year.
applied the method
found articles
30
31
1
35
4
quoted the method
without applying it
 First of all, from the 35 articles that we were supposed to analyze, we
could only find 31. Out of the articles we found, we have concluded that 30
of these have indeed applied the method and that only 1 made a reference
to it without applying it. So, the discussion of the results will be based on
the analysis of the articles that we found (31).
TABLES OF RESULTS
TABLE 1 – “Results Agreement: reproducibility of the answers given to the checklist”
TABLE 2 – “Our Results: the answers to the checklist “
• Despite our initial aim was to analyze the results from a sample of 70
articles (35 from each class), we ended up analyzing just the 35 articles
selected for our class, due to:
 lack of information related to the reproducibility of some results;
 not getting all data on time.
• We were expecting that the majority of scientific publications applied
correctly the method, but the results were not statistically significant (pvalue superior to 0.05). We could see a tendency pointing to:
 a generalized lack of comprehension of the method;
 a major mistakes when it comes to its application.
• It was also expected that the journals with the greatest impact
factor applied the method in a more correct way, however, we can’t
conclude about the relationship between the impact factor and
the correct application of the method because we obtained a
p-value superior to 0.05.
not significant
• The most recent articles often present a histogram or tell us about the
distribution of the differences. This is what we expected: the more recent
the articles were, the better they would have applied the method.
Our Work
Evolution
1.
During this project we came across…
○ We had face some troubles related to data variability (journal, medical
area of the documents…) and so we had to eliminate some of our aims
reducing the number of the initial ones – because some of them could not
get any answer satisfied.
○ The checklist suffered successive changes because there were always
new parameters showing up in each article that we were not expecting
(for example, the method used for primary and secondary data).
Our Work
Dificulties
2.
During this project we came across…
○ The evaluation of the method application ends up to be associated with some
extra difficulties: it involves total comprehension of the method and requires a
careful reading of each article analyzed;
○ Some of the articles did not fit clearly into any point of the questions included
in the checklist, which raised many doubts and, possibly, some errors;
Our Work
Dificulties
3.
During this project we came across…
○ We recorded a not strong reproducibility of the checklist. Despite the
checklist evolution, its adaptation to new variables and the presence of
explanations for each question, we think that the checklist still might have
induced different interpretations between our class students ;
○ Since the beginning, our goal was to examine 70 articles with the
collaboration of the class 13; unfortunately, due to the incompatibility of the
methods used to create the different databases we were not allowed to
complete this stage.
How important
was this work?
4.
This work developed our capabilities to:
 elaborate and apply a checklist;
 analyze and write articles;
 collect and treat data;
 do statistical analysis;
 deal with typical difficulties of a research work.
Finally, a major contribution of this work was understanding
Bland-Altman’s method, since it is an important statistical
method, perhaps important in our academic and medical future.
Professora Doutora Cristina da Costa Santos
Professor Doutor Altamiro Manuel Rodrigues Costa Pereira
Nicola Massy-Westropp B App Sc O.T., PhD.
Dr Paul Henderson
Ennio Lubrano, MD, PhD
Turma 13