Practical Research Methods for Diagnostic Radiographers
RESEARCH PROPOSAL
Research Question /Hypotheses
Is Ultrasound effective in the diagnosis of metatarsal fractures as compared
to X-ray, MRI or CT scan?
0
INDEX
Content
Page number
Abstract
2
Introduction
3
Overview/Rationale
3
Evidence Based Practice and Question
4
development
1
Methods
5
Criteria for selecting studies
5
Database search strategy
7
Selection of studies
8
Assessment of methodology quality
9
Data extraction
11
Data synthesis and analysis
13
Synopsis of the research proposal
16
Timetable
16
References
17
ABSTRACT
The aim of this research proposal is to assess the relative effectiveness of Ultrasound (US) in
the detection of metatarsal fractures compared to the current gold standards for diagnosis
(e.g., MRI, CT scan or X-ray), accessible in tertiary and secondary care centres. The proposal
was performed according to the PRISMA guidelines and used an electronic database specific
search strategy with Cochrane library, Medline and Embase. 1050 potential papers were
obtained from the original search. The titles and abstracts of these papers were screened for
eligible studies and 5 relevant studies were identified. The 5 studies were by the following
authors, Banal et al., (2009); Yesilaras et al., (2014); Kozaci et al., (2017); Ebrahimi et al.,
(2019) and Crombach et al., (2020). Of the 5 eligible studies found, 4 studies included
metatarsal fractures only and 1 study included fractures at the 5thmetatarsal, distal fibula and
tibia. Diagnostic performance of ultrasound was evaluated with a quantitative sub-group
meta-analysis on the 4 studies that included metatarsal fractures only using measures of
specificity and sensitivity.
2
1.1 INTRODUCTION
US is a non-invasive diagnostic method used to image inside the body. One of the most
common uses of US is during pregnancy, to monitor the growth and development of the
fetus. US has been used in the detection of paediatric fractures (weeks et al., 2016) and to
detect occult adult fractures that might be missed with conventional radiographs (Wang et al.,
1999). Injuries to the distal lower limb specifically the foot and ankle occurs as buckling or
blunt trauma and generally lead to a sprain, strain and less often fractures (Crowley et al.,
2019). Although such injuries are not fatal, early diagnosis and treatment are important for
patients with foot fractures in order to prevent long-term complications (Wood et al.,2019).
1.2 Overview/Rationale
The diagnosis of foot injuries is usually performed by the use of X-Ray radiography and this
is considered as the standard reference (Nery, Raduan and Baumfeld, 2016). X-Ray
radiography involves exposure to ionisation radiation with the possibility of causing
teratogenic or carcinogenic effects (Ait-Ali et al., 2009).
Other imaging modalities, e.g., Magnetic Resonance Imaging (MRI), Computed Tomography
(CT), as well as US, have been considered to improve the diagnostic accuracy of foot
fractures (LiMarzi et al., 2016). Nevertheless, selection of MRI as standard reference is
considered to increase unnecessary time and expenses and CT examination might bring
unnecessary ionisation radiation to patients (Ait-Ali et al., 2009).
Recent findings have shown that US is a possible alternative for the diagnosis of bone
fractures in the emergency department. A high accuracy in paediatric metacarpal, distal
radius, elbow, ankle, phalanx and metatarsal fractures have been reported (Lee and Yun,
2019); (Douma-den Hamer et al., 2016); (Champagne et al., 2019); (Zhao et al., 2019).
US have the advantage of not producing ionisation radiation. It is easily accessible in the
emergency department and can be performed immediately. This help to reduce diagnostic
delays.
This imaging modality is easy to teach (Mircea et al., 2012) and does not involve the
experience of pain (Saul, 2013). The method is repeatable, portable and there is the
possibility of obtaining information on the musculoskeletal system (Sconfienza et al., 2018).
Furthermore, US can be a valuable tool in remote and resource-poor settings which can help
in point of care fracture diagnosis. This could enhance rapid and adequate management of
fractures in the community.
3
1.3 Evidence Based Practice and Question development
US usage in fracture detection has formerly been reviewed in lower extremity stress fractures
(Jeans, 1986), in long bone fractures (Chartier et al., 2016), in upper and lower limb fractures
(Champagne et al., 2019), in paediatric forearm fractures (Douma-den Hamer et al., 2016), in
foot and ankle fractures (Wu, Wang and Wang, 2020) and in acute extremity fractures (Joshi
et al., 2013). An important feature in these investigations is the actual diagnostic accuracy of
US for detecting bone fractures. Therefore, this research proposal is a systematic review and
meta-analysis synthetising the diagnostic performance of US specifically in fractures of the
metatarsal bones.
Statement of Problem/Purpose
The aim of this research proposal is to assess the relative effectiveness of US in the detection
of metatarsal fractures compared to the current gold standards for diagnosis (e.g., MRI, CT
scan or X-ray).
Research Question /Hypotheses
Is US effective in the diagnosis of metatarsal fractures as compared to X-ray, MRI or CT
scan?
The Population Intervention Comparison Outcome (PICO) model was used to frame the
clinical question forming the bases of this research proposal (Shamseer et al., 2015). This is
illustrated in table 1.
Table 1. PICO model
P
Study participants (patients). US operators
I
Point-of-care US (patient bedside, prehospital setting or in Emergency Department)
Fractures clinically believed to be of the metatarsal bones
C
Gold standard diagnostic imaging (MRI, CT scan or X-ray)
O
Diagnostic accuracy (specificity, sensitivity)
Population: The population here refers to patients because the studies are clinical in nature.
Data on US operators and their qualifications and competence were also extracted because
the proposal is based on the use of US as a diagnostic method.
Intervention: The research proposal investigates point-of-care US as a diagnostic modality in
a prehospital or hospital setting, for clinically suspected metatarsal fractures.
4
Comparison: The gold standard for comparison is X-Rays, CT scan or MRI and studies must
be blinded for inclusion.
Outcome: The diagnostic accuracy of US in the identification of fractures of the metatarsal
bones, compared to gold standard imaging using measures of specificity and sensitivity.
2.0 METHODS
The proposal was performed following the Preferred Reporting Items for Systematic Reviews
and Meta-Analyses (PRISMA) guidelines (Moher et al., 2015).
2.1 Criteria for selecting studies
To increase validity and reproducibility screening of appropriate studies was performed based
on a prearranged set of inclusion and exclusion criteria. This included the parameters of the
study question as itemised using the PICO framework in table 1. (see table 2).
Table 2. Eligibility criteria
Domain
Inclusion
Exclusion
Study type
• Randomised control trials
•Case reports, guidelines, letters,
•Prospective studies
consensus statements, and unpublished
•Studies involving patients with
articles
clinically suspected metatarsal
•Studies that contained an overlapped
fractures.
population
•Full texts published in a peer
•Studies without sufficient data to
review journal
construct diagnostic 2x2 tables
•Nonclinical studies
•Literature review
•Conference proceedings
Participants • Humans
•Non-human subjects
• Adults
•Simulated patients
•Clinical setting
•Simulated fractures
• Blinded US
•Non-blinded image interpretation
• Operators
•Non blinded US operators
•Exclusively paediatric patients
•Mixed adult and paediatric populations
5
Setting
•Emergency Department
•None
•Hospital bedside
•Prehospital setting
Procedure
•Diagnostic
•Therapeutic ultrasonography
•Ultrasonography
•Diagnostic X-ray, MRI or CT
scan believed to be metatarsal
fracture clinically
Outcome
•Fracture identification
•Fracture treatment
•Fracture diagnosis
•Bone density assessment
Following an initial review of the literature, further searches that were performed focused on
studies in adults using ultrasonography as the test method examining cases of fractures
believed to be of the metatarsal bones. Because the proposal is on clinical practice, trials that
did not include humans or simulated fractures were not selected for inclusion. In order to
adequately study how accurate ultrasonography is, the distribution of participants ought to
have been random or consecutive. Therefore, studies were chosen considering the methods
used, to select only randomised controlled trials or prospective studies that involved adequate
diagnostic imaging interpretation blinding. During the selection process, restriction was not
made on the language or year that the study reports were published, but complete articles
issued in a peer-review journal is what was needed for a study to be selected. Clinical studies
which employed therapeutic US scans were not selected because the research proposal is
based on ultrasonography as an imaging method for diagnostic objective. Furthermore, any
study that noted any of the primary outcome mentioned in table one (PICO framework) were
selected. Studies that did not report diagnostic accuracy of sonography in comparison with
one of the reference standards were not selected for the proposal. Finally, secondary outcome
measures such as patient management, user’s perspective and comparative time to diagnosis,
were taken into account separately and did not affect study selection.
6
2.2 Database search strategy
Medline, Cochrane library and Embase are electronic databases which were systematically
searched.
Medline provides more than twenty-six million references to life sciences and biomedical
journal articles. It has a comprehensive journal selection process, and added value of using
the Medical Subject Headings, NLM controlled vocabulary to index citations. However, this
database is US bias. The indexing is poor, and it can be difficult to pinpoint the relevant
references (Nih.gov. 2019).
Embase is a biomedical database that encompasses international biomedical literature from
1947 to the present. The articles in this database are properly indexed using Elsevier's Life
Science thesaurus. However, it is European bias and not so easy to search for non-medical
content (Elsevier 2009).
Cochrane library is a database of systematic reviews containing 4000 plus reviews on
healthcare topics produced by 49 subject specific review groups. This database contains high
quality research data, but it is very small compared to traditional databases and some subject
areas for instance, allergy and intolerance are under-represented (Cochranelibrary.com.
2009).
These databases were systematically searched to locate eligible studies from when they were
published to April 2021. Searches were performed using the MeSH and keywords terms
concerning each component of the research question. Table 3 depicts the search strategy
used.
Table 3. Medical Subject Heading and search terms
PUBMED
((((ultrasonography[MeSH Terms]) OR (ultraso*[Title/Abstract])) OR
(sonograph*[Title/Abstract])) AND (((foot[Title/Abstract]) OR (metatars*[Title/Abstract]))
AND (((traum*[Title/Abstract]) OR (injur*[Title/Abstract])) OR (fracture*[Title/Abstract]))
EMBASE
#1AND#2AND#3
#1 (fracture*:ab,ti OR injur*:ab,ti OR traum*:ab,ti)
#2 (sonograph*:ab,ti OR ultraso*:ab,ti)
#3 (metatars:ab,ti OR foot*:ab,ti )
COCHRANE
7
#1AND#2AND#3
#1 (sonograph*):ti,ab,kw OR (ultraso*):ti,ab,kw
#2 (fracture*):ti,ab,kw OR (injur*):ti,ab,kw OR (traum*):ti,ab,kw
#3 (metatars):ti,ab,kw OR (foot*):ti,ab,kw
2.3 Selection of studies
Fig. 1 PRISMA flowchart (Moher et al., 2015) of the search process.
8
All the studies from the original search conducted in all the databases with titles relevant to
the proposal were assembled and RefWorks was used to remove all duplicate studies. 1050
potential papers were obtained from the original search. A list containing the titles and
abstracts of the articles was compiled for screening.1039 articles were excluded as it was
evident from the abstract or title that these studies were not appropriate to the proposal.
Finally, screening of 11 full text articles against inclusion and exclusion criteria identified 5
potentially relevant studies. 6 studies were excluded at this stage of the process because the
studies were based on ankle or foot fractures. The references in the bibliography of the 5
relevant studies were manually searched, but no further studies were found. The 5 studies
were by the following authors, Banal et al., (2009); Yesilaras et al., (2014); Kozaci et al.,
(2017); Ebrahimi et al., (2019) and Crombach et al., (2020).
2.4 Assessment of methodology quality
The Quality Assessment of Diagnostic Accuracy Studies-2 (QUADAS-2) tool was used to
assess the methodological quality and risk of bias (Whiting, 2011) (see table 4).
Table 4. Applicability Judgments and Risk of Bias in Quadas-2. Table obtained from Whiting, (2011).
9
Using the Quadas-2 tool the quality of every selected study was estimated by an appraisal of
the risk of bias of the 4 domains (Patient selection, Index test, Reference standard, as well as
Flow and timing). Additionally, the first three domains (Patient selection, Reference standard,
Index test) were assessed for clinical applicability of the study characteristic. To assist in
reaching a proper decision on the risk of bias, additional signalling questions were
incorporated. The signalling questions concentrates on those aspects of the study design that
have influence on the validity of the findings or results reported and conclusions drawn.11
signalling questions were selected across the 4 domains. The signally questions included
were answered as “no”, “yes”, or “unclear”. Both risk of bias and concerns regarding
applicability were rated as “low”, “high” or “unclear”. The “unclear” category was used
when the data reported was insufficient (see table 5).
Table 5. Quality assessment of the included studies using QUADAS-2 tool.
Study
Banal 2009
Yesilaras 2014
Ebrahimi 2019
Kozaci 2017
Crombach 2020
Risk of Bias
Applicability Concerns
Patient
selection
Index text
Reference
standard
Flow and
timing
Patient
selection
Index text
Reference
standard
□
□
■
□
□
□
□
□
□
□
□
□
?
?
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
□
■ High risk
□ Low risk
?
?
?
?
? Unclear risk
The QUADAS-2 tool was used because it is simple and relatively easy to complete. The tool
provides additional and improved features, that include distinguishing between applicability
and bias, identifying four key domains supported by signalling questions to assist better
decision on risk of bias. The tool also grades risk of bias and concerns about applicability
simply as “low” and “high” (Whiting, 2011).
The Quadas -2 tool does not contain a quality score which is necessary only when an overall
indicator of quality is needed to evaluate the meta-analysis. Methods of determining quality
scores are most often random, as such it is not always possible to produce a quality score that
is not bias (Juni, 2001).
A weakness with this method resides in the differentiation between the quality of a study and
how it was reported. Quality assessment relates highly to the reporting of results. If the
methods and results are not reported in adequate detail, a study that has been appropriately
10
conducted will not score well on a quality assessment (Whiting et al., 2003).
Another aspect to consider is the differentiation between bias and variability. Variability may
influence the external validity of study results while bias will restrict the validity. The tool
incorporates items which covers the quality of reporting, variability and bias. Most items in
the tool relate to bias with just two items each relating to variability and reporting (Whiting et
al., 2003).
2.5 Data Extraction
A predefined data extraction form with information fields based on the recommendations
made by the Cochrane Handbook for Systematic Reviews of Interventions (Shuster, 2011),
was used to collect relevant information from the studies that were selected (see table 6).
11
Table 6. Characteristics of the included studies.
Author
Year
Country
Study Type
Sample Size
Banal et al
2009
France
Prospective
Yesilaras
et al
Kozaci et
al
Ebrahimi
et al
2014
Turkey
Prospective
2017
Turkey
Prospective
2019
Iran
Prospective
Consecutive
37
Consecutive
84
Consecutive
72
Non-random
purposive
sampling 102
Crombach
et al
2020
Netherland
Prospective
242
Conservative
Probe
Frequency
(MHz)
7.5–13
Number
with
fractures
12
Fracture
Prevalence
(%)
32.4
Fracture
Site
Blinding
M/ F
Study Period
TP
FP
Metatarsal
Double blinded
9/28
11/2006-12/2007
10
6
7.5-10
34
40.4
Double blinded
36/48
11/2011-3/2013
33
0
7.5
28
38.8
The fifth
Metatarsal
Metatarsal
NR
48/24
5/2015-7/2016
26
5
10
31
30.3
Metatarsal
Double blinded
58/44
1/2016-9/2016
30
11
10
35
25
5thmetatarsal,
Distal fibula
and tibia
Single blinded
61/97
08/2015-12/2017
PPV
Expert - 96.7% ;95% CI 80.3 to 99.5%
Sonographer -70%;95% CI 57.0 to80.3%
FN
TN
Age (yr)
Reference
Standard
US Operator
Training
Study Setting
US Equipment
Sensitivity (%)
Specificity (%)
2
19
52.7±14.1
MRI
Experienced
rheumatologists
No
Rheumatology
department
An Esaote
Technos MP
system
83
1
50
36±15
X-ray
No
39
33±18
X-ray
Emergency
department
Emergency
department
Mindray M5
2
An emergency
physician
Emergency
physician
1
60
35.14 ±
14.32
X-ray
Emergency
medicine
specialist
>17yrs
X-ray
US Expert
/Sonographer
NPV
Expert - 95.3%; 95%CI
91.0 to 98.2%.
Sonographer - 94.1%
;95% CI 89.1 to 96.9%
12
Yes. 1 hr
theory 1hr
practical
NR
Yes.
12M
course
Bedside US
US Diagnostic Criteria
76
Time
Between Xray and US
Same day
NR
Stress
fracture
97.1
100
NR
Esaote
Firenze Italy
93
89
NR
Yes
Injury
Yes
Trauma
Hypoechoic periosteal
elevation, cortical
disruption, & increased
vascularity
Cortical disruption
Emergency
department
NR
96.7
84.5
NR
Yes
Emergency
department
Zonare Z One/
SonoSite
Xporte
Expert - 82.8
Sonographer - 80
Expert - 99.2
Sonographer90.3
NR
NR
Trauma
Cortical disruption
Presence of cortical
disruption or stepping or
axial deviation of the
bone surface
Cortical disruption or
axial deviation of the
bone surface
2.6 Data synthesis and analysis
Meta-DiSc 2.0 software (Zamora et al., 2006) was used to perform a meta-analysis of the
data. Meta-DiSc software was used because it is user friendly and it performs statistical
pooling of specificity, sensitivity, Diagnostic Odds Ratio (DOR), Likelihood Ratios (LRs),
using random effects models and fixed effects model. It also allows exploration of
heterogeneity such as I-squared, Chi-square and Cochran-Q test, and meta-regression.
Nevertheless, distinct grouping of specificities and sensitivities as well as the Littenberg &
Moses model have some built in statistical deficiencies, as the between-study variance isn’t
added in this tool (Wang and Leeflang, 2019).
In analysis, a bivarian random effects model was implemented because heterogeneity is
anticipated among studies (Riley, Higgins and Deeks, 2011). Factors such as differences in
participant characteristics (e.g., sex, age, ethnicity, number of subjects), types of radiographic
and US equipment’s used can cause significant statistical heterogeneity and inaccurate
summary estimates (Anker, Reinhart and Feeley, 2010). Therefore, these factors have to be
considered when conducting the meta-analysis, as such a random effect was applied.
The summary estimates of DOR, negative likelihood ratio (NLR), positive likelihood ratio
(PLR), specificity and sensitivity with their related confidence intervals (CIs) were
determined. These estimates were used to determine the accuracy of US in the diagnosis of
metatarsal fractures. A subgroup quantitative meta -analysis was conducted on the 4 studies
that included only metatarsal fractures (see table 7 and figure 2).
Table 7. Pooled diagnostic results of included studies.
Pooled Results
Confidence Intervals
Lower CI
Upper CI
Sensitivity
0.942
0.86 -
0.97
(95%)
I2 Statistic
0.32
0.16 -
0.48
(97.5%)
Specificity
0.904
0.74 -
0.96
(95%)
I Statistic
0.27
-0.04 -
0.58
(97.5%)
Diagnostic Odds Ratio
154.34
-122.42 -
431.10
(95%)
Positive Likelihood Ratio
9.83
- 0.811 -
20.47
(95%)
Negative Likelihood Ratio
0.063
0.003 -
0.12
(95%)
2
13
Figure 2. Forest plots of US diagnosing metatarsal fractures, depicting specificity and sensitivity.
Sensitivity
Specificity
a) Heterogenicity
Heterogeneity was evaluated using the I squared (I2) statistic. ‘’The I2 statistic describes the
percentage of all variation across studies because of heterogeneity rather than chance’’
(Higgins and Thompson, 2002). I2 was calculated from Cochran’s heterogeneity statistic. A
value of zero percent indicates no observed heterogeneity, while bigger values indicate
increasing heterogeneity. A value of > 50% was considered moderate heterogeneity and >
75% high heterogenicity (Higgins and Thompson, 2002).
The I2 was used because as a percentage it can be interpreted easily, and a CI can be attached
to it. I2 can be calculated from published meta-analysis and is simple to determine.
Furthermore, the number of studies in the meta- analysis does not affect the results obtained
(Huedo-Medina et al., 2006).
A limitation with this method is that the CI around I2 applied to evaluate the homogeneity
hypothesis in meta-analysis endure the issues of low power when the number of studies is
low (Huedo-Medina et al., 2006). Another limitation of I2 is that there are no experimentally
prepared cut-points to decide when heterogeneity is too much to do a meta-analysis (Israel
and Richter, 2011).
14
b) Sensitivity and Specificity
The DOR was an outcome measure used to determine test performance. The DOR was
calculated as the ratio of the odds of positivity in patients with metatarsal fractures relative to
the odds of positivity in patients without metatarsal fractures. The minimum value of DOR is
zero and maximum is infinity. DOR values which are larger are considered of having a better
discriminatory test performance (Kraemer, 1992). The pooled DOR consisting of a combined
summary estimate of specificity and sensitivity was used to establish the diagnostic test
accuracy. In order to ascertain whether the results might have been affected by one study, a
sensitivity analysis was conducted by removing studies sequentially from the combined
analysis (Anker, Reinhart and Feeley, 2010).
The DOR approach was used because it is simple and is in accordance with normal metaanalysis methods. Because results from various studies are combined into summary estimates
in meta-analysis of diagnostic studies, the use of DOR as a measure of test performance is
advantageous (Glass et al., 2003).
A limitation with this method is that it is not feasible to estimate the false positive and true
positive rates individually (Trikalinos et al., 2003). Because DOR is an overall measure, at
prevalence, it is not possible to assess a test error rates. The specificity and sensitivity can be
different for the same DOR, with different clinical consequence (Littenberg and Moses,
1993).
c) Likelihood ratios
Likelihood ratio was also used to measure diagnostic accuracy of US. The likelihood ratio
was calculated as the probability of patients with metatarsal fractures divided by the
probability of patients without metatarsal fractures (Fagem, 1975). Positive LR above 10 and
negative LR below 0.1 was considered as a better test performance in the diagnosis of
metatarsal fractures (Jaeschke, 1994).
The likelihood ratio approach was used because it can be used to combine the results of
several diagnostic tests. With this approach, the results of likelihoods ratios from various
clinical trials applying any technique (random or fixed effects) or both, can be combined to a
point estimate from diagnostic meta-analysis (Stengel et al., 2003). Furthermore, they also
have advantages over specificity and sensitivity because they are less likely to change with
the prevalence of the disease and can be calculated for several levels of the test or symptoms
(McGee, 2002).
A weakness with this method is the wide CIs around the LRs, especially those that are
capable of ruling in or ruling out a diagnosis. This is due to paucity of data at the extreme
15
points of the disease spectrum where the LRs are expected to be the most helpful.
Furthermore, as the LRs are calculated from the specificity and sensitivity, like these
parameters they may be affected by severity of disease (Moons et al., 1997).
d) Confidence Intervals
Confidence intervals of 95% and 97.5% were used for the summary estimates obtain in the
meta-analysis. ‘’The CI describes the uncertainty inherent in these estimates and describes a
range of values within which we can be reasonably sure that the true effect lies’’(Hespanhol
et al., 2019).
The CI was used as the statistical test because they constitute the information of a
significance test and are easier to understand. Regarding hypothesis testing, the results in CI
are given directly at the level of data measurement. This is an advantage that CIs have over pvalues. Another advantage is that information on the direction and strength of the effect of a
test as well as statistical significance are all included in CIs (Shakespeare et al., 2001).
A limitation with this approach is that the frequentist CI is commonly misinterpreted by
health care professionals and researchers (Sim and Reid, 1999). Regarding the 95% CI, a
frequent misreading is as follows: ‘’there is a 95% probability that the true (unknown) effect
estimate lies within the 95% CI’’. This translation is not precise; the fixed (unknown) value is
either outside or inside the interval with 0% (100%) probability because the population
parameter is treated as a fixed value (Pocock and Hughes, 1990).
2.7 Synopsis of the research proposal
The overall aim of this research proposal is to assess the relative effectiveness of US in the
detection of metatarsal fractures compared to the current gold standards for diagnosis (e.g.,
MRI, CT scan or X-ray), accessible in tertiary and secondary care centres.
3.0 Timetable
February 2021 – April 2021: Extensive literature review on diagnostic Ultrasonography,
Computer Tomography, Magnetic Resonance Imaging and X-rays.
April 2021 – May 2021: Writing of research proposal.
October 2022 – January 2023: Writing of research project.
16
References
Ait-Ali, L., Andreassi, M.G., Foffa, I., Spadoni, I., Vano, E. and Picano, E. (2009).
Cumulative patient effective dose and acute radiation-induced chromosomal DNA damage in
children with congenital heart disease. Heart, 96(4), pp.269–274.
Anker, A., Reinhart, A.M. and Feeley, T.H. (2010). Meta-Analysis of Meta-Analyses in
Communication: Comparing Fixed Effects and Random Effects Analysis
Models. Communication Quarterly, 58(3), pp.257–278.
Banal, F., Gandjbakhch, F., Foltz, V., Goldcher, A., Etchepare, F., Rozenberg, S., Koeger,
A.-C., Bourgeois, P. and Fautrel, B. (2009). Sensitivity and Specificity of Ultrasonography in
Early Diagnosis of Metatarsal Bone Stress Fractures: A Pilot Study of 37 Patients. The
Journal of Rheumatology, 36(8), pp.1715–1719.
Champagne, N., Eadie, L., Regan, L. and Wilson, P. (2019). The effectiveness of ultrasound
in the detection of fractures in adults with suspected upper or lower limb injury: a systematic
review and subgroup meta-analysis. BMC Emergency Medicine, 19(1).
Chartier, L.B., Bosco, L., Lapointe-Shaw, L. and Chenkin, J. (2016). Use of point-of-care
ultrasound in long bone fractures: a systematic review and meta-analysis. CJEM, 19(2),
pp.131–142.
Cochranelibrary.com. (2009). About the Cochrane Library | Cochrane Library. [online]
Available at: https://www.cochranelibrary.com/about/about-cochrane-library.
Crombach, A., Azizi, N., Lameijer, H., El Moumni, M. and ter Maaten, J.C. (2020). Point-ofcare bedside ultrasound examination for the exclusion of clinically significant ankle and fifth
metatarsal bone fractures; a single blinded prospective diagnostic cohort study. Journal of
Foot and Ankle Research, 13(1).
Crowley, S.G., Trofa, D.P., Vosseller, J.T., Gorroochurn, P., Redler, L.H., Schiu, B. and
Popkin, C.A. (2019). Epidemiology of Foot and Ankle Injuries in National Collegiate
Athletic Association Men’s and Women’s Ice Hockey. Orthopaedic Journal of Sports
Medicine, 7(8), p.232596711986590.
Douma-den Hamer, D., Blanker, M.H., Edens, M.A., Buijteweg, L.N., Boomsma, M.F., van
Helden, S.H. and Mauritz, G.-J. (2016). Ultrasound for Distal Forearm Fracture: A
Systematic Review and Diagnostic Meta-Analysis. PLOS ONE, 11(5), p.e0155659.
Ebrahimi M, Habibzadeh SR, Ahmadi SR, Khajeh Nasiri S, Kaveh MM, Foroughian M
(2019). Diagnostic Accuracy of Ultrasonography in Diagnosis of Metatarsal Bone Fracture; a
Cross Sectional Study. Arch Acad Emerg Med 2019;7:e49.
Elsevier (2009). Biomedical research – Embase | Elsevier. [online] Elsevier.com. Available
at: https://www.elsevier.com/solutions/embase-biomedical-research.
Fagan, T.J. Nomogram for Bayes’s Theorem (1975). New England Journal of Medicine,
293(5), pp.257–257.
17
Glas, A.S., Lijmer, J.G., Prins, M.H., Bonsel, G.J. and Bossuyt, P.M.M. (2003). The
diagnostic odds ratio: a single indicator of test performance. Journal of Clinical
Epidemiology, 56(11), pp.1129–1135.
Hespanhol, L., Vallio, C.S., Costa, L.M. and Saragiotto, B.T. (2019). Understanding and
interpreting confidence and credible intervals around effect estimates. Brazilian Journal of
Physical Therapy, [online] 23(4), pp.290–301.
Higgins, J.P.T. and Thompson, S.G. (2002). Quantifying heterogeneity in a metaanalysis. Statistics in medicine, 21(11), pp.1539–58.
Huedo-Medina, T.B., Sánchez-Meca, J., Marín-Martínez, F. and Botella, J. (2006). Assessing
heterogeneity in meta-analysis: Q statistic or I2 index?. Psychological Methods, 11(2),
pp.193–206.
Israel, H. and Richter, R.R. (2011). A Guide to Understanding Meta-analysis. Journal of
Orthopaedic & Sports Physical Therapy, 41(7), pp.496–504.
Jaeschke, R. (1994). Users’ guides to the medical literature. III. How to use an article about a
diagnostic test. A. Are the results of the study valid? Evidence-Based Medicine Working
Group. JAMA: The Journal of the American Medical Association, 271(5), pp.389–391.
Jeans, W.D. (1986). Book reviewsFuture Use of New Imaging Technologies in Developing
Countries. Technical Report Series No. 723. Report of a WHO Scientific Group, pp. 67, 1985
(World Health Organization, Geneva), Sw. Fr. 7.00. ISBN 92–4–120723–X. The British
Journal of Radiology, 59(701), pp.460–460.
Joshi, N., Lira, A., Mehta, N., Paladino, L. and Sinert, R. (2013). Diagnostic Accuracy of
History, Physical Examination, and Bedside Ultrasound for Diagnosis of Extremity Fractures
in the Emergency Department: A Systematic Review. Academic Emergency Medicine, 20(1),
pp.1–15.
Juni, P. (2001). Systematic reviews in health care: Assessing the quality of controlled clinical
trials. British Medical Journal, 323(7303), pp.42–46.
Kozaci, N., Ay, M.O., Avci, M., Beydilli, I., Turhan, S., Donertas, E. and Ararat, E. (2017).
The comparison of radiography and point-of-care ultrasonography in the diagnosis and
management of metatarsal fractures. Injury, 48(2), pp.542–547.
Kraemer, H.C. (1992). Risk ratios, odds ratio, and the test QROC. In: Evaluating medical
tests. SAGE Publications, Inc.; pp 103–113.
Lee, S.H. and Yun, S.J. (2019). Diagnostic Performance of Ultrasonography for Detection of
Pediatric Elbow Fracture: A Meta-analysis. Annals of Emergency Medicine, 74(4), pp.493–
502.
LiMarzi, G.M., Scherer, K.F., Richardson, M.L., Warden, D.R., Wasyliw, C.W., Porrino,
J.A., Pettis, C.R., Lewis, G., Mason, C.C. and Bancroft, L.W. (2016). CT and MR Imaging of
the Postoperative Ankle and Foot. RadioGraphics, 36(6), pp.1828–1848.
Littenberg, B. and Moses, L.E. (1993). Estimating Diagnostic Accuracy from Multiple
Conflicting Reports. Medical Decision Making, 13(4), pp.313–321.
18
McGee, S. (2002). Simplifying likelihood ratios. Journal of General Internal Medicine,
17(8), pp.647–650.
Mircea, P.-A., Badea, R., Fodor, D. and Buzoianu, A.D. (2012). Using ultrasonography as a
teaching support tool in undergraduate medical education - time to reach a decision. Medical
Ultrasonography, 14(3), pp.211–216.
Moons, K.G.M., van Es, G.-A., Deckers, J.W., Habbema, J.D.F. and Grobbee, D.E. (1997).
Limitations of Sensitivity, Specificity, Likelihood Ratio, and Bayesʼ Theorem in Assessing
Diagnostic Probabilities. Epidemiology, 8(1), pp.12–17.
Nery, C., Raduan, F. and Baumfeld, D. (2016). Foot and Ankle Injuries in Professional
Soccer Players. Foot and Ankle Clinics, 21(2), pp.391–403.
Nih.gov. (2019). About MEDLINE® and PubMed®: The Resources Guide. [online]
Available at: https://www.nlm.nih.gov/bsd/pmresources.html.
Pocock, S.J. and Hughes, M.D. (1990). Estimation issues in clinical trials and
overviews. Statistics in Medicine, 9(6), pp.657–671.
Riley, R.D., Higgins, J.P.T. and Deeks, J.J. (2011). Interpretation of random effects metaanalyses. British Medical Journal, 342, pp.d549–d549.
Saul, T. (2013). Point-of-care ultrasound in the diagnosis of upper extremity fracturedislocation. A pictorial essay. Medical Ultrasonography, 15(3), pp.230–236.
Sconfienza, L.M., Albano, D., Allen, G., Bazzocchi, A., Bignotti, B., Chianca, V., Facal de
Castro, F., Drakonaki, E.E., Gallardo, E., Gielen, J., Klauser, A.S., Martinoli, C., Mauri, G.,
McNally, E., Messina, C., Mirón Mombiela, R., Orlandi, D., Plagou, A., Posadzy, M. and de
la Puente, R. (2018). Clinical indications for musculoskeletal ultrasound updated in 2017 by
European Society of Musculoskeletal Radiology (ESSR) consensus. European Radiology,
28(12), pp.5338–5351.
Shakespeare, T.P., Gebski, V.J., Veness, M.J. and Simes, J. (2001). Improving interpretation
of clinical studies by use of confidence levels, clinical significance curves, and risk-benefit
contours. The Lancet, 357(9265), pp.1349–1353.
Shamseer, L., Moher, D., Clarke, M., Ghersi, D., Liberati, A., Petticrew, M., Shekelle, P. and
Stewart, L.A. (2015). Preferred reporting items for systematic review and meta-analysis
protocols (PRISMA-P) 2015: elaboration and explanation. British Medical Journal,
349(jan02 1), pp.g7647–g7647.
Shuster, J.J. (2011). Review: Cochrane handbook for systematic reviews for interventions,
Version 5.1.0, published 3/2011. Julian P.T. Higgins and Sally Green, Editors. Research
Synthesis Methods, 2(2), pp.126–130.
Sim, J., Reid, N. (1999). Statistical inference by confidence intervals: issues of interpretation
and utilization. Physical Therapy,79(2):186–195.
Stengel, D., Bauwens, K., Sehouli, J., Ekkernkamp, A. and Porzsolt, F. (2003). Original
Paper: A likelihood ratio approach to meta-analysis of diagnostic studies. Journal of Medical
Screening, 10(1), pp.47–51.
19
Trikalinos, T.A., Balion, C.M., Coleman, C.I., Griffith, L., Santaguida, P.L., Vandermeer, B.
and Fu, R. (2012). Chapter 8: Meta-analysis of Test Performance When There is a “Gold
Standard.” Journal of General Internal Medicine, 27(S1), pp.56–66.
Wang, C.-L., Shieh, J.-Y., Wang, T.-G. and Hsieh, F.-J. (1999). Sonographic detection of
occult fractures in the foot and ankle. Journal of Clinical Ultrasound, 27(8), pp.421–425.
Wang, J. and Leeflang, M. (2019). Recommended software/packages for meta-analysis of
diagnostic accuracy. Journal of Laboratory and Precision Medicine, 4, pp.22–22.
Weeks, B.K., Hirsch, R., Nogueira, R.C. and Beck, B.R. (2016). Is calcaneal broadband
ultrasound attenuation a valid index of dual-energy x-ray absorptiometry-derived bone mass
in children? Bone & Joint Research, 5(11), pp.538–543.
Whiting, P., Rutjes, A.W., Reitsma, J.B., Bossuyt, P.M. and Kleijnen, J. (2003). The
development of QUADAS: a tool for the quality assessment of studies of diagnostic accuracy
included in systematic reviews. BMC Medical Research Methodology, 3(1).
Whiting, P.F. (2011). QUADAS-2: A Revised Tool for the Quality Assessment of Diagnostic
Accuracy Studies. Annals of Internal Medicine, 155(8), p.529.
Wood, J.N., Henry, M.K., Berger, R.P., Lindberg, D.M., Anderst, J.D., Song, L., Localio, R.
and Feudtner, C. (2019). Use and Utility of Skeletal Surveys to Evaluate for Occult Fractures
in Young Injured Children. Academic Pediatrics, 19(4), pp.428–437.
Wu, J., Wang, Y. and Wang, Z. (2020). The diagnostic accuracy of ultrasound in the
detection of foot and ankle fractures: a systematic review and meta-analysis. Medical
Ultrasonography.
Yesilaras, M., Aksay, E., Atilla, O.D., Sever, M. and Kalenderer, O. (2014). The accuracy of
bedside ultrasonography as a diagnostic tool for the fifth metatarsal fractures. The American
Journal of Emergency Medicine, 32(2), pp.171–174.
Zamora, J., Abraira, V., Muriel, A., Khan, K. and Coomarasamy, A. (2006). Meta-DiSc: a
software for meta-analysis of test accuracy data. BMC Medical Research Methodology, 6(1).
Zhao, W., Wang, G., Chen, B., Xiao, J., Sun, X., Wu, T., Ren, H. and Li, X. (2019). The
value of ultrasound for detecting hand fractures. Medicine, 98(44), p.e17823.
20