Practical Research Methods for Diagnostic Radiographers RESEARCH PROPOSAL Research Question /Hypotheses Is Ultrasound effective in the diagnosis of metatarsal fractures as compared to X-ray, MRI or CT scan? 0 INDEX Content Page number Abstract 2 Introduction 3 Overview/Rationale 3 Evidence Based Practice and Question 4 development 1 Methods 5 Criteria for selecting studies 5 Database search strategy 7 Selection of studies 8 Assessment of methodology quality 9 Data extraction 11 Data synthesis and analysis 13 Synopsis of the research proposal 16 Timetable 16 References 17 ABSTRACT The aim of this research proposal is to assess the relative effectiveness of Ultrasound (US) in the detection of metatarsal fractures compared to the current gold standards for diagnosis (e.g., MRI, CT scan or X-ray), accessible in tertiary and secondary care centres. The proposal was performed according to the PRISMA guidelines and used an electronic database specific search strategy with Cochrane library, Medline and Embase. 1050 potential papers were obtained from the original search. The titles and abstracts of these papers were screened for eligible studies and 5 relevant studies were identified. The 5 studies were by the following authors, Banal et al., (2009); Yesilaras et al., (2014); Kozaci et al., (2017); Ebrahimi et al., (2019) and Crombach et al., (2020). Of the 5 eligible studies found, 4 studies included metatarsal fractures only and 1 study included fractures at the 5thmetatarsal, distal fibula and tibia. Diagnostic performance of ultrasound was evaluated with a quantitative sub-group meta-analysis on the 4 studies that included metatarsal fractures only using measures of specificity and sensitivity. 2 1.1 INTRODUCTION US is a non-invasive diagnostic method used to image inside the body. One of the most common uses of US is during pregnancy, to monitor the growth and development of the fetus. US has been used in the detection of paediatric fractures (weeks et al., 2016) and to detect occult adult fractures that might be missed with conventional radiographs (Wang et al., 1999). Injuries to the distal lower limb specifically the foot and ankle occurs as buckling or blunt trauma and generally lead to a sprain, strain and less often fractures (Crowley et al., 2019). Although such injuries are not fatal, early diagnosis and treatment are important for patients with foot fractures in order to prevent long-term complications (Wood et al.,2019). 1.2 Overview/Rationale The diagnosis of foot injuries is usually performed by the use of X-Ray radiography and this is considered as the standard reference (Nery, Raduan and Baumfeld, 2016). X-Ray radiography involves exposure to ionisation radiation with the possibility of causing teratogenic or carcinogenic effects (Ait-Ali et al., 2009). Other imaging modalities, e.g., Magnetic Resonance Imaging (MRI), Computed Tomography (CT), as well as US, have been considered to improve the diagnostic accuracy of foot fractures (LiMarzi et al., 2016). Nevertheless, selection of MRI as standard reference is considered to increase unnecessary time and expenses and CT examination might bring unnecessary ionisation radiation to patients (Ait-Ali et al., 2009). Recent findings have shown that US is a possible alternative for the diagnosis of bone fractures in the emergency department. A high accuracy in paediatric metacarpal, distal radius, elbow, ankle, phalanx and metatarsal fractures have been reported (Lee and Yun, 2019); (Douma-den Hamer et al., 2016); (Champagne et al., 2019); (Zhao et al., 2019). US have the advantage of not producing ionisation radiation. It is easily accessible in the emergency department and can be performed immediately. This help to reduce diagnostic delays. This imaging modality is easy to teach (Mircea et al., 2012) and does not involve the experience of pain (Saul, 2013). The method is repeatable, portable and there is the possibility of obtaining information on the musculoskeletal system (Sconfienza et al., 2018). Furthermore, US can be a valuable tool in remote and resource-poor settings which can help in point of care fracture diagnosis. This could enhance rapid and adequate management of fractures in the community. 3 1.3 Evidence Based Practice and Question development US usage in fracture detection has formerly been reviewed in lower extremity stress fractures (Jeans, 1986), in long bone fractures (Chartier et al., 2016), in upper and lower limb fractures (Champagne et al., 2019), in paediatric forearm fractures (Douma-den Hamer et al., 2016), in foot and ankle fractures (Wu, Wang and Wang, 2020) and in acute extremity fractures (Joshi et al., 2013). An important feature in these investigations is the actual diagnostic accuracy of US for detecting bone fractures. Therefore, this research proposal is a systematic review and meta-analysis synthetising the diagnostic performance of US specifically in fractures of the metatarsal bones. Statement of Problem/Purpose The aim of this research proposal is to assess the relative effectiveness of US in the detection of metatarsal fractures compared to the current gold standards for diagnosis (e.g., MRI, CT scan or X-ray). Research Question /Hypotheses Is US effective in the diagnosis of metatarsal fractures as compared to X-ray, MRI or CT scan? The Population Intervention Comparison Outcome (PICO) model was used to frame the clinical question forming the bases of this research proposal (Shamseer et al., 2015). This is illustrated in table 1. Table 1. PICO model P Study participants (patients). US operators I Point-of-care US (patient bedside, prehospital setting or in Emergency Department) Fractures clinically believed to be of the metatarsal bones C Gold standard diagnostic imaging (MRI, CT scan or X-ray) O Diagnostic accuracy (specificity, sensitivity) Population: The population here refers to patients because the studies are clinical in nature. Data on US operators and their qualifications and competence were also extracted because the proposal is based on the use of US as a diagnostic method. Intervention: The research proposal investigates point-of-care US as a diagnostic modality in a prehospital or hospital setting, for clinically suspected metatarsal fractures. 4 Comparison: The gold standard for comparison is X-Rays, CT scan or MRI and studies must be blinded for inclusion. Outcome: The diagnostic accuracy of US in the identification of fractures of the metatarsal bones, compared to gold standard imaging using measures of specificity and sensitivity. 2.0 METHODS The proposal was performed following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines (Moher et al., 2015). 2.1 Criteria for selecting studies To increase validity and reproducibility screening of appropriate studies was performed based on a prearranged set of inclusion and exclusion criteria. This included the parameters of the study question as itemised using the PICO framework in table 1. (see table 2). Table 2. Eligibility criteria Domain Inclusion Exclusion Study type • Randomised control trials •Case reports, guidelines, letters, •Prospective studies consensus statements, and unpublished •Studies involving patients with articles clinically suspected metatarsal •Studies that contained an overlapped fractures. population •Full texts published in a peer •Studies without sufficient data to review journal construct diagnostic 2x2 tables •Nonclinical studies •Literature review •Conference proceedings Participants • Humans •Non-human subjects • Adults •Simulated patients •Clinical setting •Simulated fractures • Blinded US •Non-blinded image interpretation • Operators •Non blinded US operators •Exclusively paediatric patients •Mixed adult and paediatric populations 5 Setting •Emergency Department •None •Hospital bedside •Prehospital setting Procedure •Diagnostic •Therapeutic ultrasonography •Ultrasonography •Diagnostic X-ray, MRI or CT scan believed to be metatarsal fracture clinically Outcome •Fracture identification •Fracture treatment •Fracture diagnosis •Bone density assessment Following an initial review of the literature, further searches that were performed focused on studies in adults using ultrasonography as the test method examining cases of fractures believed to be of the metatarsal bones. Because the proposal is on clinical practice, trials that did not include humans or simulated fractures were not selected for inclusion. In order to adequately study how accurate ultrasonography is, the distribution of participants ought to have been random or consecutive. Therefore, studies were chosen considering the methods used, to select only randomised controlled trials or prospective studies that involved adequate diagnostic imaging interpretation blinding. During the selection process, restriction was not made on the language or year that the study reports were published, but complete articles issued in a peer-review journal is what was needed for a study to be selected. Clinical studies which employed therapeutic US scans were not selected because the research proposal is based on ultrasonography as an imaging method for diagnostic objective. Furthermore, any study that noted any of the primary outcome mentioned in table one (PICO framework) were selected. Studies that did not report diagnostic accuracy of sonography in comparison with one of the reference standards were not selected for the proposal. Finally, secondary outcome measures such as patient management, user’s perspective and comparative time to diagnosis, were taken into account separately and did not affect study selection. 6 2.2 Database search strategy Medline, Cochrane library and Embase are electronic databases which were systematically searched. Medline provides more than twenty-six million references to life sciences and biomedical journal articles. It has a comprehensive journal selection process, and added value of using the Medical Subject Headings, NLM controlled vocabulary to index citations. However, this database is US bias. The indexing is poor, and it can be difficult to pinpoint the relevant references (Nih.gov. 2019). Embase is a biomedical database that encompasses international biomedical literature from 1947 to the present. The articles in this database are properly indexed using Elsevier's Life Science thesaurus. However, it is European bias and not so easy to search for non-medical content (Elsevier 2009). Cochrane library is a database of systematic reviews containing 4000 plus reviews on healthcare topics produced by 49 subject specific review groups. This database contains high quality research data, but it is very small compared to traditional databases and some subject areas for instance, allergy and intolerance are under-represented (Cochranelibrary.com. 2009). These databases were systematically searched to locate eligible studies from when they were published to April 2021. Searches were performed using the MeSH and keywords terms concerning each component of the research question. Table 3 depicts the search strategy used. Table 3. Medical Subject Heading and search terms PUBMED ((((ultrasonography[MeSH Terms]) OR (ultraso*[Title/Abstract])) OR (sonograph*[Title/Abstract])) AND (((foot[Title/Abstract]) OR (metatars*[Title/Abstract])) AND (((traum*[Title/Abstract]) OR (injur*[Title/Abstract])) OR (fracture*[Title/Abstract])) EMBASE #1AND#2AND#3 #1 (fracture*:ab,ti OR injur*:ab,ti OR traum*:ab,ti) #2 (sonograph*:ab,ti OR ultraso*:ab,ti) #3 (metatars:ab,ti OR foot*:ab,ti ) COCHRANE 7 #1AND#2AND#3 #1 (sonograph*):ti,ab,kw OR (ultraso*):ti,ab,kw #2 (fracture*):ti,ab,kw OR (injur*):ti,ab,kw OR (traum*):ti,ab,kw #3 (metatars):ti,ab,kw OR (foot*):ti,ab,kw 2.3 Selection of studies Fig. 1 PRISMA flowchart (Moher et al., 2015) of the search process. 8 All the studies from the original search conducted in all the databases with titles relevant to the proposal were assembled and RefWorks was used to remove all duplicate studies. 1050 potential papers were obtained from the original search. A list containing the titles and abstracts of the articles was compiled for screening.1039 articles were excluded as it was evident from the abstract or title that these studies were not appropriate to the proposal. Finally, screening of 11 full text articles against inclusion and exclusion criteria identified 5 potentially relevant studies. 6 studies were excluded at this stage of the process because the studies were based on ankle or foot fractures. The references in the bibliography of the 5 relevant studies were manually searched, but no further studies were found. The 5 studies were by the following authors, Banal et al., (2009); Yesilaras et al., (2014); Kozaci et al., (2017); Ebrahimi et al., (2019) and Crombach et al., (2020). 2.4 Assessment of methodology quality The Quality Assessment of Diagnostic Accuracy Studies-2 (QUADAS-2) tool was used to assess the methodological quality and risk of bias (Whiting, 2011) (see table 4). Table 4. Applicability Judgments and Risk of Bias in Quadas-2. Table obtained from Whiting, (2011). 9 Using the Quadas-2 tool the quality of every selected study was estimated by an appraisal of the risk of bias of the 4 domains (Patient selection, Index test, Reference standard, as well as Flow and timing). Additionally, the first three domains (Patient selection, Reference standard, Index test) were assessed for clinical applicability of the study characteristic. To assist in reaching a proper decision on the risk of bias, additional signalling questions were incorporated. The signalling questions concentrates on those aspects of the study design that have influence on the validity of the findings or results reported and conclusions drawn.11 signalling questions were selected across the 4 domains. The signally questions included were answered as “no”, “yes”, or “unclear”. Both risk of bias and concerns regarding applicability were rated as “low”, “high” or “unclear”. The “unclear” category was used when the data reported was insufficient (see table 5). Table 5. Quality assessment of the included studies using QUADAS-2 tool. Study Banal 2009 Yesilaras 2014 Ebrahimi 2019 Kozaci 2017 Crombach 2020 Risk of Bias Applicability Concerns Patient selection Index text Reference standard Flow and timing Patient selection Index text Reference standard □ □ ■ □ □ □ □ □ □ □ □ □ ? ? □ □ □ □ □ □ □ □ □ □ □ □ □ □ □ □ □ ■ High risk □ Low risk ? ? ? ? ? Unclear risk The QUADAS-2 tool was used because it is simple and relatively easy to complete. The tool provides additional and improved features, that include distinguishing between applicability and bias, identifying four key domains supported by signalling questions to assist better decision on risk of bias. The tool also grades risk of bias and concerns about applicability simply as “low” and “high” (Whiting, 2011). The Quadas -2 tool does not contain a quality score which is necessary only when an overall indicator of quality is needed to evaluate the meta-analysis. Methods of determining quality scores are most often random, as such it is not always possible to produce a quality score that is not bias (Juni, 2001). A weakness with this method resides in the differentiation between the quality of a study and how it was reported. Quality assessment relates highly to the reporting of results. If the methods and results are not reported in adequate detail, a study that has been appropriately 10 conducted will not score well on a quality assessment (Whiting et al., 2003). Another aspect to consider is the differentiation between bias and variability. Variability may influence the external validity of study results while bias will restrict the validity. The tool incorporates items which covers the quality of reporting, variability and bias. Most items in the tool relate to bias with just two items each relating to variability and reporting (Whiting et al., 2003). 2.5 Data Extraction A predefined data extraction form with information fields based on the recommendations made by the Cochrane Handbook for Systematic Reviews of Interventions (Shuster, 2011), was used to collect relevant information from the studies that were selected (see table 6). 11 Table 6. Characteristics of the included studies. Author Year Country Study Type Sample Size Banal et al 2009 France Prospective Yesilaras et al Kozaci et al Ebrahimi et al 2014 Turkey Prospective 2017 Turkey Prospective 2019 Iran Prospective Consecutive 37 Consecutive 84 Consecutive 72 Non-random purposive sampling 102 Crombach et al 2020 Netherland Prospective 242 Conservative Probe Frequency (MHz) 7.5–13 Number with fractures 12 Fracture Prevalence (%) 32.4 Fracture Site Blinding M/ F Study Period TP FP Metatarsal Double blinded 9/28 11/2006-12/2007 10 6 7.5-10 34 40.4 Double blinded 36/48 11/2011-3/2013 33 0 7.5 28 38.8 The fifth Metatarsal Metatarsal NR 48/24 5/2015-7/2016 26 5 10 31 30.3 Metatarsal Double blinded 58/44 1/2016-9/2016 30 11 10 35 25 5thmetatarsal, Distal fibula and tibia Single blinded 61/97 08/2015-12/2017 PPV Expert - 96.7% ;95% CI 80.3 to 99.5% Sonographer -70%;95% CI 57.0 to80.3% FN TN Age (yr) Reference Standard US Operator Training Study Setting US Equipment Sensitivity (%) Specificity (%) 2 19 52.7±14.1 MRI Experienced rheumatologists No Rheumatology department An Esaote Technos MP system 83 1 50 36±15 X-ray No 39 33±18 X-ray Emergency department Emergency department Mindray M5 2 An emergency physician Emergency physician 1 60 35.14 ± 14.32 X-ray Emergency medicine specialist >17yrs X-ray US Expert /Sonographer NPV Expert - 95.3%; 95%CI 91.0 to 98.2%. Sonographer - 94.1% ;95% CI 89.1 to 96.9% 12 Yes. 1 hr theory 1hr practical NR Yes. 12M course Bedside US US Diagnostic Criteria 76 Time Between Xray and US Same day NR Stress fracture 97.1 100 NR Esaote Firenze Italy 93 89 NR Yes Injury Yes Trauma Hypoechoic periosteal elevation, cortical disruption, & increased vascularity Cortical disruption Emergency department NR 96.7 84.5 NR Yes Emergency department Zonare Z One/ SonoSite Xporte Expert - 82.8 Sonographer - 80 Expert - 99.2 Sonographer90.3 NR NR Trauma Cortical disruption Presence of cortical disruption or stepping or axial deviation of the bone surface Cortical disruption or axial deviation of the bone surface 2.6 Data synthesis and analysis Meta-DiSc 2.0 software (Zamora et al., 2006) was used to perform a meta-analysis of the data. Meta-DiSc software was used because it is user friendly and it performs statistical pooling of specificity, sensitivity, Diagnostic Odds Ratio (DOR), Likelihood Ratios (LRs), using random effects models and fixed effects model. It also allows exploration of heterogeneity such as I-squared, Chi-square and Cochran-Q test, and meta-regression. Nevertheless, distinct grouping of specificities and sensitivities as well as the Littenberg & Moses model have some built in statistical deficiencies, as the between-study variance isn’t added in this tool (Wang and Leeflang, 2019). In analysis, a bivarian random effects model was implemented because heterogeneity is anticipated among studies (Riley, Higgins and Deeks, 2011). Factors such as differences in participant characteristics (e.g., sex, age, ethnicity, number of subjects), types of radiographic and US equipment’s used can cause significant statistical heterogeneity and inaccurate summary estimates (Anker, Reinhart and Feeley, 2010). Therefore, these factors have to be considered when conducting the meta-analysis, as such a random effect was applied. The summary estimates of DOR, negative likelihood ratio (NLR), positive likelihood ratio (PLR), specificity and sensitivity with their related confidence intervals (CIs) were determined. These estimates were used to determine the accuracy of US in the diagnosis of metatarsal fractures. A subgroup quantitative meta -analysis was conducted on the 4 studies that included only metatarsal fractures (see table 7 and figure 2). Table 7. Pooled diagnostic results of included studies. Pooled Results Confidence Intervals Lower CI Upper CI Sensitivity 0.942 0.86 - 0.97 (95%) I2 Statistic 0.32 0.16 - 0.48 (97.5%) Specificity 0.904 0.74 - 0.96 (95%) I Statistic 0.27 -0.04 - 0.58 (97.5%) Diagnostic Odds Ratio 154.34 -122.42 - 431.10 (95%) Positive Likelihood Ratio 9.83 - 0.811 - 20.47 (95%) Negative Likelihood Ratio 0.063 0.003 - 0.12 (95%) 2 13 Figure 2. Forest plots of US diagnosing metatarsal fractures, depicting specificity and sensitivity. Sensitivity Specificity a) Heterogenicity Heterogeneity was evaluated using the I squared (I2) statistic. ‘’The I2 statistic describes the percentage of all variation across studies because of heterogeneity rather than chance’’ (Higgins and Thompson, 2002). I2 was calculated from Cochran’s heterogeneity statistic. A value of zero percent indicates no observed heterogeneity, while bigger values indicate increasing heterogeneity. A value of > 50% was considered moderate heterogeneity and > 75% high heterogenicity (Higgins and Thompson, 2002). The I2 was used because as a percentage it can be interpreted easily, and a CI can be attached to it. I2 can be calculated from published meta-analysis and is simple to determine. Furthermore, the number of studies in the meta- analysis does not affect the results obtained (Huedo-Medina et al., 2006). A limitation with this method is that the CI around I2 applied to evaluate the homogeneity hypothesis in meta-analysis endure the issues of low power when the number of studies is low (Huedo-Medina et al., 2006). Another limitation of I2 is that there are no experimentally prepared cut-points to decide when heterogeneity is too much to do a meta-analysis (Israel and Richter, 2011). 14 b) Sensitivity and Specificity The DOR was an outcome measure used to determine test performance. The DOR was calculated as the ratio of the odds of positivity in patients with metatarsal fractures relative to the odds of positivity in patients without metatarsal fractures. The minimum value of DOR is zero and maximum is infinity. DOR values which are larger are considered of having a better discriminatory test performance (Kraemer, 1992). The pooled DOR consisting of a combined summary estimate of specificity and sensitivity was used to establish the diagnostic test accuracy. In order to ascertain whether the results might have been affected by one study, a sensitivity analysis was conducted by removing studies sequentially from the combined analysis (Anker, Reinhart and Feeley, 2010). The DOR approach was used because it is simple and is in accordance with normal metaanalysis methods. Because results from various studies are combined into summary estimates in meta-analysis of diagnostic studies, the use of DOR as a measure of test performance is advantageous (Glass et al., 2003). A limitation with this method is that it is not feasible to estimate the false positive and true positive rates individually (Trikalinos et al., 2003). Because DOR is an overall measure, at prevalence, it is not possible to assess a test error rates. The specificity and sensitivity can be different for the same DOR, with different clinical consequence (Littenberg and Moses, 1993). c) Likelihood ratios Likelihood ratio was also used to measure diagnostic accuracy of US. The likelihood ratio was calculated as the probability of patients with metatarsal fractures divided by the probability of patients without metatarsal fractures (Fagem, 1975). Positive LR above 10 and negative LR below 0.1 was considered as a better test performance in the diagnosis of metatarsal fractures (Jaeschke, 1994). The likelihood ratio approach was used because it can be used to combine the results of several diagnostic tests. With this approach, the results of likelihoods ratios from various clinical trials applying any technique (random or fixed effects) or both, can be combined to a point estimate from diagnostic meta-analysis (Stengel et al., 2003). Furthermore, they also have advantages over specificity and sensitivity because they are less likely to change with the prevalence of the disease and can be calculated for several levels of the test or symptoms (McGee, 2002). A weakness with this method is the wide CIs around the LRs, especially those that are capable of ruling in or ruling out a diagnosis. This is due to paucity of data at the extreme 15 points of the disease spectrum where the LRs are expected to be the most helpful. Furthermore, as the LRs are calculated from the specificity and sensitivity, like these parameters they may be affected by severity of disease (Moons et al., 1997). d) Confidence Intervals Confidence intervals of 95% and 97.5% were used for the summary estimates obtain in the meta-analysis. ‘’The CI describes the uncertainty inherent in these estimates and describes a range of values within which we can be reasonably sure that the true effect lies’’(Hespanhol et al., 2019). The CI was used as the statistical test because they constitute the information of a significance test and are easier to understand. Regarding hypothesis testing, the results in CI are given directly at the level of data measurement. This is an advantage that CIs have over pvalues. Another advantage is that information on the direction and strength of the effect of a test as well as statistical significance are all included in CIs (Shakespeare et al., 2001). A limitation with this approach is that the frequentist CI is commonly misinterpreted by health care professionals and researchers (Sim and Reid, 1999). Regarding the 95% CI, a frequent misreading is as follows: ‘’there is a 95% probability that the true (unknown) effect estimate lies within the 95% CI’’. This translation is not precise; the fixed (unknown) value is either outside or inside the interval with 0% (100%) probability because the population parameter is treated as a fixed value (Pocock and Hughes, 1990). 2.7 Synopsis of the research proposal The overall aim of this research proposal is to assess the relative effectiveness of US in the detection of metatarsal fractures compared to the current gold standards for diagnosis (e.g., MRI, CT scan or X-ray), accessible in tertiary and secondary care centres. 3.0 Timetable February 2021 – April 2021: Extensive literature review on diagnostic Ultrasonography, Computer Tomography, Magnetic Resonance Imaging and X-rays. April 2021 – May 2021: Writing of research proposal. October 2022 – January 2023: Writing of research project. 16 References Ait-Ali, L., Andreassi, M.G., Foffa, I., Spadoni, I., Vano, E. and Picano, E. (2009). Cumulative patient effective dose and acute radiation-induced chromosomal DNA damage in children with congenital heart disease. Heart, 96(4), pp.269–274. Anker, A., Reinhart, A.M. and Feeley, T.H. (2010). Meta-Analysis of Meta-Analyses in Communication: Comparing Fixed Effects and Random Effects Analysis Models. Communication Quarterly, 58(3), pp.257–278. Banal, F., Gandjbakhch, F., Foltz, V., Goldcher, A., Etchepare, F., Rozenberg, S., Koeger, A.-C., Bourgeois, P. and Fautrel, B. (2009). Sensitivity and Specificity of Ultrasonography in Early Diagnosis of Metatarsal Bone Stress Fractures: A Pilot Study of 37 Patients. The Journal of Rheumatology, 36(8), pp.1715–1719. Champagne, N., Eadie, L., Regan, L. and Wilson, P. (2019). The effectiveness of ultrasound in the detection of fractures in adults with suspected upper or lower limb injury: a systematic review and subgroup meta-analysis. BMC Emergency Medicine, 19(1). Chartier, L.B., Bosco, L., Lapointe-Shaw, L. and Chenkin, J. (2016). Use of point-of-care ultrasound in long bone fractures: a systematic review and meta-analysis. CJEM, 19(2), pp.131–142. Cochranelibrary.com. (2009). About the Cochrane Library | Cochrane Library. [online] Available at: https://www.cochranelibrary.com/about/about-cochrane-library. Crombach, A., Azizi, N., Lameijer, H., El Moumni, M. and ter Maaten, J.C. (2020). Point-ofcare bedside ultrasound examination for the exclusion of clinically significant ankle and fifth metatarsal bone fractures; a single blinded prospective diagnostic cohort study. Journal of Foot and Ankle Research, 13(1). Crowley, S.G., Trofa, D.P., Vosseller, J.T., Gorroochurn, P., Redler, L.H., Schiu, B. and Popkin, C.A. (2019). Epidemiology of Foot and Ankle Injuries in National Collegiate Athletic Association Men’s and Women’s Ice Hockey. Orthopaedic Journal of Sports Medicine, 7(8), p.232596711986590. Douma-den Hamer, D., Blanker, M.H., Edens, M.A., Buijteweg, L.N., Boomsma, M.F., van Helden, S.H. and Mauritz, G.-J. (2016). Ultrasound for Distal Forearm Fracture: A Systematic Review and Diagnostic Meta-Analysis. PLOS ONE, 11(5), p.e0155659. Ebrahimi M, Habibzadeh SR, Ahmadi SR, Khajeh Nasiri S, Kaveh MM, Foroughian M (2019). Diagnostic Accuracy of Ultrasonography in Diagnosis of Metatarsal Bone Fracture; a Cross Sectional Study. Arch Acad Emerg Med 2019;7:e49. Elsevier (2009). Biomedical research – Embase | Elsevier. [online] Elsevier.com. Available at: https://www.elsevier.com/solutions/embase-biomedical-research. Fagan, T.J. Nomogram for Bayes’s Theorem (1975). New England Journal of Medicine, 293(5), pp.257–257. 17 Glas, A.S., Lijmer, J.G., Prins, M.H., Bonsel, G.J. and Bossuyt, P.M.M. (2003). The diagnostic odds ratio: a single indicator of test performance. Journal of Clinical Epidemiology, 56(11), pp.1129–1135. Hespanhol, L., Vallio, C.S., Costa, L.M. and Saragiotto, B.T. (2019). Understanding and interpreting confidence and credible intervals around effect estimates. Brazilian Journal of Physical Therapy, [online] 23(4), pp.290–301. Higgins, J.P.T. and Thompson, S.G. (2002). Quantifying heterogeneity in a metaanalysis. Statistics in medicine, 21(11), pp.1539–58. Huedo-Medina, T.B., Sánchez-Meca, J., Marín-Martínez, F. and Botella, J. (2006). Assessing heterogeneity in meta-analysis: Q statistic or I2 index?. Psychological Methods, 11(2), pp.193–206. Israel, H. and Richter, R.R. (2011). A Guide to Understanding Meta-analysis. Journal of Orthopaedic & Sports Physical Therapy, 41(7), pp.496–504. Jaeschke, R. (1994). Users’ guides to the medical literature. III. How to use an article about a diagnostic test. A. Are the results of the study valid? Evidence-Based Medicine Working Group. JAMA: The Journal of the American Medical Association, 271(5), pp.389–391. Jeans, W.D. (1986). Book reviewsFuture Use of New Imaging Technologies in Developing Countries. Technical Report Series No. 723. Report of a WHO Scientific Group, pp. 67, 1985 (World Health Organization, Geneva), Sw. Fr. 7.00. ISBN 92–4–120723–X. The British Journal of Radiology, 59(701), pp.460–460. Joshi, N., Lira, A., Mehta, N., Paladino, L. and Sinert, R. (2013). Diagnostic Accuracy of History, Physical Examination, and Bedside Ultrasound for Diagnosis of Extremity Fractures in the Emergency Department: A Systematic Review. Academic Emergency Medicine, 20(1), pp.1–15. Juni, P. (2001). Systematic reviews in health care: Assessing the quality of controlled clinical trials. British Medical Journal, 323(7303), pp.42–46. Kozaci, N., Ay, M.O., Avci, M., Beydilli, I., Turhan, S., Donertas, E. and Ararat, E. (2017). The comparison of radiography and point-of-care ultrasonography in the diagnosis and management of metatarsal fractures. Injury, 48(2), pp.542–547. Kraemer, H.C. (1992). Risk ratios, odds ratio, and the test QROC. In: Evaluating medical tests. SAGE Publications, Inc.; pp 103–113. Lee, S.H. and Yun, S.J. (2019). Diagnostic Performance of Ultrasonography for Detection of Pediatric Elbow Fracture: A Meta-analysis. Annals of Emergency Medicine, 74(4), pp.493– 502. LiMarzi, G.M., Scherer, K.F., Richardson, M.L., Warden, D.R., Wasyliw, C.W., Porrino, J.A., Pettis, C.R., Lewis, G., Mason, C.C. and Bancroft, L.W. (2016). CT and MR Imaging of the Postoperative Ankle and Foot. RadioGraphics, 36(6), pp.1828–1848. Littenberg, B. and Moses, L.E. (1993). Estimating Diagnostic Accuracy from Multiple Conflicting Reports. Medical Decision Making, 13(4), pp.313–321. 18 McGee, S. (2002). Simplifying likelihood ratios. Journal of General Internal Medicine, 17(8), pp.647–650. Mircea, P.-A., Badea, R., Fodor, D. and Buzoianu, A.D. (2012). Using ultrasonography as a teaching support tool in undergraduate medical education - time to reach a decision. Medical Ultrasonography, 14(3), pp.211–216. Moons, K.G.M., van Es, G.-A., Deckers, J.W., Habbema, J.D.F. and Grobbee, D.E. (1997). Limitations of Sensitivity, Specificity, Likelihood Ratio, and Bayesʼ Theorem in Assessing Diagnostic Probabilities. Epidemiology, 8(1), pp.12–17. Nery, C., Raduan, F. and Baumfeld, D. (2016). Foot and Ankle Injuries in Professional Soccer Players. Foot and Ankle Clinics, 21(2), pp.391–403. Nih.gov. (2019). About MEDLINE® and PubMed®: The Resources Guide. [online] Available at: https://www.nlm.nih.gov/bsd/pmresources.html. Pocock, S.J. and Hughes, M.D. (1990). Estimation issues in clinical trials and overviews. Statistics in Medicine, 9(6), pp.657–671. Riley, R.D., Higgins, J.P.T. and Deeks, J.J. (2011). Interpretation of random effects metaanalyses. British Medical Journal, 342, pp.d549–d549. Saul, T. (2013). Point-of-care ultrasound in the diagnosis of upper extremity fracturedislocation. A pictorial essay. Medical Ultrasonography, 15(3), pp.230–236. Sconfienza, L.M., Albano, D., Allen, G., Bazzocchi, A., Bignotti, B., Chianca, V., Facal de Castro, F., Drakonaki, E.E., Gallardo, E., Gielen, J., Klauser, A.S., Martinoli, C., Mauri, G., McNally, E., Messina, C., Mirón Mombiela, R., Orlandi, D., Plagou, A., Posadzy, M. and de la Puente, R. (2018). Clinical indications for musculoskeletal ultrasound updated in 2017 by European Society of Musculoskeletal Radiology (ESSR) consensus. European Radiology, 28(12), pp.5338–5351. Shakespeare, T.P., Gebski, V.J., Veness, M.J. and Simes, J. (2001). Improving interpretation of clinical studies by use of confidence levels, clinical significance curves, and risk-benefit contours. The Lancet, 357(9265), pp.1349–1353. Shamseer, L., Moher, D., Clarke, M., Ghersi, D., Liberati, A., Petticrew, M., Shekelle, P. and Stewart, L.A. (2015). Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015: elaboration and explanation. British Medical Journal, 349(jan02 1), pp.g7647–g7647. Shuster, J.J. (2011). Review: Cochrane handbook for systematic reviews for interventions, Version 5.1.0, published 3/2011. Julian P.T. Higgins and Sally Green, Editors. Research Synthesis Methods, 2(2), pp.126–130. Sim, J., Reid, N. (1999). Statistical inference by confidence intervals: issues of interpretation and utilization. Physical Therapy,79(2):186–195. Stengel, D., Bauwens, K., Sehouli, J., Ekkernkamp, A. and Porzsolt, F. (2003). Original Paper: A likelihood ratio approach to meta-analysis of diagnostic studies. Journal of Medical Screening, 10(1), pp.47–51. 19 Trikalinos, T.A., Balion, C.M., Coleman, C.I., Griffith, L., Santaguida, P.L., Vandermeer, B. and Fu, R. (2012). Chapter 8: Meta-analysis of Test Performance When There is a “Gold Standard.” Journal of General Internal Medicine, 27(S1), pp.56–66. Wang, C.-L., Shieh, J.-Y., Wang, T.-G. and Hsieh, F.-J. (1999). Sonographic detection of occult fractures in the foot and ankle. Journal of Clinical Ultrasound, 27(8), pp.421–425. Wang, J. and Leeflang, M. (2019). Recommended software/packages for meta-analysis of diagnostic accuracy. Journal of Laboratory and Precision Medicine, 4, pp.22–22. Weeks, B.K., Hirsch, R., Nogueira, R.C. and Beck, B.R. (2016). Is calcaneal broadband ultrasound attenuation a valid index of dual-energy x-ray absorptiometry-derived bone mass in children? Bone & Joint Research, 5(11), pp.538–543. Whiting, P., Rutjes, A.W., Reitsma, J.B., Bossuyt, P.M. and Kleijnen, J. (2003). The development of QUADAS: a tool for the quality assessment of studies of diagnostic accuracy included in systematic reviews. BMC Medical Research Methodology, 3(1). Whiting, P.F. (2011). QUADAS-2: A Revised Tool for the Quality Assessment of Diagnostic Accuracy Studies. Annals of Internal Medicine, 155(8), p.529. Wood, J.N., Henry, M.K., Berger, R.P., Lindberg, D.M., Anderst, J.D., Song, L., Localio, R. and Feudtner, C. (2019). Use and Utility of Skeletal Surveys to Evaluate for Occult Fractures in Young Injured Children. Academic Pediatrics, 19(4), pp.428–437. Wu, J., Wang, Y. and Wang, Z. (2020). The diagnostic accuracy of ultrasound in the detection of foot and ankle fractures: a systematic review and meta-analysis. Medical Ultrasonography. Yesilaras, M., Aksay, E., Atilla, O.D., Sever, M. and Kalenderer, O. (2014). The accuracy of bedside ultrasonography as a diagnostic tool for the fifth metatarsal fractures. The American Journal of Emergency Medicine, 32(2), pp.171–174. Zamora, J., Abraira, V., Muriel, A., Khan, K. and Coomarasamy, A. (2006). Meta-DiSc: a software for meta-analysis of test accuracy data. BMC Medical Research Methodology, 6(1). Zhao, W., Wang, G., Chen, B., Xiao, J., Sun, X., Wu, T., Ren, H. and Li, X. (2019). The value of ultrasound for detecting hand fractures. Medicine, 98(44), p.e17823. 20