Osmond, Martin Henry
$145,845
Research Proposal:
A STUDY TO DERIVE AND VALIDATE A CLINICAL DECISION RULE FOR THE USE
OF COMPUTED TOMOGRAPHY IN CHILDREN WITH MINOR HEAD INJURY
1.GOALS
This study will derive and validate a clinical decision rule for the use of computed
tomography (CT) in children with minor head injury.
1.1 Phase I: Derivation of the Rule
The goal of phase I is to derive a clinical decision rule, from variables in the history and
physical examination, that is highly sensitive for detecting acute brain injury in children with minor
head injury.
1.2 Phase II: Validation of the Rule
The goal of phase II is to prospectively assess the accuracy, reliability, and acceptability of
the decision rule in a new set of children with minor head injury. This will determine the clinical
utility of the rule and is essential if such a rule is to be widely adopted into clinical practice.
2. BACKGROUND
2.1 Statement of the Problem
2.1.1 Minor Head Injuries. Head injuries are among the most common types of trauma seen
in North American emergency departments. In the United States, head trauma in children accounts
for approximately 600,000 visits to the emergency department and 250,000 hospital admissions each
year.1,2 Between 60 and 80% of these admissions are classified as minor head injuries.3 A “minimal”
head injury (sometimes referred to as trivial head injury) is defined as a head injury in which there is
no loss of consciousness or amnesia and the patient has a completely normal physical exam.4 A
“minor” head injury (alternatively referred to as a mild head injury) is defined as a head injury, in
which there is a history of loss of consciousness or amnesia, as well as a Glasgow Coma Scale (GCS)
score ranging from 13 to 15. Appendix 15
While most cases of minor head injury can be discharged without sequelae after a period of
observation, a small proportion deteriorate and require neurosurgical intervention for an intracranial
hematoma.6-9 Emergency physicians and neurosurgeons alike are aware of patients who appear well
but later succumb to an intracranial hemorrhage (“talk and deteriorate”).10-17 The key to a good
outcome is the early diagnosis of intracranial hematoma followed by early surgery.18 In the 1970’s it
became apparent that excess mortality and delayed diagnosis could be reduced in head injury by early
use of CT.6,7,9 At the same time, plain skull radiography was discouraged because of its very low
yield in minor head injury and because of the greater yield of CT in moderate and severe head
injury.19,20
2.1.2 Computed Tomography for Minor Head Injury. In recent years the use of CT for
minor head injury has become increasingly common, especially in North America. Exact data for
Canadian emergency department use of CT in children are not available. In 1992, an estimated
270,000 CTs of the head were performed in U.S. emergency departments for head injury.21 Typical
U.S. charges for unenhanced CT range from 500 to 800 U.S. dollars, suggesting a national total cost
of 135 to 216 million U.S. dollars.
The yield of CT for intracranial lesions if applied to all children with minor head injury has
been estimated to be from 0% to 7%.1,22-24 In other words depending on the setting, 93%-100% of CT
scans performed for children with minor head injury would be expected to be normal and therefore to
not alter management. Obviously, selective use of this low-yield and expensive investigation in
children with minor head injury is desirable in order to minimize health care costs.
There is, however, considerable disagreement in the literature as to the indications for CT in
the large number of head trauma cases classified as “minor”. One group, primarily represented by
5a
Osmond, Martin Henry
$145,845
neurosurgeons, feels that CT is recommended for all patients with minor head injury. Stein, a U.S.
neurosurgeon, has recently written “We recommend routine and immediate cranial CT scanning of all
head injury patients who have lost consciousness or are amnesic, even if all other physical findings
are normal”.25 Likewise, the most recent Advanced Trauma Life Support (ATLS) guidelines
recommend that “ideally” all children with minor head injury should have a CT of the head.26
Another group, comprised of pediatricians, emergency physicians, and family physicians,
endorse a more selective use of CT in minor head injury. A recent statement (December, 1999) from
the American Academy of Pediatrics (AAP) Subcommittee on Management of Minor Head Injury
reported that either “observation in the office, clinic, emergency department, hospital, or home under
the care of a competent caregiver…” or “Cranial CT scanning along with observation” may be used
in the initial evaluation and management of children with minor head injury with brief loss of
consciousness”.27 However, this comprehensive report was unable to state which children with minor
head injury are at risk for an intracranial hematoma and should undergo CT of the head. The
Subcommittee stated that further research is required to “address the independent prognostic value of
other signs and symptoms for which the clinical significance in children is uncertain”.
The views of many physicians could be summed up by Hoffman who stated in an editorial in
1994 “I do not believe the residual risk in such patients (children with minor head injury) justifies the
cost of dollars, time, and resources of universal CT scanning or universal admission. However, until
large carefully designed prospective studies give us better data, this remains only an educated
guess.”28
Without the support of widely accepted guidelines, North American physicians are likely to
follow the conservative approach of ordering CT scans for most children seen in emergency
departments with minor head injury. This approach is likely due to the nature of emergency
department practice: high case volumes, brief physician-patient contact, uncertain follow-up, and fear
of medicolegal repercussions.29-31 There is a clear need for valid and reliable guidelines to allow
physicians to be more selective in their use of CT without compromising care of children with minor
head injury.
2.2 Methodologic Standards for Decision Rules
Recently there has been considerable interest in the area of clinical prediction or decision
rules, which attempt to reduce the uncertainty of medical decision making by standardizing the
collection and interpretation of clinical data.32-35 Concomitant with the reporting of various decision
rules has been an interest in the methodological standards for their development and validation.36-37
These standards may be summarized as follows: a) The outcome or diagnosis to be predicted must
be clearly defined and the assessment of this outcome should be made in a blinded fashion. b) The
clinical findings to be used as predictors must be clearly defined and standardized and their
assessment must be done without knowledge of the outcome. c) The reliability or reproducibility of
the clinical findings used as predictors must be demonstrated. d) The subjects in the study should be
selected without bias and should represent a wide spectrum of clinical and demographic
characteristics to increase the generalizability of the results. e) The mathematical techniques for
deriving the rules must be identified. f) Clinical decision rules should be sensible: have a clear
purpose, be relevant, demonstrate content validity, be concise, and be easy to use in the intended
clinical application. g) The accuracy of the decision rule in classifying patients with (sensitivity) and
without (specificity) the targeted outcome should be demonstrated. h) Prospective validation on a
new set of patients is an essential test of accuracy because misclassification is commonly higher
when decision rules are tested on a population other than the original derivation set. i)
Implementation to demonstrate the true effect on patient care is the ultimate test of a decision rule;
transportability can be tested at this stage.
2.3 Previous Studies
5b
Osmond, Martin Henry
$145,845
We conducted a systematic search for the existing evidence by performing a MEDLINE
search from 1966 to the present with the search strategy contained in Appendix 2. This search
generated 1112 references that were reviewed by two assessors. One hundred and forty one studies
were deemed as potentially relevant and the manuscripts were retrieved. They were assessed for the
relevance by one assessor using the following criteria: 1) children and adolescents with head injury,
2) predictor variables (e.g. clinical signs and symptoms, skull x-ray) and 3) outcome measure: CT
scan of the head. Twenty-eight articles were deemed as relevant for inclusion.
Of these 28 studies, 11 were case series of head injured patients and as such were limited in
their ability to provide any type of clinical prediction rule.11-17,38-41 Eleven studies were retrospective
cohort studies, which while helping to develop the list of predictor variables for inclusion in this
study, are too prone to bias to formulate a decision rule.3,42-51 The remaining 6 studies were
prospective cohort studies but none met the methodological standards for development of a decision
rule according to the criteria described in section 2.2.1,2,4,52-54 The best study we identified was by
Quayle and colleagues, who enrolled a prospective cohort of 322 children with non-trivial head
injury, of whom 27 (8%) developed an intracranial injury.4 They found that an intracranial injury was
associated with a skull fracture, signs of a basilar skull fracture, loss of consciousness for more than 5
minutes, altered mental status and focal neurologic abnormality. However, merely testing variables
for their significant association with intracranial injury is of limited usefulness for the clinician when
he/she is trying to decide whether or not to CT scan an individual patient. With a small sample size of
only 27 positive cases, the confidence intervals for sensitivity estimate, not available from the study,
would be very wide and hence imprecise. Two other prospective cohort studies dealt only with the
use of skull x-ray as a predictor variable. While it is a risk factor for intracranial injury, skull x-ray is
not a reliable predictor variable for use in screening minor head injury patients for the need to
perform CT scan of the head.53-55 One study dealt with the EEG for predicting positive findings on
CT scan, however, this is rarely available in emergency departments.52 Dietrich and colleagues
concluded based on prospective enrollment of 322 patients that no clinical symptom could be
predictive of a positive CT scan and concluded that all patients with minor head injury should receive
a CT scan.1 Ramudo and colleagues also concluded that no single clinical feature could predict the
need for a CT of the head but did not attempt to group symptoms together to develop a decision rule.2
The most highly predictive signs were focal motor deficit and pupillary asymmetry.2
Two studies have recently developed decision rules for CT scan in patients with minor head
injury. Haydel et al56 enrolled all patients with minor head injury > 2 years old (mean age = 36 years ,
range 3 to 97). They derived a decision rule from 560 patients with 36 positive cases. All patients
with a CT scan had one or more of 7 findings: a) headache, b) vomiting, c) age > 60 years, d) drug or
alcohol intoxication, e) deficits short-term memory, f) physical evidence of trauma above the
clavicles, or g) seizure. On a validation sample of 909 patients this rule had 100% sensitivity (95%
Confidence Interval: 95% to 100%). However, the rule dictated in this sample that 77% of all patients
required a CT. This would result in an increase in present CT scan rates in Canada. There are
many problems with this study: 1) it was under-powered and derived a rule on too few positive cases
(36 cases), 2) it did not attempt to assess the reliability of the predictor variables by having
interobserver reliability measured, 3) there was no follow-up after discharge to ensure that late
intracranial hematoms did not occur, and 4) the rule is not clinically sensible for the pediatric
population as it would require all patients with a bruise on the head or one episode of vomiting to
undergo a CT.
Stiell et al57 have recently derived a highly sensitive decision rule for CT in adults with minor
head injury. They found that high risk factors in adults were: a) failure to reach GCS of 15 within 2
hours of injury, b) suspected depressed skull fracture, or c) repeated vomiting. Medium risk factors
5c
Osmond, Martin Henry
$145,845
were: d) signs of basilar skull fracture, e) age ≥ 55, f) high-risk mechanism, or g) amnesia before
injury ≥ 30 minutes. This rule did not assess children and therefore cannot be applied to this
population. The investigators are currently in the validation phase of the project. Risk factors from
the adult CT decision rules will be specifically evaluated by this proposal in children.
In summary, the above pediatric head injury studies have a great variability in design and
none could be considered methodologically robust according to the criteria described in section 2.2.
There exists a great need to develop, validate and implement a decision rule in children and
adolescents with minor head injury, based on rigorous methodological standards.
2.4 Preparatory work by applicants
Extensive background work has gone into preparing for this study to ensure that it will be both
feasible and successful.
2.4.1 Research formulation workshop. A research formulation workshop, involving study
investigators from across Canada, was held in Ottawa, Ontario in April 1997. This workshop,
consisting of experts in pediatric emergency medicine, pediatric radiology and research methodology,
discussed important methodological issues that were key in developing this proposal. The participants
of this one-day conference reviewed the preliminary data from the Phase 0 study (see section 2.4.2),
the survey of emergency pediatricians in Canada (see section 2.4.3), and the relevant literature. At
this workshop many of the study components for this proposal were developed including
methodological standards, patient selection criteria, outcome measures, and data collection methods.
Importantly, the clinical predictor variables were derived at this time from the pediatric head injury
literature and from consensus within the group.
2.4.2 A Canadian Study of Utilization of CT Scanning in Pediatric Minor Head InjuryPhase 0. The applicants and collaborators have recently published an intensive review of 12 months’
consecutive data for the emergency departments of the nine collaborating centers.58 Using
standardized search techniques, research assistants determined the number of eligible patients with
minor head injury (according to the inclusion criteria of section 4.1.1), the referral fraction for CT,
the prevalence of acute brain injury (as defined in section 4.3.1), and the yield of CT. Detailed results
of this study are contained in the Tables and Figure of Appendix 3. During the 12 month study period
(January 1 – December 31, 1995) these nine collaborating hospitals saw 1,164 children with minor
head injury (Table 1). The prevalence of any acute brain injury was low (60 of 1,164 or 5.2%) as was
the rate of neurosurgical intervention (2 of 1,164 or 0.2%). Furthermore, 80.2% of patients were
discharged directly from the emergency department. Overall, 171 of 1,164 (15%) were referred for
CT, and 65% of CT scans were negative for any acute brain injury. Significant variation was found
between centers for CT referral (more than fourfold from a low of 6% to a high of 26%), and for the
portion of negative CT scans (50% to 89%).
2.4.3 Survey of Pediatric Emergency Physicians. We also performed a mail survey of all
full and part time physicians working in Pediatric Emergency Departments across Canada (Appendix
4). Eighty percent (98 of 123) of eligible physicians responded to our survey. All physicians had
access to a CT scanner and the vast majority (91.8%) could order a CT scan without neurological
consultation. Overall, the respondents expressed very positive attitudes towards the development of a
clinical decision rule for minor head injury with 73.2% saying they would use a decision rule, 23.7%
saying they might use a decision rule, and only 3.1% saying they would not consider using such a
rule. In addition, 86.5% responded that they would not feel comfortable using such a decision rule if
it had been developed in adult patients. The mean target sensitivity of the respondents was 94.9%
(range 90 to 100%) for detecting an acute lesion on CT scan and 97.5% (90 to 100%) for detecting a
lesion requiring neurosurgical intervention.
5d
Osmond, Martin Henry
$145,845
2.4.4 Pilot Study to Derive a CT head Decision Rule in Children. The applicants and 3
collaborating centres undertook a pilot study of this proposal for a one year period (June 1999 to June
2000). Data was collected prospectively on children 0 to 16 years of age with a minor head injury
from pediatric tertiary care emergency departments in Ottawa, Montreal, Calgary and Winnipeg.
Physicians filled in data sheets of selected predictor variables from history and physical examination
prior to viewing the head CT scan (if ordered). Detailed results of this study are contained in the
Tables of Appendix 5. In total, 291 of 417 eligible patients with minor head injury were enrolled
(recruitment rate = 70%). Patient characteristics are shown in Table 1. The rate of acute brain injury
(as defined in section 4.3.1) was 5% (19 of 417) in the eligible population and 3% (8 of 291) in the
enrolled group (Table 2). The group of patients not enrolled were significantly more likely to arrive
by ambulance (p<0.01), have brain injury (p<0.01) and be admitted to hospital (p<0.001) (Table 3).
The small number of acute brain injury cases (8) in the enrolled group was too small to allow
calculation of risk factors for this outcome, however, “worsening of headache”, “irritability”, and
“repeated vomiting” were all significantly associated with the likelihood of getting a CT scan (Table
4). The rate of CT scan ordering for minor head injury had more than doubled in the last 5
years (from 15% in 1995 to 39% in the past year). We again demonstrated a marked variation
across centers in use of CT in similar patients (from a low of 14% to a high of 56%). Only 7% of CT
scans done for minor head injury were positive (down from 35% in 1995). Epidural hematoma
was present in 0.7% (2 of 291). Seven patients were independently assessed by two ED physicians.
The Kappa coefficient for “loss of consciousness” was 0.67 and for “repeated vomiting” was 1.0.
Although they were of interest to us, we could not estimate the degree of agreement for “irritability”,
“worsening headache” and “GCS” due to small number of cases. Telephone follow-up was 90% and
93% in patients with and without CT scan, respectively (Table 5). Three patients were recalled based
on failing the “proxy primary outcome” measure. Of these, two patients were sent for CT, both of
which were normal. The third patient was judged to be clinically normal on reassessment and was not
sent for CT scan.
Much has been learned from this pilot study. We have confirmed the rising rate of CT scan
referral in Canada and the marked variation between centers. We have proven our ability to recruit
70% of eligible patients and follow-up over 90% of those enrolled. We aim to improve recruitment in
the proposed study by focusing on capturing the more acute patients who were not enrolled in the
pilot study. The low rate of acute brain injury emphasizes the need for a large trial in order to assess
the strength of association between each variable and the primary outcome. Finally, the pilot study
served as a “run-in” stage where the data collection sheets, patient assessment techniques, and patient
follow-up questions were evaluated and revised.
2.5. Rationale for current study
What are the potential benefits of a decision rule for the use of CT in patients with minor head
injury? First, patient care would be standardized and improved. The great variation in current
Canadian practice and the fact that the vast majority of CT scans are negative suggests the need for
accurate and reliable guidelines. Physicians working in smaller hospitals without CT scanners (70%
of Canadian acute care hospitals59) would have clear directions regarding which children require
costly and time-consuming transfers for a CT scan.
Second, an accurate decision rule could lead to large savings for the health care system. The
current variation in practice and the low yield of CT scans among children with minor head injury
suggests significant potential for reducing the use of CT. Our survey of Canadian pediatric
emergency physicians (see section 2.4.3) clearly indicates their willingness to adopt a decision rule
for children with minor head injury. Previous studies by Dr. Ian Stiell, a co-investigator on this grant
application, have shown a large reduction in the use of ankle radiography after the implementation of
the Ottawa ankle rules.60-61 Based on this experience, we estimate that a 25-50% relative reduction
5e
Osmond, Martin Henry
$145,845
could be safely achieved with a reliable decision rule. On the other hand, without selective
guidelines, there is a very strong likelihood that the use of CT for minor head injury will continue to
increase markedly over the next 5 years, as this technology becomes increasingly available.
Third, although CT itself is a safe procedure, some healthy children require sedation or
anesthesia to keep them immobile in the CT scanner. Any benefits gained from widespread CT
scanning should be weighed against the possible harm of sedating and/or anesthetizing a large
number of children. An accurate decision rule, by decreasing the number of CT scans required, would
expose fewer children with minor head injury to the potential risks of sedation.
Based on the 2 studies detailed above (sections 2.4.2 and 2.4.4), we have already found that
the use of CT scanning in pediatric minor head injury has more than doubled in a 5 year period (from
15% to 39%). This has occurred despite the fact that acute brain injury rates are essentially
unchanged (5.2% vs 4.6%). With more CT scanners being installed every year in Canadian hospitals
the use of this expensive diagnostic technology is likely to continue to increase unless a decision rule
can be developed to guide physicians in the efficient use of this resource.
We feel the likelihood of a physician missing acute brain injury while using our decision rule
is quite remote given: a) our intention to develop a rule with 100% sensitivity and a narrow
confidence interval around this sensitivity, and b) our intention to prospectively validate the decision
rule prior to recommending the rule for clinical use. Although many guidelines have had little
influence on physician practice,62 we have clearly demonstrated that accurate, reliable, and clinically
sensible decision rules, such as the Ottawa ankle rules, have been readily implemented by many
physicians and have lead to a real change in clinical behaviour.60-61 We strongly believe that a well
derived and validated decision rule would be widely adopted by emergency physicians and that the
rule would significantly improve the quality of patient care.
This study will achieve two goals: the derivation of a decision rule (phase I), and the
prospective validation of the decision rule (phase II). Unfortunately, many proposed decision rules
are not prospectively validated and, consequently, have had little impact on clinical practice. Major
medical journals generally will not publish results of a clinical decision rule study without a
prospective validation. We believe that the credibility and utility of the decision rule derived in phase
I will be lost without validation in phase II.
2.6 Future Research
Very few decision rules have undergone field trials to test their effectiveness, despite the fact
that the rationale for such rules lies in their ability to alter actual patient care.36,63 Consequently, we
expect, in a subsequent study, to assess the clinical effectiveness of implementing the decision rule in
practice (phase III). This would involve a controlled clinical trial and would closely parallel Dr.
Stiell’s previous study on the implementation of decision rules for acute ankle injuries.60,61 Other
future research would involve a formal health economic analysis of the impact of implementing a
decision rule for CT.
3. SPECIFIC OBJECTIVES
3.1 Phase I: Derivation of the Rule
The goal of phase I is to derive a clinical decision rule that is highly sensitive for detecting
acute brain injury and which will allow emergency department physicians to maximize the
efficiency of their use of CT in children with minor head injury. Specific objectives are:
3.1.1 To develop standardized clinical assessment methods for patients with minor head injury.
3.1.2 To apply these standardized clinical assessments to patients with acute minor head injury.
3.1.3 To determine the interobserver reliability of the clinical findings.
3.1.4 To determine the association between the clinical findings and acute brain injury.
3.1.5 To use multivariate techniques to derive a highly sensitive clinical decision rule for acute
patients with minor head injury to guide the use of CT.
5f
Osmond, Martin Henry
$145,845
3.1.6 To assess the classification performance of the derived decision rule.
3.1.7 To determine the emergency physicians’ comfort in ordering no CT.
3.1.8 To determine emergency physicians’ accuracy in predicting acute brain injury without the
decision rule.
3.1.9 To perform an economic analysis of the impact of implementing the decision rule.
3.2 Phase II: Validation of the Rule
The goal of phase II is to prospectively assess the accuracy, reliability, and acceptability of
the decision rule on a new set of patients with minor head injury. Specific objectives are:
3.2.1 To determine the classification performance of the decision rule when applied prospectively.
3.2.2 To determine the interobserver reliability of the rule.
3.2.3 To determine physicians’ comfort with implementing the rule.
3.2.4 To determine the potential of the rule to reduce the number of CTs ordered.
4. METHODS - PHASE I: DERIVATION OF THE RULE
4.1 Study Population
4.1.1 Inclusion Criteria. All children 0 to 16 years of age presenting to one of the study hospital
emergency departments after sustaining acute minor head injury will be eligible. Patient eligibility as
an « acute minor head injury » case will be determined by the attending staff emergency physician
based on the patient having all of the following characteristics at the time of arrival in the emergency
department:
1) blunt trauma to the head resulting in at least one of the following: a) definite loss of
consciousness, b) amnesia, c) disorientation (These may be determined from the patient
or from the report of a witness and are considered present no matter how brief. Amnesia
will be determined by asking the following questions: “do you remember the injury?”,
“how did you get to the hospital?” and “have you talked to me before?”), d) persistent
vomiting (> 3 times over at least 4 hours), or e) irritability in children < 2 years
(irritability out of keeping with the child’s normal temperament).
2) initial emergency department GCS or modified infant GCS64 of 13 or greater as
ascertained by the attending emergency physician (see Appendix 1), and
3) injury within the past 24 hours.
4.1.2 Exclusion Criteria. Patients will be excluded if they meet one of the following criteria:
1) patient age greater than 16 years, 2) no clear history of trauma as the primary event (e.g., seizure or
syncope as the primary event), 3) penetrating skull injury or known depressed skull fracture, 4) focal
neurological deficit (motor or cranial nerve) that cannot be attributed to an extra-cerebral cause, (e.g..
traumatic mydriasis or peripheral neuropathy are not exclusion criteria), 5) referred patients who have
already had a CT scan of the head, 6) severe, chronic generalized neurological delay, 7) cerebrospinal
fluid shunt, 8) pregnant females, 9) previous CT head study enrollment and 10) head injury secondary
to child abuse
4.1.3 Patient Selection. Consecutive patients with minor head injury will be entered into the
study if they meet the inclusion and exclusion criteria and if one of the designated assessor physicians
(described in section 4.2.2) is on duty, which will be virtually 100% of the time. For comparative
purposes, demographic and outcome data will be collected from the record of treatment for eligible
patients who are not enrolled into the study.
4.1.4 Study Setting. The patients will be enrolled from the emergency departments of nine
Canadian pediatric hospitals from the provinces of Quebec (2), Ontario (3), Manitoba (1), Alberta (2)
and British Columbia (1). These centers constitute 9 of the 12 pediatric hospitals in Canada and have
a combined annual emergency department volume of approximately 420,000 patient visits.
4.2 Standardized Patient Assessment
5g
Osmond, Martin Henry
$145,845
4.2.1 Patient Assessment. Full or part time staff physicians of the study hospital emergency
departments will make all patient assessments. Interns and residents may see eligible head injury
patients but will be asked to have staff physicians make the study assessments. The physician
assessors will be trained by means of a one hour lecture and by a practical demonstration to assess the
clinical variables in a uniform manner. A standardized description of each examination technique
will be appended to the data collection sheets. The physicians will record their findings on the data
collection sheets (Appendix 6) prior to sending the patients for CT, if a CT scan is deemed
necessary.
Physicians will initially assess the patients shortly after their arrival in the emergency
department and will reassess patients immediately prior to CT, prior to discharge, or at 6 hours after
arrival in the emergency department (whichever comes first). There will be no preset minimum
period of observation but patients may be discharged from the emergency department once they
have achieved normal mental status (GCS score of 15) or have had a normal computed tomogram of
head. Patients will be given a standardized information sheet upon discharge.
4.2.2 Quality Assurance. There will be ongoing evaluation of the quality of the patient
assessments judged by completeness of data collecting sheets and compliance in enrolling eligible
patients. Based on the analysis of our pilot study, research nurses will ensure that the more acute
patients are not missed by providing clinicians with monthly feedback of a general nature as well as
specific review of any individual problems that may arise. Clinicians will not, however, be given any
indication of the preliminary accuracy or reliability of individual variables.
4.2.3 Selection of Variables. The variables selected for assessment in the study were chosen
by the investigators at the Research Formulation Workshop (section 2.4.1) based on clinical
experience and data from the literature. These variables are felt to be useful in predicting whether or
not the patients with minor head injury have acute brain injury.
4.2.4 Variables from History. Variables will be coded as “No/Yes” unless otherwise
specified. Those variables that are to be included in the physician’s reassessment are indicated “initial
and reassess”. a) Age (Years), b) Gender (Male/Female), c) Mechanism of injury (Motor vehicle
collision / Motorcycle / Bicycle / Pedestrian struck/Off road vehicle) / Assault hands or feet / Assault
blunt object/collision with stationary object / Fall from height > 3 feet or 5 stairs / Sports / Other, d)
If motor vehicle accident, i) Speed (Stopped / City speed / Highway speed), ii) Collision, iii) Seat
belt / Car seat use, e) Helmet use, f) Witnessed loss of consciousness, g) If yes, duration of loss of
consciousness in seconds and minutes, h) Amnesia, i) If yes, duration of retrograde amnesia for
period prior to injury in minutes and hours (initial and reassess), j) Worsening headache (initial and
reassess), k) Time from injury to assessment in hours, l) vomiting, If yes, how often and how long
in hours, m) seizure, If yes, on impact or after, how long after injury and duration of seizure, n)
Known bleeding disorder, If yes, specify, o) Previous visit for same head injury.
4.2.5 Variables from Neurological Examination. All variables will be included in
physician’s initial assessment as well as the reassessment: a) GCS or modified infant GCS64 score
in emergency department; (initial and reassess; failure to improve to GCS score of 15 or deterioration
of GCS will be specifically noted), b) Pupillary changes (anisocoria) (initial and reassess), c)
Lateralizing motor weakness (arm drift, grip strength) (initial and reassess).
4.2.6 Variables from General Examination or Diagnostic Tests. a) Unstable vital signs,
b) Lethargy (drifts off to sleep when left alone), c) Injuries other than head, If yes, specify, d)
Possible skull penetration or depressed fracture, e) Any sign of basilar skull fracture (drainage
from ear, CSF rhinorrhea, hemotympanum, Battle’s sign, “raccoon eyes”), f) Fracture on skull
radiography (radiography ordered if physician concerned about possible skull penetration or
depressed fracture).
5h
Osmond, Martin Henry
$145,845
4.2.7 Physicians’ Judgement. The physicians will also be asked to answer three questions
(not to be used as predictor variables) regarding their attitude to radiography and their clinical
judgement: a) Theoretical comfort with ordering no CT for that patient, on a five point scale (very
comfortable to very uncomfortable), b) Probability of acute brain injury (defined in 4.3.1), to the
closest decile, and c) Probability of patient requiring neurological intervention (defined in 4.3.2),
to the closest decile.
4.2.8 Interobserver Reliability. Ten percent of patients will be assessed for the clinical
variables by a second emergency physician who will be blinded to the results of the first assessment.
These second assessments will be performed in all centres on a feasibility basis whenever two study
physicians are available in the emergency department.
4.3 Outcome Measures
4.3.1 Primary Outcome. Patients will undergo standard CT of head after the clinical
assessment and their tomograms will be interpreted by fully qualified independent staff radiologists.
The radiologists may be provided routine clinical information on the radiology requisition but will be
blinded to the contents of the data collection sheet. CT will be without contrast, will be performed
with third generation equipment, will involve a minimum of 10 mm cuts from the foramen magnum
to the vertex, and will include both soft tissue and bone windows.
The primary outcome, acute brain injury, is defined as any acute brain or skull finding
revealed on CT, including depressed and basilar skull fractures but excluding linear fractures. The
presence of acute brain injury implies that intervention or treatment may be required. Linear skull
fractures, in the absence of acute brain injury, generally do not require intervention and will be
considered as secondary outcomes (section 4.3.3). We believe that the primary outcome, acute brain
injury, is more objective and transportable than such secondary outcomes as “need for intervention”
or “need for admission”. The latter are subject to inter-physician and inter-community variability.
The acute brain or skull findings on CT will be classified as follows: a) Hematoma: (i) Epidural, (ii)
Subdural, (iii) Intracerebral, or (iv) Cerebellar, b) Diffuse Cerebral Edema, c) Cerebral Contusion, d)
Hemorrhage: (i) Subarachnoid, or (ii) Intraventricular, e) Pneumocephalus, or f) Skull Fractures: I)
Depressed or ii) Basilar. These findings will be classified as Solitary or Combined as well as
Predominant (most clinically important) or Secondary.
The reliability of the CT interpretations will be assessed by having all abnormal and 10%
(randomly selected) of normal scans reviewed by a second radiologist who is blinded to the first
interpretation. A single radiologist at the Ottawa coordinating centre will arbitrate any disagreements
of interpretation which cannot be resolved locally.
4.3.2 Proxy Primary Outcome. Review of current practice at the study hospitals indicates variation
from centre to centre and that many eligible patients with minor head injury routinely do not undergo
CT. We believe that the study protocol cannot demand that all children have CT. This would be
costly, unacceptable to physicians, and potentially dangerous for children who must be sedated or
anaesthetized for the procedure. Generally, patients who are not referred for CT by the emergency
physician have less severe injuries and are extremely unlikely to have acute brain injury as defined
above. All such patients will have telephone follow-up and will be classified as having no acute
brain injury if they meet all the following explicit criteria at 14 days: a) headache is absent or
mild, b) no complaints of memory or concentration problems, c) have not had a seizure or
developed focal motor findings, d) no vomiting e) return to baseline, appropriate for age, normal
daily activities (e.g., sleep, eating, play, school or sports).
The assessment of these criteria will be made by a research assistant who is unaware of the
patient’s status for the individual predictor clinical variables. Patients who cannot fulfil these criteria
will be recalled for clinical assessment and possible CT. This list of criteria was developed by the
investigators based on their clinical judgement and is similar in concept to questionnaires applied in
5i
Osmond, Martin Henry
$145,845
previous ankle and knee injury studies. We feel that this measure is appropriate as Dr. Stiell has
recently confirmed the validity of a similar “proxy primary outcome tool” on a sub-sample of patients
from his adult Canadian CT Head Rule study.65 As with the adult tool, we will assess the validity of
these criteria to exclude acute brain injury in children by applying the telephone follow-up
questionnaire to all patients, including those with normal and abnormal CT scan results.
4.3.3 Secondary Outcomes. The ability to predict the following secondary outcomes will
also be modeled: a) Need for neurological intervention defined as the need for any of the following
within seven days: craniotomy, elevation of skull fracture, intubation, intracranial pressure
monitoring, or anticonvulsant medication, b) Admission to hospital, and c) Death prior to hospital
discharge. The secondary outcomes will be determined from the medical record by a research
assistant who is unaware of the contents of the data collection sheet. Assessment of long-term
outcomes, such as neurological deficit or neuropsychological deficit, is felt to be beyond the scope of
this study. Assessment of all patients at six months by a neuropsychologist, for example, would be
extremely expensive.
4.4 Data Analysis
4.4.1 Interobserver Agreement. The interobserver agreement for each variable will be
measured by calculating the kappa coefficient, the proportion of potential agreement beyond chance,
along with 95% confidence intervals.66-67 For variables with three or more ordered categories, a
weighted kappa measure of interobserver agreement will be calculated.68 A variable will be deemed
to have acceptable interobserver agreement if the kappa coefficient has a value of at least 0.6. The
value of 0.6 is considered to represent “substantial agreement”66-67 and has proven to be a satisfactory
threshold in previous research involved with decision rule development.
4.4.2 Univariate Analysis. Univariate analyses will be used to determine the strength of
association between each variable and the primary outcome, acute brain injury. This process will aid
selection of the best variables for the multivariate analyses. The appropriate univariate technique will
be chosen according to the type of data: for nominal data, the chi-square test with continuity
correction; for ordinal variables, the Mann-Whitney U test; and, for continuous variables, the
unpaired 2-tailed t-test, using pooled or separate variance estimates as appropriate.
4.4.3 Multivariate Analysis. Multivariate analyses will derive a model to predict acute brain
injury. Those variables found to be both reliable (kappa > 0.6) and strongly associated with the
outcome measures (P < 0.05) will be combined using one of two different multivariate techniques,
recursive partitioning or logistic regression. Second order interaction among predictor variables will
be evaluated using the Mantel-Haenszel and logistic model procedures and appropriate composite
variables will be considered for incorporation into the multivariate analyses.
The objective will be to find the best combinations of predictor variables that are highly
sensitive for detecting the outcome measure while achieving the maximum possible specificity. To
be clinically acceptable, the model must be nearly 100% sensitive. The derived models must be easy
to use by clinicians and therefore should contain as few variables as possible. A model will only be
acceptable, therefore, if it fulfils these criteria: a) 99% or greater sensitivity, b) adequate specificity
to lead to a 20% relative reduction in use of CT, and c) has no more than six component variables.
Assuming more than one model meets the minimum acceptable criteria, the best model will be the
one which has the highest specificity and the fewest number of component variables.
Recursive partitioning will be used to determine the most significant variables in a model
highly sensitive for the primary outcome measure and will be performed using KnowledgeSEEKER
Version 2.1 Software (Angoss Software International, Toronto).69-72 Dr. Ian Stiell has had extensive
experience in using recursive partitioning to develop decision rules for ankle and for knee injury
radiography. Our experience suggests that recursive partitioning may be more suitable than logistic
regression when the objective is to correctly classify one outcome group at the expense of the other,
5j
Osmond, Martin Henry
$145,845
i.e. where high sensitivity is more important than overall accuracy. Identification of all acute brain
injury cases is the priority of the model building and misclassification of some cases without injury
will be quite acceptable. We can deliberately drive the analysis to correctly classify the acute brain
injury outcome group at the expense of the group with no brain injury. We can also deliberately avoid
complex models with significant interactions that would be difficult for clinicians to interpret or
apply.
Attempts will also be made to find the best model by performing logistic regression as an
alternate multivariate technique to recursive partitioning. Model building will proceed with forward
stepwise selection until no variables meet the entry (0.05) or removal (0.10) criteria for the
significance level of the likelihood-ratio test. In order to provide a more simple model for clinicians,
cutpoints will be sought for continuous variables. Variables selected for the model will be given
weights equal to the coefficient rounded to the nearest whole number. This approach has been used
successfully in the past for developing a decision rule for delayed functional recovery in knee injury
patients.
The variables chosen by the best model will constitute the decision rule for selecting patients
with minor head injury for CT. The decision rule will be presented in clear narrative form that does
not require computation or use of statistical aids. A hypothetical example of the decision rule is as
follows:
“Children with acute minor head injury require CT if they demonstrate any one of these
findings: (i) loss of consciousness greater than five minutes, (ii) retrograde amnesia for a
period of greater than 60 minutes, (iii) any focal deficit or deterioration in GCS within 6 hours
of injury, or (iv) failure to achieve Glasgow Coma Score of 15 within 6 hours of injury.”
4.4.4 Classification Performance. The derived decision rule will be cross-validated by
comparing the classification of each patient to their actual status for the primary outcome, acute brain
injury. This will allow an estimate, with 95% confidence intervals, of the sensitivity and specificity
of the rule. A more robust validation will be carried out prospectively on a new set of patients in
Phase II. The sensitivity and specificity of the rule will also be calculated for the subgroups and for
the secondary outcomes.
4.4.5 Physicians’ Judgement. Data from the three questions on physicians’ comfort and
predictions will be tabulated in a simple descriptive format. In addition, information on the predicted
probabilities will be used to calculate receiver operating characteristic (ROC) curves and likelihood
ratios for determining acute brain injury as well as the need for neurological intervention,
respectively. The accuracy of the physicians’ predictions will be compared to that of the derived
decision rule by ROC curve analysis.
4.4.6 Interim Analysis. Multivariate analyses will initially be performed after 80% of the
phase I sample size has been accrued in order to determine the likelihood of achieving a satisfactory
decision rule (defined in section 4.4.3). The expectation of deriving a satisfactory decision rule will
permit the investigators to plan for the timely start-up of phase II (validation of the rule) without a
prolonged period between phases I and II. A prolonged downtime between phases would result in
loss of cooperation of the participating physicians as well as inefficient use of the paid study
personnel. On the other hand, the investigators will abandon plans to proceed to phase II if a
satisfactory decision rule cannot be derived during phase I. In such a case, funding for the latter two
years of the study will be returned to the CIHR.
4.5 Sample Size
Based on our retrospective multi-center study of 1995 data, we know there were 1164 patients
per year eligible at our study sites and 5.2% of those patients had positive CT scans (n=121 patients).
(Appendix 3) Prior experience with recruitment in decision rule studies has resulted in patient
recruitment rates of around 70%. We are confident with this recruitment rate as it is in complete
5k
Osmond, Martin Henry
$145,845
agreement with our Pilot study which also found a recruitment rate of 70% (291 of 417 patients). If
over 2 years there are 2328 patients and 70% are recruited this means that we can realistically
recruit 1630 patients with minor head injury over the 2 year study period. 5.2% of these patients
(n=85) will have a positive finding on CT scan as defined in section 3.6.1. Based on our survey of
physicians working in pediatric emergency departments, we estimate that a 99% sensitivity will be an
acceptable level for a decision rule detecting a positive finding on CT scan. Based on sample size
calculations by Arkin and Wachtel, this means that this study can generate a 95% confidence
interval around 99%, ranging from 97% to 100%.73 These estimates are well within the range of
comfort indicated by the emergency physicians where the mean target sensitivity was 94.9%.
5. METHODS - PHASE II: VALIDATION OF THE RULE
5.1 Study Population
The inclusion and exclusion criteria will be those used for Phase I and will be prospectively
applied to a new group of patients in the same nine hospital centres. We believe that the
transportability and generalizability of the decision rule will be clearly demonstrated by assessing the
rule: a) in new patients, b) in a large number of hospitals from different areas of Canada. To validate
the rule in hospitals different from those of phase I would not be feasible for the current research
team i.e. would require an entirely new group of collaborators.
5.2 Standardized Patient Assessment
All eligible patients will be assessed by the same staff emergency physicians to determine the
patients’ status for each of the component variables within the decision rule derived in Phase I. The
physicians will record these findings along with their interpretation of the decision rule itself on the
data collection sheets prior to radiography. Variables not found to be useful in phase I will not be
assessed in phase II. A subset of patients, when feasible, will be independently assessed by a second
emergency physician to judge interobserver agreement for interpretation of the rule itself as well as
for the component variables. In addition, the physicians will indicate, on a five-point scale, their
theoretical comfort in applying the rule for each patient.
5.3 Outcome Measures
The primary outcome measure will be ascertained in the same manner as in phase I (4.3.1).
Acute brain injury will be determined by standard CT of the head which will be interpreted by a
staff radiologist blinded to the contents of the data collection sheet. The treating physicians will be
encouraged not to alter their normal radiography ordering practice. Consequently, we expect that
some patients will have their outcome status determined by telephone follow-up, using the same
proxy primary outcome as in phase I. Patients will be considered not to have acute brain injury if
they fulfil the same criteria listed in section 4.3.2. This judgement will be made at 14 days by a
research assistant unaware of the classification on the data collection sheet. Patients who do not fulfil
the criteria will be recalled for reassessment and CT. As in phase I (4.3.3), secondary outcomes
(need for neurological intervention, admission, and death) will be determined from the medical
record by a research assistant who is unaware of the classification indicated on the data collection
sheet.
5.4 Data Analysis
5.4.1 Classification Performance. The ability of the rule to classify patients for acute brain
injury will be calculated with 95% confidence intervals for sensitivity, specificity, negative predictive
value, and positive predictive value. Calculation of likelihood ratios will allow one to estimate the
probability of acute brain injury for patients, given a negative status for the decision rule.
Classification performance will also be determined for the secondary outcomes, need for neurological
intervention and death. The actual performance of the rule will be compared to the predictions of
physicians made in phase I.
5l
Osmond, Martin Henry
$145,845
5.4.2 Interobserver Agreement. The kappa, and in some cases the weighted kappa,
coefficient will be calculated for each of the component clinical variables in the decision rule as well
as for interpreting the actual rule. The overall accuracy of the physicians in interpreting the rule will
be calculated.
5.4.3 Physicians’ Theoretical Comfort. Data will be tabulated in a simple descriptive
format to allow some interpretation of the physicians’ acceptance of the decision rule and the
potential for implementation into actual clinical practice.
5.4.4 Potential CT Saving. The referral fraction for CT of the head according to the decision
rule will be compared to the baseline referral fractions of the Retrospective Study at the study
hospitals. This will estimate the potential for reducing the cost and number of computed tomograms
ordered, if the rule had been applied.
5.5 Sample Size
Sample size assumptions are similar to those for phase I. To validate that the decision rule is
99% sensitive for detecting acute brain injury with clinically acceptable 95% confidence bounds, 85
patients with acute brain injury will be enrolled. This will yield the upper and lower confidence
levels of 100% and 97.0% respectively (95% confidence interval). This, in turn, requires a total of
another 1630 patients with minor head injury and an approximate accrual period of 24 months.
6. RELEVANCE
Each year physicians in North American emergency departments treat approximately 420,000
children with minor head injury (i.e. loss of consciousness or amnesia with GCS of 13-15). Only
5.2% of these patients have suffered any type of brain injury visible on CT scan and only 0.5% have
suffered a potentially life-threatening epidural hematoma. There are no widely accepted guidelines to
help physicians standardize quality of care or to maximize the efficiency of their use of CT for minor
head injury. Authoritative voices in editorials and the ATLS guidelines are calling for CT scanning of
all children with minor head injury. Our data clearly show that Canadian emergency physicians
accept and endorse the concept of selective ordering of CT. At the present time, there is considerable
variation among Canadian hospitals in the referral rate for CT (ranging from 14% to 56% for similar
patients). This rate has more than doubled in the last 5 years without any change in the acute brain
injury rate. Use of CT is not efficient with 93% of CT scans ordered in Canadian centres being
negative for any injury. Inefficient use of CT scans significantly contributes to health care costs.
Our pediatric emergency research collaboration has recently successfully validated the Ottawa
ankle rules for use in children.74 As a result the research nurses and physicians in collaborating
centers are experienced in carrying out studies on the development of decision rules.
Building on our Pilot Study and using techniques successfully applied to ankle and knee
radiography, we propose to develop (phase I) and prospectively validate (phase II) a clinical decision
rule which will permit physicians to be more accurate in their diagnosis of acute brain injury among
children with minor head injury. Through the use of this rule physicians will be more selective in
their use of CT without jeopardizing the quality of patient care. Such a decision rule should,
therefore, lead to improved patient care as well as considerable savings for North American health
care systems. The potential impact of this decision rule will be similar to that of the Ottawa ankle
rules that have been readily adopted by many physicians and which have been shown to significantly
reduce health care costs.
5m
Osmond, Martin Henry
$145,845
7. REFERENCES
1. Dietrich AM, Bowman MJ, Ginn-Pease ME, Kosnik E, King DR. Pediatric head injuries: can
clinical factors reliably predict an abnormality on computed tomography? Annals of Emergency
Medicine 1993; 22:1535-1540.
2. Ramundo ML, McKnight T, Kempf J, Satkowiak L. Clinical predictors of computed tomographic
abnormalities following pediatric traumatic brain injury. Pediatric Emergency Care 1995; 11:1-4.
3. Mitchell KA, Fallat ME, Raque GH, Hardwick VG, Groff DB, Nagaraj HS. Evaluation of minor
head injury in children. Journal of Pediatric Surgery 1994; 29:851-854.
4. Quayle KS, Jaffe DM, Kuppermann N, Kaufman BA, Lee BCP, Park TS, et al. Diagnostic Testing
for Acute Head Injury in Children: When Are Head Computed Tomography and Skull Radiographs
Indicated? Pediatrics 1997; 99:e11.
5. Davis RL, Hughes M, Gubler KD, Waller PL, Rivara FP. The use of cranial CT scans in the triage
of pediatric patients with mild head injury. Pediatrics 1995; 95:345-349.
6. Galbraith S. Misdiagnosis and delayed diagnosis in traumatic intracranial haematoma. Br Med J
1976; 1:1438-1439.
7. Mendelow AD, Karmi MZ, Paul KS, Fuller GAG, Gillingham FJ. Extradural haematoma: effect
of delayed treatment. Br Med J 1979; 1:1240-1242.
8. Seelig JM, Becker DP, Miller JD, Greenberg RP, Ward JD, Choi SC. Traumatic acute subdural
hematoma: major mortality reduction in comatose patients treated within four hours. N Engl J Med
1981; 304:1511-1518.
9. Marshall LF, Toole BM, Bowers SA. The National Traumatic Coma Data Bank. Part 2: Patients
who talk and deteriorate: implications for treatment. J Neurosurg 1983; 59:285-288.
10. Ceviker N, Baykaner K, Keskil S, Cengel M, Kaymaz M. Moderate head injuries in children as
compared to other age groups, including the cases who had talked and deteriorated. Acta
Neurochirurgica 1995; 133:116-121.
11. dos Santos AL, Plese JP, Ciquini Junior O, Shu EB, Manreza LA, Marino Junior R. Extradural
hematomas in children. Pediatric Neurosurgery 1994; 21:50-54.
12. Jamjoom A, Cummins B, Jamjoom ZA. Clinical characteristics of traumatic extradural
hematoma: a comparison between children and adults. Neurosurgical Review 1994; 17:277-281.
13. Pillay R, Peter JC. Extradural haematomas in children. South African Medical Journal 1995;
85:672-674.
14. Benoit BG, Russell NA, Richard MT, Hugenholtz H, Ventureyra EC, ChooSH. Epidural
hematoma: report of seven cases with delayed evolution of symptoms. Canadian Journal of
Neurological Sciences 1982; 9:321-324.
5n
Osmond, Martin Henry
$145,845
15. Ammirati M, Tomita T. Posterior fossa epidural hematoma during childhood. Neurosurgery 1984;
14:541-544.
16. Schutzman SA, Barnes PD, Mantello M, Scott RM. Epidural hematomas in children. Annals of
Emergency Medicine 1993; 22:535-541.
17. Chandrasekaran S, Zainal J. Delayed traumatic extradural haematomas. Australian & New
Zealand Journal of Surgery 1993; 63:780-783.
18. Godano U, Seracchioli A, Servadei F, Donati R, Piazza G. Intracranial lesions of surgical interest
in minor head injuries in paediatric patients. Childs Nervous System 1992; 8:136-138.
19. Bell RS, Loop JW. The utility and futility of radiographic skull examination for trauma. N Engl
J Med 1971; 284:236-239.
20. Masters SJ, McClean PM, Argarese JS, Brown RF, Campbell JA, Freed HA, et al. Skull x-ray
examinations after head trauma. N Engl J Med 1987; 316:84-91.
21. National Center for Health Statistics. National Hospital Ambulatory Medical Care Survey 1992.
Hyattsville: National Center for Health Statistics, 1994.
22. Dacey RG Jr, Alves WM, Rimel RW, Winn HR, Jane JA. Neurosurgical complications after
apparently minor head injury: assessment of risk in a series of 610 patients. J Neurosurg. 1986;
65:203-210.
23. Hahn YS, McLone DG. Risk factors in the outcome of children with minor head injury. Pediatr
Neurosurg. 1993; 19:135-142.
24. Rosenthal BW, Bergman I. Intracranial injury after moderate head trauma in children. J Pediatr.
1989; 115:346-350.
25. Stein SC, Ross SE. Mild head injury: a plea for routine early CT scanning. J Trauma 1992;
33:11-13.
26. American College of Surgeons. Advanced trauma life support instructor manual. Edition 6.
Chicago: American College of Surgeons, 1997.
27. American Academy of Pediatrics. The Management of Minor Closed Head Injury in Children.
Pediatrics 1999; 104:1407-1415.
28. Hoffman JR. CT for head trauma in children [editorial; comment]. Annals of Emergency
Medicine 1994; 24:713-714.
29. Feinstein AR. The “chagrin factor” and qualitative decision analysis. Arch Intern Med 1985;
145:1257-1259.
5o
Osmond, Martin Henry
$145,845
30. Stiell IG, McDowell I, Nair RC, Aeta H, Greenberg GH, McKnight RD, et al. Use of
radiography in acute ankle injuries: physicians’ attitudes and practice. Can Med Assoc J 1992;
147:1671-1678.
31. Long AE. Radiographic decision-making by the emergency physician. Emerg Med Clin N Am
1985; 3:437-446.
32. Pozen MW, D’agostino RB, Mitchell JB, Rosenfeld M, Guglielmino JT, Schwartz ML, et al.
The usefulness of a predictive instrument to reduce inappropriate admissions to the coronary care
unit. Ann Intern Med 1980; 92:238-242.
33. Goldman L, Cook EF, Brand DA. A computer protocol to predict myocardial infarction in
emergency department patients with chest pain. N Engl J Med 1988; 318:797-803.
34. Heckerling PS, Tape TG, Wigston RS. Clinical prediction rule for pulmonary infiltrates. Ann
Intern Med 1990; 113:664-670.
35. Bates DW, Cook EF, Goldman L, Lee TH. Predicting bacteremia in hospitalized patients: a
prospectively validated model. Ann Intern Med 1990; 113:495-500.
36. Wasson JH, Sox HC, Neff RK, Goldman L. Clinical prediction rules: application and
methodological standards. N Engl J Med 1985; 313:793-799.
37. Feinstein AR. Clinimetrics. New Haven: Yale University Press, 1987.
38. Yamamoto LG, Bart RD, Jr. Transient blindness following mild head trauma. Criteria for a
benign outcome. [Review]. Clinical Pediatrics 1988; 27:479-483.
39. Parish RA, Woolf AR, Eichner M, Cauldwell C. The significance of Babinski signs in children
with head trauma. Annals of Emergency Medicine 1985; 14:329-330.
40. Jaimovich R, Monges JA. Delayed posttraumatic intracranial lesions in children. [Review] [8
refs]. Pediatric Neurosurgery 1991; 17:25-29.
41. Kaye EM, Herskowitz J. Transient post-traumatic cortical blindness: brief v prolonged syndromes
in childhood. Journal of Child Neurology 1986; 1:206-210.
42. Stein SC, Spettell CM. Delayed and progressive brain injury in children and adolescents with
head trauma. Pediatric Neurosurgery 1995; 23:299-304.
43. Hennes H, Lee M, Smith D, et al. Clinical predictors of severe head trauma on children.
American Journal of Diseases in Children 1988; 142:1045-1047.
44. Hennes H, Losek JD, Sty JR, Gill JC. Computerized tomography in hemophiliacs with head
trauma. Pediatric Emergency Care 1987; 3:147-149.
5p
Osmond, Martin Henry
$145,845
45. Davis RL, Mullen N, Makela M, Taylor JA, Cohen W, Rivara FP. Cranial computed tomography
scans in children after minimal head injury with loss of consciousness [see comments]. Annals of
Emergency Medicine 1994; 24:640-645.
46. Dietrich AM, James CD, King DR, Ginn-Pease ME, Cecalupo AJ. Head trauma in children with
congenital coagulation disorders. Journal of Pediatric Surgery 1994; 29:28-32.
47. Ros SP, Ros MA. Should patients with normal cranial CT scans following minor head injury be
hospitalized for observation? Pediatric Emergency Care 1989; 5:216-218.
48. Rivara F, Tanaguchi D, Parish RA, et al. Poor prediction of positive computed tomographic scans
by clinical criteria in symptomatic pediatric head trauma. Pediatrics 1987; 80:579-584.
49. Rosenthal BW, Bergman I. Intracranial injury after moderate head trauma in children. Journal of
Pediatrics 1989; 115:346-350.
50. Schunk JE, Rodgerson JD, Woodward GA. The utility of head computed tomographic scanning
in pediatric patients with normal neurologic examination in the emergency department. Pediatric
Emergency Care 1996; 12:160-165.
51. Kang JK, Park CK, Kim MC, Kim DS, Song JU. Traumatic isolated intracerebral hemorrhage in
children. Childs Nervous System 1989; 5:303-306.
52. Liguori G, Foggia L, Buonaguro A, Colucci M, Cantone G, Ambrosio A. EEG findings in minor
head trauma as a clue for indication to CT scan. Childs Nervous System 1989; 5:160-162.
53. Teasdale GM, Murray G, Anderson E, Mendelow AD, MacMillan R, Jennett B, et al. Risks of
acute traumatic intracranial haematoma in children and adults: implications for managing head
injuries. British Medical Journal 1990; 300:363-367.
54. Chan KH, Mann KS, Yue CP, Fan YW, Cheung M. The significance of skull fracture in acute
traumatic intracranial hematomas in adolescents: a prospective study. Journal of Neurosurgery 1990;
72:189-194.
55. Lloyd DA, Carty H, Patterson M, Butcher CK, Roe D. Predictive value of skull radiography for
intracranial injury in children with blunt trauma. Lancet 1997; 349:821-824.
56. Haydel MJ, Preston CA, Mills TJ, Luber S, Blaudeau E, DeBlieux PMC. Indications for
computed tomography in patients with minor head injury. N Engl J Med 2000; 343:100-105.
57. Stiell I, Lesiuk H, Vandemheen K, Wells G, Clement C, De Maio V, et al. Revised Canadian CT
Head Rule for Patients with Minor Head Injury (Abstract). Acad Emerg Med 2000; 7:564-565.
58. Klassen TP, Reed MH, Stiell IG, Nijssen-Jordan C, Tenenbein M, Joubert G, et al. Variation in
Utilization of Computed Tomography Scanning for the Investigation of Minor Head Trauma in
Children: a Canadian Experience. Acad Emerg Med 2000; 7:739-744.
59. Guide to Canadian Healthcare Facilities. Ottawa: Canadian Healthcare Association, 1995.
5q
Osmond, Martin Henry
$145,845
60. Stiell IG, McKnight RD, Greenberg GH, McDowell I, Nair RC, Wells GA, et al.
Implementation of the Ottawa ankle rules. J A M A 1994; 271:827-832.
61. Stiell I, Wells G, Laupacis A, Brison R, Verbeek R, Vandemheen K, et al. A multicentre trial to
introduce clinical decision rules for the use of radiography in acute ankle injuries. Br Med J 1995;
311:594-597.
62. Greco PJ, Eisenberg JM. Changing physicians’ practices. N Engl J Med 1993; 329:1271-1274.
63. Lee TH. Evaluating decision aids: the next painful step. J Gen Intern Med 1990; 5:528-529.
64. Raimondi A, Hirschauer J. Head injury in the infant and toddler: Coma scoring and outcome
scale. Child's Brain 1984; 11:12-35.
65. Stiell I, Morrison L, Schull M, Vandemheen K, Clement C, Brison R, et al. Validity Evaluation
of the Brain Injury Proxy Outcome Assessment Tool in the CCC Study (Abstract). Acad Emerg Med
2000; 7:533-534.
66. Kramer MS, Feinstein AR. Clinical biostatistics: LIV. The biostatistics of concordance. Clin
Pharmacol Ther 1982; 29:111-123.
67. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics
1977; 33:159-174.
68. Fleiss JL. Statistical methods for rates and proportions. 2nd ed. New York: John Wiley &
Sons, 1981.
69. Ciampi A, Hogg SA, McKinney S, Thiffault J. RECPAM: a computer program for recursive
partition and amalgamation for censored survival data and other situations frequently occurring in
biostatistics. I. Methods and program features. Comput Meth Prog Biomed 1988; 26:239-256.
70. Ciampi A, Chang CH, Hogg S, McKinney S. Recursive partition: a versatile method for
exploratory data analysis in biostatistics. In: MacNeill IB, Umphrey GJ, editors. Time series and
econometric modelling / Biostatistics Vol. V. Boston: D. Reidel Publishing Company, 1987:23-50.
71. Friedman JH. A recursive partitioning decision rule for nonparametric classification. IEEE
Trans Comput 1977; 16:404-408.
72. Ciampi A, Thiffault J, Nakache J, Asselain B. Stratification by stepwise regression,
correspondence analysis and recursive partition: a comparison of three methods of analysis for
survival data with covariates. Comput Stat Data Anal 1986; 4:185-204.
73. Arkin CF, Wachtel MS. How many patients are necessary to assess test performance? JAMA
1990; 263:275-278.
5r
Osmond, Martin Henry
$145,845
74. Plint AC, Bulloch B, Osmond MH, Stiell IG, Dunlap H, Reed M, Tenenbein M, Klassen TP.
Validation of the Ottawa Ankle Rules in children with ankle injuries. Acad Emerg Med 1999;6:10051009.
5s