IMPACT OF ISABEL ON CLINICAL DECISION-MAKING IN ACUTE
PAEDIATRICS
SUMMARY OF RESULTS FROM A SIMULATED FIELD TRIAL
TRIAL DESIGN
Balanced, experimental design
“Repeated measures” evaluation
Paediatricians of various grades registered as ISABEL users, and final-year medical students
from Imperial College Medical School, St Mary’s Campus, were invited to participate in a
simulated trial which aimed to assess changes in clinical decision-making in acute paediatrics
after provision of decision support in the form of the ISABEL system.
24 case histories that described the initial presenting features of children in an acute paediatric
setting were chosen as cases for the trial. They were balanced in terms of level of difficulty (1
– uncommon to 3 – common) and speciality assignment (12 different paediatric specialities).
A
specially
configured,
password-protected
trial
website
was
created
at
www.isabel.org.uk/trial that automatically recorded decisions made by the subjects. Subjects
were asked to make decisions regarding differential diagnosis, immediate investigation plan
and management steps for a randomly chosen set of 12 cases first. In a second step, based on
the diagnostic suggestions provided by ISABEL for each case, subjects were given the
opportunity to modify any of their previous clinical decisions. It was not possible to proceed
from the first step to the second unless the subject’s own decisions (without decision-support)
were clearly recorded on the trial site. The main site at www.isabel.org.uk was barred for use
for all trial users for the length of time they participated in the trial.
Data collected other than the clinical decisions included time taken to process each page,
length of each subject’s paediatric experience and previous use of ISABEL.
Page 1 of 3
PRINCIPAL OUTCOME MEASURES
1. Quality scores for differential diagnosis, investigations and management steps
2. Number of clinically important diagnoses reminded by ISABEL
3. Time spent on each page
4. Number of decisions with negative clinical impact
RESULTS
Table 1: General data regarding trial participants
Description
Number
Total number of participants
Participants who completed all 12 cases
Breakdown of subjects by grade
76
52
Consultants: 18
Registrars: 24
SHOs: 19
Medical students: 15
751 ([52 x 12 = 624] + 127)
Total number of episodes of usage available
Quality scores
Table 2: Increase in quality scores
Average pre-ISABEL (SD)
Average post-ISABEL (SD)
Differential diagnosis
37.9 (19.1)
42.7 (18.7)*
Investigations
35 (19.5)
36.9 (19.7)*
Management
17.33 (14.7)
18.23 (15)*
* significant p value (paired t test) p<0.001
Clinically significant diagnoses
In 77 episodes/751 total episodes, the user was prompted to consider the ‘correct’ diagnosis
after consultation with ISABEL (10.25%). In an additional 2.4%, a ‘near-correct’ diagnosis
was added by the user to the list after ISABEL decision support. In summary, in 12.7% cases,
a clinically significant diagnosis was added by the user only after ISABEL consultation.
Time
In total, time data was complete in 633 episodes.
Median time to process first step (pre-ISABEL): 6 min 2 sec (cons: 5 min 42 sec, students: 8
min 36 sec)
Median time to process second step (post-ISABEL): 1 min (cons: 42 sec, students: 3 min 42
sec)
Negative impact
7 tests scored
negative score
26 steps scored
negative score
70 significant
tests
49 significant
steps