The script concordance test as a
measure of clinical reasoning: a
national validation study
Nouh et al
The American Journal of Surgery
2012; 203: 530-534
SCT-Clinical Reasoning
national validation study
• SCT- prior single center study,
differentiated between Jr and Sr SR
• To determine validity and reliability,
administered on a national level
• Test given to 202 Residents - 9 Gen Surg
programs across Canada
• SCT distinguishes intra-operative decision
making skills of Jr. vs Sr. gen surgical
residents
SCT
• Based on script theory from cognitive
psychology
• Uses authentic clinical scenarios to
compare a trainee’s judgment skills with
those of experts
• SCT scores- intended to reflect
respondents’ competence, interpreting
clinical data under conditions of
uncertainty
Script Theory
Elements
• Clinical encounter:
– perceiving features of situation
– assessing relevant hypotheses
– checking for sx & signs that confirm or refute
hypotheses
– Using related knowledge to guide
investigations + Rx
Charlin et al Med Educ 2007: 41,1178-1184
Script Theory
Elements
• Provides explanations for how events are
processed
• Fields of computer science and
psychology
• Organization of experiences in script-like
formations
– Referred to in future
– Understand same or similar new situations
SCT
A Practical + Reliable Instrument
•
•
•
•
•
•
•
•
•
2004 Radiology
2006 Surgery
2009 Radiation oncology
Neurology
Pediatric emerg med
2011 Dermatology
ENT
Preclinical med students
2012 Geriatrics
SCT
Example
If you are considering
the
following investigation
...
and then you find . . .
you would consider
the proposed
investigation to be . . .
A V/Q scan to
R/O pulmonary
embolism
CXR
Shows areas of
collapse
∘ much less useful
∘ slightly less useful
∘ neither less nor
more useful
∘ slightly more useful
∘ much more useful
SCT- Natl Study Methods
•
•
•
•
Res level of training: post grad yrs 1-5
Jr = R1-2
Sr= R3,4,5
Test development: 9 prgrm dir; scenario,26(?)s
• Ans key: 5 pt scale, ranging from (-2),
comp contra-ind to (+2), comp indic
• 153 (?)s
SCT- Natl Study Methods
Scoring
• Every choice sel by expert→ Credit
• 22 Gen Sur experts took test
• Credit for each answer was transformed
proportionally to get a max score of 1 for
the modal experts’ choice on each item
• Stat analysis
SCT- Natl Study Methods
Results
• Scores  progressively from R1to R4 with
a dip in R5
• Test differentiated R1+2 from R3+4+5
• Site of training had sig effect on scores
Discussion
• Limitations
–
–
–
–
–
–
No observed drop in variability from R1 to R5
Choice of scale unsettled
Scores depend on cohort of experts
Influence of generalist vs subspecialist
Item production
Setting of “passing” scores
• Advantages
– Ease of administration; online
– Reliable; valid (?)