Objective Structured Clinical
Examination (OSCE)
Arnuparp Lekhakula M.D.,M.S.
Faculty of Medicine
Prince of Songkla University
Hat Yai, Songkhla
What is OSCE?
• Multiple stations, each testing different
competencies or tasks
• Time limit for each station
• Students rotate through all the station and
are tested on the same material
• All are judges by the same preset
standards – usually using checklists or
rating scales
Why OSCE?
• Before OSCE (1975)
• Valid?
Know how NOT show how
• Reliable?
Different patients
Different examiners
• OSCE - more valid, more reliable
• Show how
• Same task / patient / examiner /
structured marking sheet
Advantages of OSCEs
• Candidate’s knowledge and skills over
whole range of topics can be tested
• Use of different test methods
• Test not only knowledge and skills, but
attitudes as well
• Efficient use of limited resources
• More objective
• Specific profiling of each component of
performance
Climbing the pyramid......
Does
Performance assessment in vivo:
Masked SPs, Video, Audits…..
Shows how
Shows
how
Performance assessment in vitro:
OSCE, SP-based test…..
Knows how
Knows
how
(Clinical) Context based tests:
MCQ, essay type, oral…..
Knows
Knows
Factual tests:
MCQ, essay type, oral…..
Common Instruments
Direct
MCQ MEQ CRQ OSCE Observation
Knowledge +++
Problemsolving
+
Practical
skill
Interpersonal
skill
Attitudes
++
+++
+
+
++
++
+
++
+++
++
++
+
+++
++
+
+
Common Instruments
MCQ MEQ
Validity
Content
Face
Construct
Reliability
Objectivity
Feasibility
+++ ++
+
+++
+
+++
+++ ++
+++ ++
+++ +
Direct
CRQ OSCE observation
+++
++
+
++
++
+
++
+++
+
+++
+++
+
+++
+++
+++
++
++
++
Limitations of OSCEs
• Labor-intensive, lengthy preparation
• Costly
• Require trainees to perform isolated
aspects of the clinical encounter
• Rely on task specific checklists and
scoring
• What can be simulated constrain the
type of patient problems used
• Need more observation skills of staff
Limitations of OSCEs
• Standardization of simulated patients
and examiners
• Low inter-station correlation
• Test security
• Repetitive - boring
Factors Leading to Lower Reliability
• Too few stations or too little testing
time
• Checklists or items that don’t
discriminate
• Unreliable patients or inconsistent
portrayals by standardized patients
• Examiners who score idiosyncratically
• Administrative problems
What Assessed by OSCE
• Various clinical skills – history taking,
physical examination, technical procedure
(on model), communication, interpersonal
skills
• Knowledge and understanding
• Data interpretation
• Problem solving
• Attitudes
Types of OSCE stations
• Process - various skills, attitudes
• Product – data interpretation, diagnosis,
problem solving, report writing, order sheet
writing, drug description
• Mixed – process and product
Components of OSCE station
• Stem with clearly stated task
• Checklist
- Appropriate for level of training
- Task-based
- Observable
• Training information
Stem
John Smith, aged 37, arrived in the emergency room
Complaining of acute abdominal pain that began
12 hours previously
Instruction to student In the next five minutes, conduct
a relevant physical examination
Checklist
Stem
Examiner to fill in box to each item that student successfully
completes
Marks
Drapes patient appropriately
2
Inspects abdomen
1
Auscultation abdomen
1
Percussion abdomen
1
Asks patient for the location of pain
1
Start to palpate abdomen from the
area which is not pain
2
Lightly palpates each quadrant
2
Deeply palpates each quadrant
2
Checks for peritoneal irritation
2
Training Information
Stem
Checklist
History of pain
The pain started 12 hours ago
Symptoms
The pain is in the right lower quadrant, at “at least 9”,
and is constant. His abdomen is tense even when palpate
lightly. With deeper palpation, there is guarding in the RLQ,
and McBurney point is acutely tender.
Obturator (raising right knee against resistance) and psoas
signs (extension of right leg at hip-kicking backward) are
positive.
Steps in Developing OSCEs
•
•
•
•
Assembling working groups
Defining competencies/content assessed
Developing blueprint
Writing an OSCE station – process or
product
- Writing a scenario
- Develop instructions to the candidate:
precise, concise, unambiguous
- Write instructions to the simulator/SPs,
a script for role-playing
- Draw up a list of resources: space,
people, equipment, materials
Steps in Developing OSCEs
•
•
•
•
•
•
Developing checklist
Draw a scoring scheme
Revised by committee
Setting standards/passing score
Pilot testing
Reviewing and rewriting
Table of Specification
Area tested
A1
A2
A3
A3
A5
A6
A7
A8
…..
Hx
3
PE
3
Procedures
X
X
X
X
4
Data
interpret.
X
X
X
X
4
Problemsolving
2
Com skill
1
Pt Ed
1
Attitudes
2
Total
2
3
3
2
3
3
2
2
20
Organizing an OSCE
• What is to be assessed – produce a table
of specification
• Duration of stations – 4-15 minutes, mostly
5 minutes
• Number of stations – 15-20 stations
• Format of stations – isolated, linked
• Use of examiners – fully briefed prior to
examination
• Preparation of checklists
• Organization of examination
OSCE Station
1
Basic
3
10
9
Doubletime
2
4
5
8
7
6
3
4
3
4
2
1
10 min
2
Linked
1
2
Organizing an OSCE
• Assigning priority
• Resource requirement – examiners,
patients (real, simulated), equipment,
paperwork
• Plan of examination and direction
• Change signal
• Records
How to Improve Checklist
• Stem
- Clear
- Observable
- Not to long
• Overall
- Not too long
• Rating scale
- More clarification of each scale
- Done, complete/Done, incomplete/
Not done
- Rater training
Observation
• Direct
• Indirect
- One-way mirror
- Monitor
- Video
Instructions to Simulated Patient
• Situation
• Information about illness
• Role playing
- Facial/emotional/non-verbal expression
- Response to students :
normal/abnormal
- Questions asked to students
- Answers prepared to students’
questions
• Dressing/Costume
• Make-up
Summary of OSCE stations
Station
Competency
Area
tested
Linked
station
Process Product
SP
Check
-list
Examiner
Equipments
Timeline for OSCE
• 8 wk
• 6 wk
• 3 wk
• 2 wk
Select date and appoint
overall coordinator and
site coordinator
Decide on the station tasks,
book site and refreshments,
make local arrangements
Review all station details
Have all station paperwork
printed, signs made,
equipment prepared, remind
examiner
Timeline for OSCE
• 24 hr
• 2 hr
• 1 hr
• 30 min
Walk through examination site
with coordinator
All coordinator on-site
Final briefing of examiners
and SPs
All examiners and SPs at
stations
Students’ orientation/briefing
Standard Setting - Angoff Method
• Performance levels at “minimally
acceptable” or “borderline” is used to
derive cut-off score
• Undertaken by a panel (usually a minimal
of 6)
• Judges independently estimate what
proportion (%) of minimally competent
examinees would answer the item correctly
Example - Angoff
Items
Full
mark
Judge
Angoff
Score
Explain to patient
1
80%
0.8
Position of patient
1
80%
0.8
Inspection of skin
2
80%
1.6
Temperature change
1
60%
0.6
Pulsation : Femoral
1
80%
0.8
Popliteal
1
50%
0.5
Posterior tibial
1
50%
0.5
Dorsalis pedis
1
50%
0.5
Capillary refill
1
50%
0.5
Total
10
6.6
Example - Angoff
Items
Not
done
Done
poorly
Done
statis.
Explain to patient
0
0.5
1.0
Position of patient
0
0.5
1.0
Inspection of skin
0
1.0
2.0
Temperature change
0
0.5
1.0
Pulsation : Femoral
0
0.5
1.0
Popliteal
0
0.5
1.0
Posterior tibial
0
0.5
1.0
Dorsalis pedis
0
0.5
1.0
0
0.5
1.0
Capillary refill
Maximum total = 10.0
Angoff score = 6.0
Standard Setting - Borderline Method
• Marking form : checklist + global rating
• All categorized ‘borderline’ students
Stu
1
2
3
4
5
6
7
8
9
Score
80
90
60
70
75
55
80
65
85
Global
rating
S
SS
BU
BS
S
U
S
BS
S
• Mean score of borderline group
Passing score = (60+70+65)/3 = 65
Conclusions : OSCE
• What?
Stations + tasks + checklist
• Why?
More valid, more reliable, more objective
• How?
How to organize?
How to analyze?
OSCE – Task to Do Ahead (1)
•
•
•
•
•
•
•
•
Create blue print
Set timeline
Get authors for case-writing workshop
Review and finalize case
Arrange workshop on setting standards
Recruit standardized patients/examiners
Train SPs
Print scoring sheet, mark signs
OSCE – Task to Do Ahead (2)
• List all supplies for set-up of OSCE stations
• Remind everyone of date
• Make sure students have all the
information
• Plan of the examination day
- Diagram of station layout
- Directions for examiners, SPs and staff
- Registration table for examiners, SPs and
students
- Timing and signals (stopwatch, wristles or
bell)
- Procedures for ending examination