Workplace-based Assessment
(WBA) – Beyond the numbers
James Kwan
Associate Professor, Emergency
Medicine and Medical Education
Acknowledgements
•
•
•
•
•
A/Prof Lukas Kairitis
Dr Karuna Keat
Professor Jenny Reath
Professor Wendy Hu
Members Year 3/5 Committee
Outline
•
•
•
•
Competence vs. Performance
Overview WBA at UWS
Assessor judgements on performance
Delivering Feedback
Definitions
• Competence
– “Ability to do something successfully”
(Oxford Dictionaries, 2012)
• Competence in medicine
–
–
–
–
–
Ability to perform a specific clinical task
Requires integration of competencies
Measurable in terms of observable behaviour
Context specific
Process of growth and development
Professional authenticity
Miller’s pyramid
Does
Nonstandardised
assessment
(emerging)
Shows how
Knows how
Knows
Miller GE. The assessment of clinical skills/competence/performance.
Academic Medicine (Supplement) 1990; 65: S63-S7.
Standardised
Assessment
(mostly
established)
Goals of WBA
• Make a judgement on a student’s performance
of a specific task in the clinical setting
• Provide guidance to the student through
effective feedback to improve future
performance of the task
Examples
• Encounter based
– Case Discussion (Long case in the workplace)
– Mini-CEX (direct observation clinical/procedural
skill)
• Global
– Clinical Attachment Assessment
– Multi-Source Feedback (360)
– Shift Feedback Form
Year 3 Mini-CEX
• Student (or assessor) selects a patient
• Student performs a focused clinical task e.g.
history taking, physical examination,
performing a procedural skill, counseling a
patient
• Assessor directly observes the encounter
• Assessor rates the student’s performance in a
range of domains and provides feedback
• No longer than 30 min including feedback
Year 3 Mini-CEX
Challenges
• Competence is developmental vs. normative (not
satisfactory, borderline, satisfactory, good and
excellent)
• Assessors do not read the back page – use their
own internal rubric
• Reluctance to rate a student as “not satisfactory”
– “Failure to fail”
• Feedback is often limited and not as helpful as it
could be
• A “tick box” exercise resulting in students chasing
a number
Crossley, J., Johnson, G., Booth, J., & Wade, W.
(2011). Good questions, good answers:
construct alignment improves the performance
of workplace-based assessment scales. Med
Educ, 45(6), 560-569.
Construct aligned scales
• Judgement on extent to which a supervisor would
trust a trainee (or student) to perform a task
independently
• Descriptors reflect the need for close or more
distant supervision
• Different contexts will have a different way of
construing “independence”
• Greater reliability with fewer assessments
– 3 vs. 6 Mini-CEXs required to achieve a
Generalisability coefficient 0.7
Domains of assessment
Construct aligned Mini-CEX
Providing effective feedback
Effective Feedback
•
•
•
•
•
Characteristics
Perceived need for change
Technique
Creating an action plan
Coaching
Delivering Feedback
• Enable learner reaction
• Encourage self assessment
• Provide an assessment of strengths and
weaknesses
• Develop an action plan
• Document the feedback
Year 5 Emergency Medicine – Shift Feedback Form
STUDENT & ASSESSOR DETAILS
Please PRINT FULL NAMES
Student Name
Supervisor Name
Supervisor
Position
o Staff Specialist
Clinical Setting
(Please tick one)
o
Acute
o
Resus
o
Subacute
o
Triage
o
UCC
o
Application of knowledge and understanding of disease
mechanisms
Communication
History and Examination
Patient Management
Record keeping (written or electronic)
Technical Skills
Involvement in ED activities
Interpersonal relationships in the ED
Professional Behaviour
Criteria Directly
Observed
(Please tick one or
more)
o
o
o
o
o
o
o
o
o VMO
o Other
________________________
Year 5 Emergency Medicine
Feedback based on criteria observed (Please refer to the descriptors over the page for
guidance on each criteria)
The supervisor should focus on areas (minimum of two) that were performed well and also identify
areas (minimum of two) for improvement.
Agreed action:
Concerns about student progress:
Supervisor’s Signature: ___________________
o No
o Yes
Date: ___________________
Shift Feedback Form
Summary
• Competence is developmental
• Construct aligned scales may help align both
assessor judgements on performance and the
development of competence by the student
• Delivering effective feedback is an important
part of the assessment process to guide
student learning
– Action plan and coaching
Questions?
j.kwan@uws.edu.au