Leighton Seal
Workplace Based
Assessments for
Undergraduates:
Training for
Assessors
Responsible Examiner for WBA and Consultant
Endocrinologist
Dr Judith Ibison: Senior Lecturer Primary Care & MBBS Personal and
Professional Development Theme lead,
Dr Elizabeth Miles:
Senior Lecturer Staff Development and CD PGCHE
Dr Clare Shoults:
T Year Lead and Consultant Acute Medical Unit
1.
2.
3.
4.
5.
6.
Workplace Based
Assessments for
Undergraduates:
Training for
Assessors
Context of assessment for students
WPBA
How to supervise and assess WPBA
Common pitfalls
Video – try rating one
Feedback
Outcomes 1 − The doctor as a
scholar and a scientist
The
graduate will be able to
– Apply to medical practice biomedical scientific
principles, method and knowledge relating to:
anatomy, biochemistry, cell biology, genetics,
immunology, microbiology, molecular biology,
nutrition, pathology, pharmacology and physiology.
– Apply psychological principles, method and
knowledge to medical practice.
– Apply social science principles, method and
knowledge to medical practice.
– Apply to medical practice the principles, method
and knowledge of population health and the
improvement of health and healthcare.
– Apply scientific method and approaches to medical
research.
Outcomes 2 − The doctor as a
practitioner
The
graduate will be able to
– Carry out a consultation with a patient.
– Diagnose and manage clinical presentations.
– Communicate effectively with patients and
colleagues in a medical context.
– Provide immediate care in medical emergencies.
– Prescribe drugs safely, effectively and
economically.
– Carry out practical procedures safely and
effectively.
– Use information effectively in a medical context.
Outcomes 3 − The doctor as a
professional
The
graduate will be able to
– Behave according to ethical and legal
principles.
– Reflect, learn and teach others.
– Learn and work effectively within a
multiprofessional team.
– Protect patients and improve care.
Why use workplace based
assessments in medical
students?
Importance
of clinical skills
Deficiencies in clinical skills
Problems with traditional
assessments
Effectiveness of feedback
Importance of clinical skills
Interview
and PE are primary source of
diagnostic data
Several studies document deficiencies
Data gathering errors are the most
common source of diagnostic errors
Good communication skills improve health
status of patients, increase efficiency of
care and REDUCE likelihood of litigation
Feedback
Feedback is sought after and valued by
medical students and trainees
 Feedback is a strong motivator for behaviour
change and has been shown to improve
confidence and clinical performance
In meta-analyses of the impact of various
educational strategies on medical education
outcomes, feedback alone effective is effective
in 71% of studies (Veloski et al. 2006)

Resources
http://www.elu.sgul.ac.uk/wba/
How to use the MBBS
Workplace-based
Assessment (WBA) resource
All
users of the MBBS WBA materials on
this website are strongly recommended
to:
watch the “Introduction to WBA” video
watch the “Giving Effective Feedback”
cluster of 3 videos (which set the scene
for effective student learning through
formative assessment)
How to use the MBBS
Workplace-based
Assessment (WBA) resource
Select
additional resources to view by following
the guidance after the statement (1-3)
1.
“I am unsure about WBA and what is expected
of the assessor and the MBBS student.”
2. “I already know about/have taken part in WBA
at postgraduate level and want to know what
standard to expect of the MBBS students.”
3. “I already know about WBA and what level to
expect, but want to check about formative
assessment and giving feedback to MBBS
students”.
How to use the MBBS
Workplace-based
Assessment (WBA) resource
Watch
each of the three clusters of clips
– WBA
– WBA Examples
– Feedback
(there
are points and questions to
consider beside each clip):
– a. Case-based Discussion
– b. Mini Clinical Examination
– c. Directly Observed Procedural Skill
How to use the MBBS
Workplace-based
Assessment (WBA) resource
There
are downloads attached to clips
and under the Documents tab:
– “Professional Behaviour & Clinical Practice
Assessment Guide”
– “Workplace-based Assessment” PowerPoint
– “Changes to Assessment of St. George’s
Students on Clinical Attachments”
– WBA assessment forms
Effectiveness of feedback
Critical
to learning
Significant influence on
achievement – meta-analyses show
efficacy of feedback alone in
improvement
But
traditionally students are rarely
observed….
Promoting effective
feedback
To facilitate the student’s own selfreflection.

Prevent the student having
misconceptions about performance.

Ascertain whether there is
disagreement or resistance to the
observer’s comments.

https://intranet.sgul.ac.uk/teach/Clinical%20skills%20videos/cbd/cbd_bad.wmv
Problems that can occur
with giving feedback
include:
Non-specific
feedback ‘clerk more patients’
Giving
personal or emotional response to
performance
Fear
of criticising the student.
Fear
of emotional responses from student.
No
time
https://intranet.sgul.ac.uk/teach/Clinical%20skills%20videos/cbd/cbd_good.wmv
Promoting effective
feedback
Optimise
Start
the value of feedback by:
with the learner’s self-assessment of what
went well and what could be improved.
Reinforce important areas and points of specific
feedback.
Review understanding and feeling in response to
those areas and points.
Formulate a specific and realistic plan with clear,
direct goals.
Workplace based
assessments
WORK BASED
ASSESSMENTS
Objective Structured
Clinical Examination
Short Answer Question /
Extended Matching Item
Single Best Answer
Miller’s pyramid
WPBA are one component of a
comprehensive assessment strategy
Agreement
of learning
objectives
and action
plan for
learner
Constructive
feedback by
assessor and
rating
Agreement
between
learner and
assessor on
focus of
assessment
Suitable
WPBA tool
WPBA:
‘the
learning
moment’
Assessor
facilitates
self appraisal
by learner
Self
assessment
by learner
Learner
demonstrates
skill by on or
about real
patient
Agreement
of learning
objectives
and action
plan for
learner
Constructive
feedback by
assessor and
rating
Agreement
between
learner and
assessor on
focus of
assessment
Suitable
WPBA tool
WPBA:
‘the
learning
moment’
Assessor
facilitates
self appraisal
by learner
Self
assessment
by learner
Learner
demonstrates
skill by on or
about real
patient
Agreement
of learning
objectives
and action
plan for
learner
Constructive
feedback by
assessor and
rating
Agreement
between
learner and
assessor on
focus of
assessment
Suitable
WPBA tool
WPBA:
‘the
learning
moment’
Assessor
facilitates
self appraisal
by learner
Self
assessment
by learner
Learner
demonstrates
skill by on or
about real
patient
Agreement
of learning
objectives
and action
plan for
learner
Constructive
feedback by
assessor and
rating
Agreement
between
learner and
assessor on
focus of
assessment
Suitable
WPBA tool
WPBA:
‘the
learning
moment’
Assessor
facilitates
self appraisal
by learner
Self
assessment
by learner
Learner
demonstrates
skill by on or
about real
patient
Agreement
of learning
objectives
and action
plan for
learner
Constructive
feedback by
assessor and
rating
Agreement
between
learner and
assessor on
focus of
assessment
Suitable
WPBA tool
WPBA:
‘the
learning
moment’
Assessor
facilitates
self appraisal
by learner
Self
assessment
by learner
Learner
demonstrates
skill by on or
about real
patient
Agreement
of learning
objectives
and action
plan for
learner
Constructive
feedback by
assessor and
rating
Agreement
between
learner and
assessor on
focus of
assessment
Suitable
WPBA tool
WPBA:
‘the
learning
moment’
Assessor
facilitates
self appraisal
by learner
Self
assessment
by learner
Learner
demonstrates
skill by on or
about real
patient
Agreement
of learning
objectives
and action
plan for
learner
Constructive
feedback by
assessor and
rating
Agreement
between
learner and
assessor on
focus of
assessment
Suitable
WPBA tool
WPBA:
‘the
learning
moment’
Assessor
facilitates
self appraisal
by learner
Self
assessment
by learner
Learner
demonstrates
skill by on or
about real
patient
What is a workplace based
assessment?
WBA are observed assessments
of clinical practice in the workplace,
with feedback to the learner on the
strengths and weaknesses of his or
her performance.

The key is observed practice and
developmental feedback.

Workplace based
assessments
Vehicles
for offering direct,
timely and relevant feedback
High
validity,
High acceptability,
High educational impact.
? Cost –effectiveness

Poor reliability
Key components of a
WPBA
Designed
to assess a
specific skill/task
Use rating scales – to
provide information on
specific behaviours
and the quality and
fitness of their
performance (vs
checklists)
Opportunity for
feedback to improve
subsequent
performance
Skills
History
Examination
Diagnostic
Reasoning
Management – investigations and
interventions
Ethics
Practical procedures
Acute or chronic contexts
General or specific
Personal behaviours relevant to clinical
practice
Learner
Training programme
Public/Employer
Gain feedback
Demonstrating trainee
progression
Ensuring trainees
are competent
Motivation through positive
feedback
Contributing evidence for
summative decision making
Encouraging
excellence
Encourage aspiration to
mastery
Identifying doctors in
difficulty
Promoting learning and
informing learning
objectives
Providing data to support
decisions involving trainees
in difficulty
Providing evidence to
inform annual review of
competence progression
Providing information when
designing remediation
packages
Demonstrating progression
Identifying patterns of
behaviour
Academy of Medical Colleges 2009
WBA: Assessment
challenges: reliability
Inter-observer
variability
Intra-observer
reliability
Case-specificity
Therefore
we are using them formatively
Mini CEX
Should
be based on
– observed focused history taking,
– observed examination
– observed higher communication skills
Main
domains
– history taking,
– physical examination skill,
– clinical judgement and synthesis,
– humanistic skills
Also
overall rating
Different details in different schemes
– number of items and rating scales
How many mini-CEX do you
need?
In
T year
2 CEX per long clinical attachment
Surgery
GP
Medicine
1 In AMU week
How many mini-CEX do you
need?
In
P year
2 CEX per long clinical
attachment
1
–
–
–
–
–
O+G
Psychiatry Medicine
Surgery
Senior Medicine
Paediatrics
Short Attachments
Exceptions
– AMU week
– Speciality
– Neurology 2 CEX
(Including GCS)
How many mini-CEX do you
need?
In
F year
1CEX per long firm
Surgery AHO
Medicine AHO
GP AHO
Direct Observation of
Procedural Skills
DOPs
allows students to move from
procedures learnt in a skills lab to
performance in real patients.
Performance
of practical procedures
involves technical skills but also
communication, patient approach and
attitudes to analgesia and patient
comfort
DOPS 4P’s
Preparation
Performance
Practice
Based Discussion
Progression
DOPS Assessment
Preparation
for DOPS
Proficient in SIMS
Observation in work place of a
competent practioner
Consent
DOPS Assessment
Performance
See
Clinical Skills LOBS
Pre clinical and T year LOBS for
clinical skills
DOPS Assessment
Practice
Based Discussion
Indication
Safety
Interpretation of result
DOPS Assessment
Progression
Reflection
on performance of the
skill in a clinical setting
Learning needed
Further assessment
DOPS
DOPS
compulsory but assessment
formative.
Practical
procedures must be supervised
and assessed by F1 or above, or nonmedic who performs procedure routinely
in clinical practice e.g. nurse or e.c.g.
technician
Standard: Procedurally correct and safe
Supervising DOPS
Are
you confident to do the procedure?
If not refuse…….
Is the student ready to perform the
procedure on a real patient?
Talk through whether the student
knows how to do the procedure…..can
they accurately describe the detail of
the process……if not refuse!
End of Firm Assessment
Previously
had problems
Benchmarking results
Clinical Practice Grades by
attachment
AHO
Medicine
AHO
Surgery
AHO
GP
A&E
CCA
Excellent
214 (73%)
165 (56%)
77 (26%)
n/a
160
(54%)
n/a
Acceptabl
e
80
129
217
294
133
294
Public Health
Clinical Practice: AHO Surgery by
site
No. of
students
Clinical Practice: AHO medicine by
site
No. of students
No of
students
Clinical Practice: A+E grades by
clinical site
GP: Clinical Practice grades by
site
No of students
Old Assessment Criteria
Ex - Excellent
A - Acceptable
Excellent T year knowledge
of basic and clinical science
with ability to apply to
unfamiliar problems.
Adequate T year knowledge of
basic and clinical science,
applied to clinical practice.
Good thorough historytaking and consistent fluent
examination of all major
systems.
Competent basic examination
and adequate history-taking,
not necessarily with fluency.
Demonstrates an excellent
for T year, broad analytical
approach to diagnostic
problems
Adequate T year standard
approach to common
diagnostic problems.
Excellent, above average T
year, consideration of a
broad variety of treatment
options and able to explain
preferred option.
Adequate T year approach to
considering treatment options
Excellent awareness and
articulation of wider social
and societal issues.
Adequate T year awareness of
broader psychosocial issues.
CC - Cause for Concern
U - Unacceptable
□
Very limited knowledge
of basic and clinical
science.
□
□
Unsystematic
examination with
limited skill and
incomplete histories
without diagnosis.
□
□
Able to describe key
features of common
disease but cannot
explain them.
□
□
□
Aware of common
treatments but not
rationale.
Limited awareness of
psychosocial issues.
□
□
Lacks knowledge of basic
and clinical science
Unable to do basic
examination or formulate
sensible diagnoses or
rational management.
Unable to explain
disease processes.
Not aware of very basic
treatment groups
Cannot identify
psychosocial issues.
Other
..........................................................
New End of Firm Sign Off
Doctor as a scholar
Knowledge
of basic and clinical
science, including pharmacology, as
it relates to the clinical practice
Correct identification of important
psychosocial and ethical aspects of
clinical medicine:
Doctor as a practioner
Information
Gathering
–History content and technique
–Examination content and technique
–Patient centredness: communication,
attention to modesty and pain, disability.
Doctor as a practioner
Synthesis
of Clinical Problem
–Case synthesis: Correctly identifies and
prioritises differential diagnoses
–Reasoned and detailed approach to
investigation choices, or interpretation
–Treatment Plan: Correct content and
prioritisation with reasoned justification.
–Structured and concise medical record
keeping
Doctor as a practioner
Clinical
Management
–Understand how to prescribe drugs
safely, effectively and economically
–Carry out practical procedures safely
and effectively
–Communicates effectively with patients
–Communicates and colleagues in a
medical context
Advantages
Aligns
with Training Doctors
Aligns with postgraduate
benchmarking
Allows a more tangible assessment
of what stage the student is at in the
course
Gives more extensive structured
feedback
Summary
Clinical
Skills and professionalism
are core to a doctors practice
DaP is the key to formally assessing
this by peer evaluation
Students appreciate effective and
timely feedback
Benchmarking is problematic
Summary
The
new assessments
Use assessment tools based on validated post
graduate training assessment
– Trainers/trainees already familiar with them
New
system will be electronic
– Rapid feedback
– Easier
– More secure
Benchmarking
on expected performance for
stage of training
– Should allow more standarization of assessments with
better QA