Trainees` Guide to Workplace Based Assessment

advertisement
Trainees’ Guide to Workplace Based
Assessment
Introduction
The College curriculum for specialist training in Psychiatry is
supported by an extensive assessment programme comprising both
workplace based assessments (WPBA) and formal MRCPsych
examinations. Full details of the MRCPsych examinations appear on
the College website (www.rcpsych.ac.uk).
The guide describes WPBA’s, outlines their purposes, tells you how
many you need to get done and tells you how to plan and organise
them. You should read it in conjunction with the guidance about the
specific tools that is available on the HcAT / RCPsych website
(www.hcat.nhs.uk/rcpsych).
The College Curriculum is the basis of the Specialty Training
Programme in Psychiatry. It has been approved by PMETB and
comprises the Core and General module, 6 specialty modules, and 3
sub-specialty modules.
It uses the same WPBA methods
throughout. The entire curriculum is mapped to Good Medical
Practice. The Core and General module indicates the phase of
training through the use of colour: red = ST1; gold = ST2&3; violet
+ ST4&5; and green = ST6. The colour codes are used in all
documents concerned with a specific phase of training, including the
WPBA forms. Documents that are not specific to a particular phase
of training (such as the Core and General module and some of the
WPBA forms) are colour coded blue. You should use the curriculum
as an electronic hyperlinked document.
There is also a list of competencies for each training level. Within
each list there are links back to the curriculum. This list of
competencies is a key document. You must familiarise
yourselves with this in order to manage your training with
the help of your educational supervisor.
Underlying Principles
Workplace based assessments must comply with the following:




They must focus on performance (i.e. what you actually do in
the workplace)
They must be evidence based
Evidence must be triangulated whenever possible (i.e. it must
be provided by different assessors, on different occasions, and
if possible using different methods)
Records must be permanent – assessment forms must not be
lost or destroyed (this is achieved by online storage).
Purpose
The purpose of the workplace based assessments is threefold.
Firstly, they have a formative function as the basis for feedback and
educational planning. This includes helping identify any specially
focused teaching and learning that might be necessary to fill gaps or
resolve weaknesses. You should have WPBA’s completed regularly
throughout your placement to demonstrate effective review and
appraisal. Therefore, do not leave them until the end of each stage
of training or clinical placement.
Secondly, although individually they have no ‘pass’ or ‘fail’ score
taken together WPBAs contribute to the Annual Review of
Competence Progression (ARCP), which has a summative function
providing evidence of the attainment of competencies.
Thirdly, participating in a specific number of WPBAs is mandatory
for eligibility for the MRCPsych examinations (see later).
Assessments
There are 8 workplace based assessments available. These are
described in detail in the trainee and assessor guidance for each
WPBA tool and are merely outlined below.
o Assessment of Clinical Expertise (ACE)
The assessor observes a whole new patient encounter in
order to be able to assess your ability to take a full history
and mental state examination and arrive at a diagnosis and
management plan.
o Mini-Assessed Clinical Encounter (mini-ACE)
The assessor observes part of a patient interaction, for
example history taking or negotiating a treatment plan,
and rates your performance.
o Case-based Discussion (CbD)
Trainees’ Guide to Workplace Based Assessment
October 2007
© Royal College of Psychiatrists 2007
2
You select two sets of notes of patients you have recently
seen and the assessor picks one to discuss. The discussion
will allow demonstration of clinical decision-making and the
application of clinical knowledge.
o Case Presentation (CP)
This tool can be used when you give clinical presentations
and involves assessment of domains such as presentation
skills and interpretation of evidence.
o Journal Club Presentation (JCP)
This can be used when you present a journal article and
covers domains such as analysis and critique and
answering questions.
o Directly Observed Procedural Skills (DOPS)
This has more limited use in psychiatry compared to other
areas of medicine but can used in situations such as
administering ECT.
o Mini-Peer Assessment Tool (mini-PAT)
It allows co-workers to assess your attitudes
behaviours and ability to work well with colleagues.
and
o Assessment of Teaching (AoT)
This is a new tool that has been developed after feedback
from the pilot programme. It allows an assessment to be
made of teaching skills and may relate to a lecture, tutorial
or small group teaching session that you lead.
Requirements
The requirements for the number of WPBAs to be completed in ST
1-3 are:
WPBA
Minimum number required
per year
ACE
2 in ST 1, 3 in ST 2 and
3 in ST 3
mini-ACE
4
CbD
2
Trainees’ Guide to Workplace Based Assessment
October 2007
© Royal College of Psychiatrists 2007
3
mini-PAT
2
WPBA
Recommended number per
year
CP
1
JCP
1
DOPS
As the opportunity arises
AoT
As the opportunity arises
The guidelines in the above tables should be regarded as the
minimum number of WPBA’s: many trainees will need to do more,
for example to provide additional evidence of achievement in a
particular area of professional practice. It is important to again note
that there is no ‘pass’ or ‘fail’ score for WPBA. Therefore for exam
eligibility purposes you are only required to demonstrate that you
have undertaken the required WPBA, not that you have achieved a
specific ‘score’.
WPBA and Examination eligibility
The requirements for each examination are summarised in the table
below. These requirements may be subject to change and it is
essential that you consult the College website for the most
up to date information. Trainees who enter ST 1 must fulfil the
requirements below to be eligible to sit the MRCPsych examinations.
There are transitional arrangements for eligibility for the MRCPsych
examinations which can be found in full on the College website. All
trainees who enter run through training at ST 2 or ST 3 and have
passed Part 1 may take Paper 3 and the OSCE under the previous
eligibility criteria (duration of training completed not WPBA). All
trainees who enter run through training at ST 2 or ST 3 who have
not passed Part 1 may take Paper 1 and Paper 2 under the previous
Trainees’ Guide to Workplace Based Assessment
October 2007
© Royal College of Psychiatrists 2007
4
eligibility criteria but must complete the new criteria for the OSCE to
take Paper 3 and the OSCE.
Exam
WPBA requirement
Paper 1
4 mini-ACE at ST 1 level
2 CbD at ST 1 level
Paper 2
4 mini-ACE at ST 2 / 3 level
2 CbD at ST 2 / 3 level
Paper 3
Cumulative criteria of Paper One
and Paper Two
(8 mini-ACE and 4 CbD)
OSCE
8 ACE across a range of
specialties
(2 in ST 1, 3 in ST 2, 3 in ST 3)
For trainees at the level of ST4 and above tools are currently being
developed and adapted to assess higher level performance and
competencies such as leadership skills. The existing WPBA tools
should be used to guide educational development and as evidence
for your ARCP. You should complete two rounds of mini-PAT a year
and in addition aim to undertake one WPBA a month.
You must note that, since they form the basis for feedback and
educational planning, you must arrange WPBA’s at appropriate
intervals throughout your training rather than leaving them until the
end of each stage. In particular, we strongly suggest that within the
early stages of each phase of training or clinical placement (i.e.
within the first month) you have a mini-ACE or ACE and a CbD. This
is because these tools reflect closely the expected day-to-day work
of a junior doctor in Psychiatry both in terms of the assessment of
cases and clinical reasoning and judgement.
The purpose of these early assessments is to identify what you are
doing well and in which areas you need to develop. They contribute
valuable information to the educational planning of your placement.
Therefore it is most unlikely that assessments made early will
produce ratings consistent with performance at the standard
Trainees’ Guide to Workplace Based Assessment
October 2007
© Royal College of Psychiatrists 2007
5
expected for the completion of the stage of training.
Scale Ratings
All the WPBA forms use a 6-point Likert-type rating scale. The
standard for completion of each stage of training (ST1, 3, 5 and 6)
corresponds to a rating of 4. Higher ratings indicate performance
above the minimum standard for completion of that stage; lower
ratings indicate performance below the required completion
standard. Since when you are in the early part of any phase of
training you will probably not be performing at the standard
required for completion of that phase, (and will, therefore, often
receive ratings below 4), most assessment forms have an additional
item where the assessor can indicate your global performance
relative to your stage of training. On the other hand, there might be
some occasions when your performance exceeds that indicated for
your current stage of training. In such circumstances you can
choose to be assessed using criteria (and rating forms) for a higher
stage of training and, if you achieve ratings of 4 or more, can record
this in the appropriate portfolio. Thus, for example, an exceptional
ST1 Trainee might choose to be assessed on their consultation skills
at ST2/3 level and, if rated at 4 or better, would record this in their
ST2/3 portfolio as well as in their ST1 portfolio.
Electronic list of competencies
Progress and Attainment’)
(formerly,
‘Portfolio
of
Each phase of training (ST1; 2/3; 4/5; and 6) has a designated List
of Competencies: Log of Progress and Attainment. You must keep
the Curricula and associated documents in a single folder with their
names unaltered and you must enable macros for the hyperlinks to
work. The name that you give the folder is irrelevant. Although any
of the documents can be printed, they are large (especially the core
curriculum) and incomprehensible unless viewed in colour.
You can use the boxes in the List of Competencies to describe the
evidence you wish to use to demonstrate your achievement of that
competence. The evidence may consist of one of the WPBA tools but
may also include other evidence such as reference to supervision
notes or to reflective notes. You can print the electronic list of
competencies annually and filled it in by hand, but it is better to
complete it electronically and print it out annually. Then you can
add it to your paper portfolio prior to your ARCP. This means that
for stages of training that span two years (e.g. ST2-3) you should
Trainees’ Guide to Workplace Based Assessment
October 2007
© Royal College of Psychiatrists 2007
6
have a half completed list of competencies printed out at the end of
ST2 and a completed one at the end of ST3.
Paper portfolio
This replaces the previous College logbook. It allows you to collate
other evidence in addition to your WPBA. It will be available to
download free from the College website or a commercially printed
copy can be purchased (further details on the College website).
It has several sections including:




self-appraisal
professional development plan
reflective learning
record of publications, audit, research, teaching etc.
Competence Attainment
All trainees will have an Annual Review of Competence Progression
(ARCP) organised by their deanery. This is described in the Gold
Guide and replaces the RITA process. The key elements of this
review are:
 appraisal
o Educational
o NHS
 assessment of performance
o workplace based assessments
o experience demonstrated e.g. from portfolio including
audit, research and other aspects of non-clinical
competence
 planning.
Achievement of competencies will be considered in the Educational
Supervisor’s Report and the educational supervisor will consider
WPBA and other evidence such as examinations passed and
experiential outcomes recorded in the portfolio. The competencies
will be considered under 11 domains:
 providing a good standard of practice and care
 decisions about access to care
 treatment in emergencies
 maintaining good medical practice
 maintaining performance
 teaching and training, appraising and assessing
 relationships with patients
Trainees’ Guide to Workplace Based Assessment
October 2007
© Royal College of Psychiatrists 2007
7




dealing with problems in professional practice
working with colleagues
maintaining probity
ensuring that health problems do not put patients at risk
The Educational Supervisor’s Report and the College Tutor’s Report
will be submitted for consideration by the ARCP. It is the ARCP that
validates the achievement of competence after considering the
presented portfolio information. It is important to note that as the
ARCP considers a range of evidence, it is possible for you to have
performed well in your WPBA’s but be considered by the ARCP panel
to require further development. The outcomes for ARCP are
described in the Guide to Postgraduate Specialty Training in the UK
(Gold Guide), www.mmc.nhs.uk/download/Gold_Guide_140607.pdf.
Carrying out workplace based assessments
You are responsible for organising your assessments. You should
discuss the areas in which you need to be assessed with your
Educational Supervisor and plan your WPBA’s accordingly.
Remember that you will need to be assessed by a range of different
assessors in different situations to ensure triangulation of evidence.
Don’t rely on your Educational Supervisor to perform all the
assessments! Also remember that you must ask for the patient’s
agreement when you perform a WPBA that involves a patient.
There is an online system for the completion and storage of WPBA’s.
You can register for this through the College website. After you
register you will be able to access the forms relevant to the
mandatory WPBA’s (i.e. ACE, mini-ACE, CbD and mini-PAT). Your
assessors must register with the system and complete these
assessments online. Your assessors can continue to use the paper
forms of the other WPBA tools, which may be downloaded from the
College website. You may already have completed a number of the
mandatory WPBA’s on paper; you must now ask your assessors to
enter these assessments online, so they may be stored in the
system. You can keep other assessments that have been completed
as paper forms and can store them in your portfolio.
Trainees’ Guide to Workplace Based Assessment
October 2007
© Royal College of Psychiatrists 2007
8
Download
Study collections