Two countries separated by
a common language? –
Assessment, clinic and UK
higher education
Philip Plowden
Kevin Kerrigan
www.northumbrialawschool.co.uk
PP Intro
Some assessment issues in
clinic
• Fear that assessment tends to be subjective / arbitrary
• Concern about the relevance of the supervision process
in assessment
• Impact of the supervisor / student relationship on the
grade
• Is it possible to have a fair assessment based on very
different clinical experience (clients, area of law, volume
of work)?
• How can consistency be encouraged between clinical
faculty with different professional backgrounds and
pedagogic attitudes?
• Effectiveness of internal and external moderation in
relation to work they have not supervised
Looking at an example of
assessment
Context: Assessment in
Northumbria’s clinic – Student Law
Office
•
•
•
70% = practical work done during the
year, as evidenced by student personal
file portfolio.
30% = three pieces of reflection on
student work, handed in as part of the
portfolio.
Skills competency assessments in (i)
Interviewing/advising and (ii) Legal
writing = pass / fail.
Interview competency –
students should be able to:
•
•
•
•
•
•
•
•
•
prepare for the interview
deal with appropriate client care and professional conduct issues
allow the client to explain his or her concerns
identify the client’s goals; help the client reach priorities
elicit relevant information
use appropriate listening and questioning techniques
determine what further information is required
Identify courses of action and the legal and non-legal consequences
assist the client to make a decision regarding the best course of
action, including the costs, benefits and risks of that course of action
• agree the future action to be taken by both parties
• accurately record the interview, confirm instructions and action
• establish a professional relationship with the client
The video –
what we want you to do
1. Please use the competency assessment
criteria to decide whether this interview is
likely to be a pass or a fail
2. Identify any particular strengths or
weaknesses in this performance that
would influence your view of its quality
3. Identify any further information you think
you need to make you more confident in
the reliability of your assessment exercise
Competency Criteria
Pass or fail?
•Prepare
•Client care
•Client concerns
•Client goals
•Relevant info
•Listening;
•questioning
•Further info
•Courses of action
•Future action
•Record / confirm
•Professional
relationship
The value and limits of
competency assessment
Value
• Quick
• Simple yes or no
• Easy to complete
• Confirms student can
perform basic skills
• Helpful for quality
control of the legal
profession – ensure
minimum standards
Limits
• Only provides a
snapshot
• Fails to address
student development
• Crude measurement
• Fails to distinguish
student performance
• Fails to consider
lawyering ability
holistically
The development of grade
descriptors
• The key aims of the descriptors are
– (i) consistency - help ensure more
consistency in the marking process by having
a clear set of standards distinguishing
different levels of student performance and
– (ii) transparency - assist students to
understand how they can improve their
performance and therefore enhance their
grade in clinic
What are our grade descriptors?
• They describe different levels of student
performances by the end of the
programme: fail, lower second, upper
second, first class and strong first class
• The UK Quality Assurance Agency
Benchmark for Law:
– (i) Autonomy and
– (ii) Ability to Learn.
• the “key feature of graduateness”.
Autonomy and ability to learn
Autonomy includes
• ability to identify and
apply law with
accuracy
• ability to plan and
progress client cases
• professional
commitment to the
client’s best interests;
• efficiency in
managing the case.
Ability to learn includes
• ability to recognise areas
of weakness and to
improve
• ability to build on existing
areas of strength
• ability to reflect
effectively and relate
experiences to wider
academic, professional
and societal contexts.
Key descriptor 1st class
• A student who is able to identify legal issues with
accuracy and then to implement effective
strategies for achieving the client’s goals. Such
students may make errors of law, or may
demonstrate initial weaknesses in their legal
skills, but will show the ability to learn from
experience so that their work is of an
increasingly high standard. The supervisor will
be able to have an increasing confidence in the
student’s ability to operate with a higher degree
of autonomy than might normally be expected.
Extracts from detailed descriptor
• “This student requires little active prompting and can be
expected to approach the supervisor with suggestions
that have been thought through and are often highly
developed.”
• “The student is able to identify relevant legal principles
with accuracy, and where appropriate will show the
ability to think across legal “areas” so as to view the case
holistically.”
• “There will be a high level of commitment to the client
and empathy with their situation.”
• “The student’s written reflections demonstrate the ability
to assess both the cases that they have conducted and
their own performance in terms of wider academic,
professional and socio-economic contexts.”
Key descriptor - fail
• This student requires close supervision even towards the
later stages of the SLO. This may be because of
significant shortcomings in the student’s ability to identify
the relevant law with accuracy. Additionally or
alternatively they may be unable to apply that law
effectively and in a manner that is likely to progress the
client’s case. Additionally or alternatively such students
will demonstrate little commitment to the client’s best
interests, or may show such poor time management
skills that any attempt to progress their cases efficiently
is undermined. Similarly the student’s failure to comply
with the administrative requirements of the Law Office
may mean that they cannot be trusted to conduct any
work without continuous oversight from the supervisor.
Relevance of the grade descriptors
to the video interview?
• The student’s interview would be assessed for
competency
• It may form part of the student’s mandatory
written skills reflection
• It would be part of the evidence base for the
application of the grade descriptors at the end of
the year – e.g. Opportunity to show accurate
legal analysis / advice, ability to think across
legal ‘areas’ (landlord and tenant law into
homeless rights), empathy, commitment to
client, strategic thinking, professional conduct
etc.
Observations
• Grading clinic clearly presents challenges
• If we are going to grade clinic we need some way of
allaying the fears of faculty and students that we grade in
an arbitrary or unreliable fashion
• The grade descriptor approach is a potentially useful
mechanism for helping faculty and students understand
the overall “feel” of student performance at different
grade boundaries
• It should help us make fair decisions between our own
students and encourage consistency between clinical
faculty
• Feedback thus far has been positive from faculty and
from students
Issues with descriptors
• The descriptors need to be sufficiently detailed
to cover the range of learning outcomes we seek
for clinical students but sufficiently simple to be
readily understandable by students and helpful
for faculty members when it comes to the
marking process. This is a difficult balance to
strike.
• Having the descriptors does not guarantee that
they will be used. There is a need for an
ongoing dialogue about expectations.
The labyrinthine UK
assessment context
•
•
•
•
•
•
•
•
•
•
assessment tutor and team devise draft assessment
submit to subject specialist for comment and approval
submit to scrutiny committee for comment and approval
submit to external examiner from another institution for
comment and approval
students complete assessment
marking meeting with team – agree a marking scheme
marking by faculty
internal review / moderation process
external examiner moderation / approval
examination board approval
Clinicians aren’t the only law
teachers with assessment issues
• “An examination of the incidence of ‘error
variation’ in the grading of law
assessments” Hanlon et al 2004
www.ukcle.ac.uk
• "To give the most extreme example - one
student's paper was given a mark of just
13 by one examiner and 56 by another.”
Times Higher Education Supplement, 9th
January 2004.
Information
• Contacting us:
– Kevin Kerrigan: kevin.kerrigan@northumbria.ac.uk
– Philip Plowden: philip.plowden@northumbria.ac.uk
• Finding the Northumbria grade descriptors: via
www.studentlawoffice.co.uk
• Visiting us: visitors very welcome - we’re in Newcastle
upon Tyne, in the North of England, near the Scottish
border.