Independent Peer Review of Legal Advice and Legal Work
A consultation paper from the Legal Services Commission
“the most significant step that the LSC has taken on the path to ensuring that quality
legal service is available to all recipients of legal aid” – Clare Dodgson, Chief
Executive of the Legal Services Commission (LSC)
Consultation paper available at (pdf copy so you need Adobe Acrobat to download)
http://www.legalservices.gov.uk/docs/criminal_consultations/Peer_Review_Consultatio
n_Final_Doc_march_2005.pdf
Responses by 25th June 2005 to:
Jennifer Will
Supplier Development Group
Legal Services Commission
85 Gray’s Inn Road
London
WC1X 8TX
e-mail - peer.review@legalservices.gov.uk
The purpose of this consultation is to comment on the model of the peer review
process, as constructed by the Institute of Advanced Legal Studies (IALS) on behalf of
the LSC. This is with a view to independent peer review being used on a national
basis as a direct measure of the quality of advice and legal work by Legal Help
suppliers, both in the criminal and civil fields. This briefing note is only concerned with
civil law suppliers, and concentrated on welfare benefits.
Contents of this briefing
p.1 – What is peer review?
p.2 – Who are the peer reviewers?
p.3 – How does peer review work?
p.8 – Identifying and enabling best practise.
p.8 – Equal Opportunities
p.9 – Peer review criteria, including welfare benefits specific criteria
p.10 – Questions.
What is Peer Review?
In this context, peer review is a form of contract audit based on a review of a sample
of a supplier’s files and undertaken by an experienced practitioner who is trained in
the peer review framework. This involves the assessment of files using a standard
criteria and ratings system to determine the quality of advice and legal work provided
to individual clients, with an overall judgement produced on the quality of the supplier.
1
Peer reviews are category specific and are carried out by a practitioner who is
experienced and skilled in that area of law.
The assessment of files for peer review is carried out using a standard criteria (the
Peer Review Criteria) and a ratings system developed by IALS researchers with the
Not for Profit sector (see Appendix 1A, p.9). The criteria are designed to highlight:



the information gained from the client and other sources;
the advice given based on that information;
the steps taken following that advice.
As well as generic criteria for all civil law categories, there are also category-specific
criteria under Debt, Employment, Housing, Immigration, Mental Health and Welfare
Benefits.
For each civil file reviewed (usually a sample of 15 files), an overall assessment of
quality is made with a 1 (highest) to 5 (lowest) rating. Following the sample files being
reviewed, the peer reviewer prepares a quality assessment report with particular
emphasis on trends and patterns identified across the sample, in order to determine
an overall supplier rating from 1 to 5 being given for the quality of legal advice and
work, as well as a value-for-money assessment. An overall rating of 3 or above
constitutes a “pass” at peer review.
The peer reviewer should not automatically arrive at the final rating simply by
averaging the scores on the individual files, but rather should use their skill,
experience and training to inform the overall rating of the supplier from the trends and
patterns seen on individual files. The peer reviewer does not employ any specific
formula to arrive at the overall rating as the fundamental nature of peer review is that it
is the judgement of an experienced practitioner.
Who are the Peer Reviewers?
Peer reviewers are carefully selected, trained and monitored by IALS. A Peer Review
Panel has been established in nine categories of law, with ongoing recruitment being
required to maintain these panels. Peer reviewers are recruited on a category specific
basis. Appointment to the panel is through an open selection process. All peer
reviewers are contracted with the LSC to carry out peer reviews under the guidance
and monitoring of IALS.
Standard application criteria are used during the shortlisting phase of the recruitment
process for peer reviewers. Alongside extensive experience and specialisation,
applicants must demonstrate as a minimum:



five years post qualification experience or equivalent in category of law;
three years supervisory experience in category of law;
three years experience of LSC Specialist Quality Mark/contract work.
2
Applicants must be currently working under an LSC contract in a caseworker and
supervisory capacity. Shortlisted applicants are interviewed and must undertake a
written exercise, assessing a category-specific case file. This is to investigate
applicants’ experience, their motivation to become a peer reviewer, their
understanding or consideration of concepts of competence, and their openmindedness and independence.
Candidates that successfully pass the interview and written exercise then undergo a
category-specific peer review of their own work, conducted by an existing member of
the peer review panel. Candidates are only accepted onto the panel if their work is
rated at a level of 2 or above. References will also be sought.
Training is then provided on the peer review process, including using the Peer Review
Criteria, the ratings and feedback reports, as well as undertaking example
assessments and working with existing peer reviewers from the panel. Applicants are
selected and trained to be open minded towards different practice approaches,
provided that these conform to appropriate standards of competence. The training
also aims to bring about consistency in assessments.
Following the training, trainees are required to conduct a mini peer review in order to
assess whether they will be effective peer reviewers and successfully apply the
training that they have received. If, following all of the above, the applicant is judged
by IALS to be suitable and competent, they will be accepted onto the panel. In order to
verify competency, an assessment is carried out on the first two peer reviews
conducted by a new reviewer. Further, there is regular performance monitoring, both
at regular and random intervals, along with further training exercises, guidance and
assessment of model files. A peer review of the peer reviewers will be conducted
every three years.
How does peer review work?
Peer review is used to monitor supplier performance for a variety of reasons, including
concerns over supplier performance, for national benchmarking of performance, etc. A
key part of the methodology is that peer reviewers are not informed of the reason for
the review.
A peer review result of Threshold Competence (3) or above is usually valid for three
years from the date of the review, provided there are no significant changes to
personnel (especially the category supervisor), or the supplier’s quality profile does
not indicate a possible reduction in performance.
File Sample
A random sample of 20 files are selected per category of law, with 15 of these files
actually being reviewed. The files will normally have been closed within the last 12
months. Checks are carried out with suppliers and reviewers to ensure there are no
conflicts of interest.
3
Conflicts, for the purpose of peer review, concern situations where there has been the
opportunity for the peer reviewer to form a view about a supplier’s level of competence
or to have their view of a supplier affected or biased in some way. For example
circumstances where the peer reviewer:




has a partner, or a member of their immediate family, or close friend, that works
in a firm/organisation that is scheduled for review.
has previously worked for the supplier.
has recently acted against the supplier.
has taken over a matter from the supplier.
Conflict checks also consider whether the organisations being reviewed are in direct
competition with the peer reviewers themselves. A peer reviewer will not review any
firm within their geographic region unless both the supplier and the reviewer consent.
All reviews take place on LSC premises – files are not sent to reviewers’ homes or
places of work.
Peer review process
There are two distinct stages of peer review possible:


initial peer review;
second peer review (only scheduled where initial peer review returns rating of 4
or 5).
Suppliers have the right to make representations where peer review rating is 3 or
below. (see p.6 for further information on representation process)
Initial peer review
Each review takes approximately one day to assess file sample, and half day to
complete the report. Peer reviewer works from the top of the list of sample files to
ensure integrity of randomisation of file selection. Each file is assessed against
standard criteria and ratings scale. The criteria provide a framework against which the
peer reviewer evaluates:



the quality of information gained from client and other sources,
the advice based on that information, and
the steps taken following that advice.
Following consideration of the files, the peer reviewer prepares a peer review report,
which details the findings of the assessment. This is the main document that is sent to
the supplier following the peer review, and is detailed with appropriate examples from
the files to support and substantiate the rating given.
The report has 8 main sub-headings:
4








The review;
Positive areas of good practise or good performance;
Areas for development/major areas of concern;
Other areas of concern;
Further comments;
Suggested areas for corrective action;
Overall value for money score;
Overall quality of work score;
A peer review rating of (1) or (2) will not have any major areas of concern, and
therefore any issues on the file sample are more likely to be noted under “areas for
development”.
Otherwise, “major areas of concern” represents concerns that most influence the
overall rating, for example, where the quality of advice has produced an adverse
impact on the client. This could include a lack of knowledge or proper understanding
of the law, incorrect or unclear advice, lack of advice, failure to address mental health
issues, conflicts of interest and possibly costs issues (cost is not an integral part of the
peer review system but it is taken into account).
“Other areas of concern” are those that do not have an immediate impact on the client,
such as files not in good order, failing to follow standard procedures, poor
administration, or inappropriate language.
The “corrective action” section is not intended to direct the supplier how to put such
concerns right, though the report can indicate what approaches might be considered.
Suppliers should decide themselves how to best implement corrective action. The
peer reviewer will write a specific comment if they feel the supplier could benefit from
attention to any specific area, such as supervision, file review, training, etc. It is
expected that all peer reviews at level 3 or below will require some form of corrective
action.
The “value for money” rating assesses whether there has been effective use of public
resources, taking into account the value to the client and the use of resources to
achieve the outcome. No consequences are incurred as a result of this assessment
and it has no impact on a suppliers contract.
An “overall quality of work” rating is then given, ranging as follows (see p.6 below for
more information on ratings):
1.
2.
3.
4.
5.
Excellence
Competence plus
Competence
Below Competence
Failure in Performance
A supplier should be able to understand why a particular matter is considered by a
peer reviewer to be an area of concern, so that they can implement remedial
5
measures. Suppliers should also be able to clearly understand why they have
achieved their rating and not a higher or a lower one, as well as enabling them to
understand how they could improve that rating.
Representation process
Where a supplier receives a rating of (3) or less, they have the right to make
representations on the following grounds:



the supplier disputes the overall peer review rating;
the peer review file sample does not appear to the supplier to be sufficiently
representative;
any other reasonable grounds.
Representations must be made within 21 days of receipt of the peer review report.
The peer reviewer considers the representations and decides whether, in light of
supplier comments, any amendments should be made to the report and if the rating
should be changed. The peer reviewer draws up a report recommending one of three
actions:



the original rating is upheld;
the original rating is overturned and a new rating is decided;
a new review is requested.
This report is submitted to IALS for monitoring, and IALS ensure that the correct
process has been followed and that the peer reviewer has considered and addressed
the representations made. A decision with written reasons is provided to the supplier,
normally within 28 days.
Second peer review
A second peer review is scheduled in the following circumstances:


where the supplier performance is rated (5), a second review will be carried out
immediately;
where the supplier performance is rated (4), a second review will be carried out
within six months.
The second peer review is carried out on a new random sample of case files, and
conducted by a different peer reviewer, with the process being the same as under the
initial peer review. It is stated that suppliers could be expected to meet the charge of
the second review (in the region of £1,300), with the cost being reimbursed if the
second review overturns the findings of the first review. However, there are no
immediate plans to introduce a charge.
Peer review outcomes and reporting
6
Suppliers receive written feedback on the outcome of a peer review in the form of a
Peer Review Report. No report is fed back, or action taken on the basis of a rating
result, until IALS has reviewed the rating and associated comments. Where IALS
considers there to be a concern regarding the outcome of a review, (examples given
include where comments do not support rating, or reviewer appears to be consistently
awarding a particular rating) IALS will investigate and resolve the matter before the
report is fed back to the supplier. The result remains provisional until the full peer
review process has been completed, as described above.
Ratings
Excellence (1), Competence Plus (2) – indicates that the supplier is conducting work
to a high degree of competence, and is considered necessary for a supplier to be
awarded “Preferred” status. The result will normally be valid for 3 years.
Threshold Competence (3) – indicates that, although the quality of work is proficient,
there are areas of concern regarding the quality of advice and legal work which need
to be addressed, as well as indicating that Specialist Quality Mark requirements might
not be being met. The Peer Review Report identifies potential corrective action and
areas for development, and should be used to form a development plan for the
supplier. Proposals for corrective action should be submitted to the supplier’s Account
Manager within 28 days of receipt of the final confirmation of the peer review outcome,
with progress against these proposals being considered at future audits.
Below Competence (4) – indicates that supplier’s work is being conducted at a
standard below that which a client is reasonably entitled to expect from a competent
solicitor. Proposals for corrective action should be submitted to the supplier’s Account
Manager within 28 days of receipt of confirmed peer review outcome, and a second
peer review scheduled within 6 months to check that quality has improved. If the
second review also results in a rating of (4), it will normally be appropriate to terminate
the supplier’s right to conduct publicly funded legal services in the relevant category of
law, and may be appropriate to terminate the entire contract.
Failure in Performance (5) – indicates that, following two separate peer reviews, the
supplier is conducting work at a standard that is significantly below that which a client
is reasonably entitled to expect from a competent solicitor (as defined by Clause 3.2 of
Contract Standard Terms). It will normally be appropriate to terminate the supplier’s
right to conduct publicly funded legal services in the relevant category of law, and may
be appropriate to terminate the entire contract.
In the event of a termination, the supplier has a right to review as set out in Clause 23
of the Contract Standard Terms.
LSC staff will not be able to explain, justify or alter any comments that appear on a
report, as the report and the ratings decision is an independent assessment written by
the peer reviewer.
7
Identifying and enabling best practise
From peer review results, the LSC will summarise and prioritise information relating to
quality of advice and legal work provided to clients, identifying areas of good practise
and areas most in need of development or improvement. This exercise will be
conducted on a quarterly basis and published on the LSC website and in relevant
publications.
Against this information, suppliers are expected to undertake self assessment against
key areas identified and where necessary, implement changes to make
improvements.
Equal Opportunities (EO)
There is a quite rigorous EO impact assessment in terms of ethnicity of peer
reviewers, equality and diversity training for peer reviewers, file selection, ethnicity of
contracted suppliers, and benchmarking. The one area that seems to have been
overlooked is whether, and how, peer reviewers will be expected to analyse the EO
policies of suppliers and whether they are effective.
There are currently 80 peer reviewers. Over the last two years the LSC has monitored
the peer review recruitment process. Currently, almost 14% of Legal Aid fee earners
are from minority ethnic backgrounds. 17% of applicants for the position of
Independent Peer Reviewer were from minority ethnic backgrounds and 25% of those
recruited to the position of peer reviewer were from minority ethnic backgrounds,
indicating that those applicants from minority ethnic backgrounds were more qualified
for the position. This shows peer reviewers are more ethnically diverse than the LSC
supplier base.
The LSC is currently conducting a number of random peer reviews, which have been
stratified by region and category of law. This is in order to establish a Peer Review
Benchmark of the quality of advice provided in each category of law according to the
rating achieved at peer review. In Summer 2005 an analysis of these results will be
conducted. This will compare the peer review rating and the ethnic profile of the
suppliers in order to see whether there are any disproportionate relationships between
the ethnicity of the supplier and the peer review rating achieved for the sample of files
reviewed in each category of law.
The results of the Peer Review Benchmark exercise will be published in the Fifth LSC
Annual Equal Opportunities Report in Autumn 2005. The LSC will continue to monitor
and publish on an on-going basis the ethnic profile of the suppliers peer reviewed
against the peer review rating achieved in each category of law.
8
Appendix 1A Peer Review Criteria – Civil Files
A. Communication with the Client:
1. How well does the adviser appear to have understood the client’s problem?
2. How effective were the adviser’s communication and client-handling skills?
3. How effective were the adviser’s fact- and information-gathering skills?
4. How effectively was the client informed of:
a) The merits (or not) of the claim? and
b) All developments?
B. The Advice:
1. How legally correct was the advice given?
2. How appropriate was the advice to the client’s instructions?
3. How comprehensive was the advice?
4. Was the advice given in time/at the right time?
C. The Work/Assistance:
1. If no other work was carried out, was this appropriate?
2. If any further fact-finding work was carried out:
a) How appropriate and
b) How efficiently executed was the work?
3. If any other work was carried out:
a) How appropriate was the work? and
b) How efficiently executed was the work?
4. How effective in working towards what the client reasonably wanted/needed was
any further work carried out?
5. If no disbursements were incurred was this appropriate?
6. How appropriate were any disbursements incurred?
7. Where this is necessary, did the adviser consider/advise on/act on an effective
referral?
8. Throughout the file how effectively did the organisation use resources?
9. Did the adviser or their work in any way prejudice the client? If yes, provide details
overleaf.
Appendix 1C Specialist Subject Criteria – Notes for Guidance
Welfare benefits:
A3. The information-gathering role of the adviser is crucial, because detailed
information about the client’s immediate situation (including details of all benefits
received) and background are needed if the adviser is to advise fully.
B3. ‘Comprehensive’ should include –
 Advice on different benefits to which the client may be entitled.
 If an appeal is being considered, advice on the risk that a reassessment may
result in a decrease or even loss of benefit rather than an increase, if the client
is assessed at a lower level of need than on the initial application.
B4. Time limits for appeals are tight, and the reviewer should check on file that an
effective diary system was used.
9
Questions
Question 1
Do you agree that the process of Independent Peer Review is independent of the
LSC? If you disagree, please provide reasons.
Question 2
Do you believe there are any additional criteria that may enhance the peer review
assessment?
Question 3
Are the ratings sufficient to differentiate between supplier performance?
Question 4
Are the recruitment criteria set at the appropriate level?
Question 5
Are there any additional training and assessment mechanisms for peer reviewers we
should consider?
Question 6
Do you agree with the process for ensuring the consistency of peer reviewers?
Question 9
Are the criteria sufficient to identify all potential conflicts of interest?
Question 10
Does the process provide an effective and efficient mechanism for confirming the
rating of the quality of advice and legal work rating of a supplier?
Question 11
Are there any additional areas that the Peer Review Report should consider in
providing a comprehensive commentary of the quality of advice and legal work of a
supplier?
Question 12
Do you agree with the preliminary equalities impact assessment?
10