Issue 1
Best Practice on Peer Review of Safety Case Submissions
Introduction
Safety cases are expensive to write, difficult to produce and the role of the safety
case is not always well understood by key stakeholders e.g. facility managers,
engineers, project managers and designers.
The safety case provides prime evidence of the licencees’ capability to build,
operate or decommission a facility. It provides traceability of the arguments,
justification of the adequacy of arrangements and documents underpinning
decision making processes that enable safe operation. The safety case is
required by law to demonstrate the risks from the operations in question are As
Low As Reasonably Practicable.
This paper has been produced by members of the Peer Review Forum and
presents the combined views of peer reviewers from the majority of nuclear
licencees within the UK.
Scope
The forum has recognised common shortcomings within safety cases submitted
for review, both in terms of technical content and safety case production
processes.
Peer Review Process
A number of these shortcomings are fundamental and as a result this paper
identifies these as areas that peer reviewers should focus on. This paper
summarises these shortcomings particularly in association with LC15 submissions;
however these same shortcomings are also found in many other safety case
submissions. It highlights the areas of concern which peer reviewers have to
investigate and also forms a useful aide memoire for peer review.
Safety case Process
In addition, the paper lists topic areas that can be used to predict the well being of
the safety case process. It is based on the comprehensive review of many safety
cases across a variety of licencees.
It is offered to the Safety Directors’ Forum for consideration and dissemination to
the safety case production community in order that these issues can be
recognised and addressed at source.
Disclaimer
It is emphasised that different licensees have different remits for their Peer Review
processes, owing to particular aspects of their business and in some cases their
unique relationship with MoD regulators and external Design Authorities.
Page 1 of 7
Issue 1
Therefore, licensees must use their discretion in applying the clauses in this
Paper, which is framed for a “typical” licensee.
Definition of Peer Review
Peer Review is the independent scrutiny of a safety submission to confirm that the
overall validity and adequacy of the safety arguments are based on sound
engineering, technical and risk arguments.
The Review Team
The Peer Review Team must be composed of suitably qualified and experienced
persons (SQEP) who are independent of the ownership, preparation and
verification of the safety submission being reviewed, and who retain that
independence throughout the review process. Each licensee has their own
arrangements for defining what constitutes a minimum level of independence.
The Peer Review Team Leader has a pivotal role in ensuring the right team is
selected and co-ordinated, so that credible and authoritative scrutiny is made of all
of the key technical, engineering and operational issues pertaining to the safety
submission. This requires effective communication of review comments, tenacity in
following up unresolved issues, discernment between minor and major issues, and
the ability to reach a consensus.
Essential Elements of the Peer Review
Experience has shown that the early involvement of peer review is highly
advantageous, as it helps the reviewer to understand the key decision making
processes and major commitments required in the project. In addition, early
involvement can help identify show stopper issues at an early stage.
At the specification stage of the submission, the Review Team should check the
following:







The scope is correct and matches the boundaries of responsibility of the
licensee.
The relationship between the safety case under review and other related cases
(or projects) is clearly defined and justified.
The proposed level of assessment and substantiation is proportionate to the
hazard and safety case categorisation.
Arrangements to demonstrate that Risk Assessors and Engineers are SQEP.
Suitable modern standards are being identified.
Project Manager understands the interface between risk assessment and
safety case principles.
Safety Case Manager understands the plant and relevant disciplines required
for substantiation (primarily engineering, but might for example extend to
process chemistry, metallurgy, reactor physics or criticality depending on the
plant).
Joined up assessment methodologies to be applied with a logical work
breakdown structure leading to delivery of the final submission.
Page 2 of 7
Issue 1






Integration of Risk Assessors, Engineers and Plant Operators/Managers within
the safety case team.
Sufficient involvement of plant operators with the necessary competence and
adequate allocation of time to contribute effectively.
Design Assessment Reports (DARs i.e. engineering substantiation) arranged
so that the assessment of integrated safety systems is not artificially split
according to engineering disciplines, but that a pragmatic approach is taken
and arrangements to achieve integration of the DARs are in place.
Long lead-time assessments (e.g. Finite Element models, Seismic
Assessments) are planned to feed into the DARs and Risk Assessment at a
fully developed stage (i.e. complete, verified, sensitivities identified).
Adequate arrangements are in place for ALARP review process with an
appropriate level of involvement of all stakeholders.
Robust systems of verification in place prior to release of documentation for
Peer Review.
Issues for the reviewer to consider in the main submission will include the
following:

Where there is an extant safety case, check that a suitable and thorough
review of the Safety Case against modern standards (e.g. hazard identification,
risk assessment) has been undertaken by the safety case team, with Shortfalls
and an integrated improvement plan suitably identified for revision of the Safety
Case documentation.

Check that the ALARP methodology for assessing improvement actions to
address Shortfalls meets the standard set by company standards that address
the HSE Policy “Reducing Risks, Protecting People” and the NII TAG
T/AST/005.

Confirm that a baseline (in terms of the current safety assessment) has been
established for judging the significance of shortfalls, and that these shortfalls
are being set in a suitable context so their significance is clear.

Confirm that all Safety Systems, Structures and Components (SSSCs) and
Safety Functional Requirements (SFRs) have been identified and derived.

Check that DARs are complete and that there is evidence of adequate and
accurate information being fed into engineering assessments and
documentation.

Ensure all DARs are reviewed:


For technical robustness and completeness by an independent
Engineer(s) of the relevant engineering discipline.
By a member(s) of the Peer Review team for general clarity, suitability
and acceptable links to the rest of the submission, checking that
immediate actions have been taken to address Shortfalls challenging
Page 3 of 7
Issue 1
the ability to meet fundamental limits and conditions of the Safety Case
or potential breaches of legislation.

Ensure that the safety case and the design justifications clearly identify the
required safety function and the adequacy of the SSSCs to deliver these
functions.

Confirm that there is adequate integration of the findings of the DARs with the
Design Basis Accident (DBA) analysis and Probabilistic Safety Assessment
(PSA) to provide consistency between all assessment strands.

Scrutinise closely the suitability of improvement actions to address Shortfalls
asking the following questions:








Are they focussed on trends and themes from a collective set of
Shortfalls?
Are they set in the wider context of ALARP priorities for the site?
Is there evidence that the proposed action is feasible and does not
introduce other unacceptable risks or hazards?
Do the improvement actions collectively reduce risk and is this manifest
from the PSA?
Are the proposed actions resourced and agreed by management
Is there evidence that completion dates are viable?
Is there evidence that the completion dates for actions are as soon as
reasonably practicable?
Check that the claims and conclusions of the summary report for the safety
submission (whether representing an intermediate stage or final completion of
the process) are rigorously substantiated by the supporting referenced
assessments, and that any limitations or caveats are transparent in the
summary report.
Examples of Good Practice

Well defined and de-conflicted plan for delivery of the various supporting
assessments at the time when they are needed in the safety case process.

Appropriate use of a staged safety case submission strategy to allow for early
findings to be reported and feedback to be received from the NII, facilitating
early commencement of the first set of improvement actions.

Engineering substantiation does not take anything for granted. All assumptions,
data, drawings, plant configuration and models validated before use or
analysis.

Documents generated in good time to allow detailed internal project/plant
operator review and verification prior to review by the Design Authority and
Peer Review Team.
Page 4 of 7
Issue 1

An integrated safety system has a substantiation report that is integrated
across the relevant engineering disciplines.

Strong ownership of the safety case process by the plant management,
including control of the ALARP review process.

Prompt discussion with plant management of any provisional Shortfalls to
determine whether immediate improvement actions should be taken.

Improvement actions aimed at addressing systematic issues and the root
causes of deviation from modern standards.

Clear and readable summary document that is fully transparent about the
assessment work undertaken, and which sets out a coherent strategy for
reducing risks in a timely manner.

Focus is on significant issues (e.g. use of low consequence methodology).

Clear statement of inventories, both radiological and chemical.

Agreement on the range of assessments (e.g. human factors, ergonomics
etc.).
Examples of Bad Practice

Key plant personnel unable to provide the time to contribute, so the role is
delegated to inexperienced junior staff.

Every Shortfall has a different (and potentially conflicting) improvement action.

Commitment of finite resource to address Shortfalls in one facility at the
expense of more worthwhile improvements elsewhere on site.

Inventory and flow sheets poorly understood and characterised.

Poor understanding of structures and safety mechanisms until a late stage in
the safety case process.

Dogmatic division of substantiation reports (DARs) into particular engineering
disciplines.

Potential breach of legislation identified, but no immediate remedial action
taken or operational embargo placed.

Assessments highly time-constrained and sent incomplete or unverified for
Peer Review.

Placement of Shortfalls to cover areas of incomplete assessment work. (NB:
This does not apply to information that genuinely cannot be obtained, for
Page 5 of 7
Issue 1
example where there are incomplete drawings for inaccessible areas of the
plant.)

Final summary report claims credit for DARs without identifying their limitations
or caveats.

Rejection of an SSSC if it solely fails to meet modern standards, whilst it is fit
for purpose. (This may stem from the absence of an adequate definition of “fit
for purpose”: i.e. SSSC substantiated to meet safety function now and for
period of review and meets modern standards so far as reasonably
practicable.)

Untraceable supporting references (e.g. private communications and draft
documents)
Table 1 summarises the major issues relating to the development of safety cases
that peer reviewers regularly encounter. Rather than some high level conclusions,
the list is offered as a health check on the preparation and progress of a safety
case and they will give a good indication of the state of the safety case project.
The first column of Table 1 can be regarded as managerial issues (there are 15);
the second column has 7 which can be categorised as technical. This
summarises the collective experiences of peer reviewers of safety case
submissions. It is not just what is done technically by assessors and engineers
that could be at fault - it is often how the project team organises itself. Licensees
have comprehensive guidance on technical methods for specific safety case
processes and generally these are followed well. It is often how projects fail to tie
these together and recognize they are mutually dependent which can result in
difficulties in safety case production.
Conclusions
Peer reviewers add significant value to a Safety Case, whilst also maintaining the
independent examination for consistency and integrity. This added value often
goes unrecognised but is now being seen as integral to the Safety Case process.
This paper is offered to the SDF for consideration and dissemination to the safety
case production community in order that these issues can be recognised and
addressed at source.
Page 6 of 7
Issue 1
Table 1 Problem Issues
BLOCKERS TO PROGRESS
MANAGERIAL ISSUES
TECHNICAL ISSUES
Lack of SQEP writers.
Poorly defined scope of safety case.
Late appointment of peer reviewer.
Ill-defined specification of safety case.
Lack of overall resource.
Incorrect categorisation of Safety
Cases.
Lack of understanding of the process Licensee may not always be an
intelligent customer.
Lack of ownership of cases by Facility
Management.
Lack of challenge to the process by
various stake holders.
Poor records (e.g. drawings) availability.
Inadequate project definition.
Problems arising during assessments
(e.g. inventory changes).
Underestimation of time required by
process.
Engineering substantiation of old
facilities.
Inaccessibility to facility by Authors.
"Fire fighting" interrupts planned
process.
Contractual problems.
Change of author during the project.
Lack of money.
Unrealistic imposed milestones and
external pressures on the project.
Regulatory interaction/
expectations/communication.
Lack of interaction during the peer
review.
Page 7 of 7