Operational Risk
Model Assurance Checklist
Bank:
Date:
Completed by:
Contents
1.
Granularity
2.
Distributional assumptions: Building calculation dataset
3.
Distributional assumptions: Identification of probability distributions
4.
Distributional assumptions: Scenario analysis
5.
Distributional assumptions: Aggregate loss distribution
6.
Correlation and dependence
7.
Use of four data elements: Internal loss data (ILD)
8.
Use of four data elements: External data (ED)
9.
Use of four data elements: Scenario data
10. Use of four data elements: BEICFs
11. Use of four data elements: Combining elements
12. Estimation of unexpected losses
13. Risk sensitive capital allocation
14. Insurance modelling
15. Documentation
16. Validation activities
17. Model governance
References
BCBS 196 - Basel Committee on Banking Supervision paper entitled Operational Risk – Supervisory Guidelines for the Advanced Measurement Approaches, published June 2011
Basel II – International Convergence of Capital Measurement and Capital Standards , a revised Framework, Comprehensive Version, published June 2006
Banks Act Reg – Basel III Regulations - Government Gazette, No.35950, published 12 December 2012
Page 1 of 24
No.
Criteria
1
Granularity
1.01
When choosing its operational risk
categories, a bank should take into
account the nature and complexity of
business activities and the operational
risks to which they are exposed (BCBS
196, par 162).
BCBS
196
162
Basel
II
Banks
Act
Reg.
Attest procedures
1.
Understand the bank’s policy for determining Operational Risk
Categories (ORCs).
2.
Inspect the bank’s documented methodology and process on
the selection of ORCs. Through inspection of the
documentation, confirm and document that the methodology;
i.
Is compliant with the policy.
ii.
Considers the structure, nature and complexity
of the bank.
iii.
Has considered the four data elements. (Refer to
sections 7 – 10)
1.02
When modelling operational risks, a
bank should ensure that the model
takes into account the bank’s
idiosyncrasies. These may include the
business profile, risk profile, history of
operational losses, business
environment and other factors. A bank
should characterise operational risks
along these factors. For modelling
purposes, it is important that risks
sharing common factors are grouped
together (BCBS 196, par 163).
163
1.
Refer above to attest procedure 1.01.
1.03
When a major change in the
organisational or the risk profile of an
institution occurs, the bank should
ensure the choice of granularity
remains valid (BCBS 196, par 164).
164
1.
Through discussions with senior management, review of
minutes of meetings, and review of the bank’s annual reports
confirm and document whether major organisational changes
have occurred.
2.
Where major organisational changes have occurred, inspect
the developmental evidence and confirm and document that
the organisational changes have been considered in the
assessment of ORCs used in the modelling process.
3.
Where granularity has not been addressed appropriately,
document management’s reasons for not considering the
organisational changes.
Inspect model developmental evidence and confirm and
document that all available data has been considered in the
assessment of granularity.
1.04
A bank should determine the optimum
balance between granularity of the
classes and volume of historical data
for each class. Using one or only a few
ORCs can lead to increased
heterogeneity for the events in each
category. A high number of ORCs can
cause the number of losses in each
category to fall below a model’s data
threshold. As such an outcome is more
likely for business lines where the
underlying risk exposure is immaterial,
the materiality of a business line may
in effect be one of the factors
determining the level of granularity.
Supervisors should be wary when an
institution uses either a very low or
very high number of ORCs, especially
when used in conjunction with a loss
distribution approach (LDA) (BCBS
196, par 165).
165
1.05
A bank should provide evidence to
supervisory authorities that its choice
of operational risk categories is
reasonable and does not adversely
impact other factors of the operational
risk model, such as diversification
assumptions, correlations and capital
allocation (BCBS 196, par 166).
1.06
A bank should support its choice of
granularity by qualitative and
quantitative means. It should be
particularly aware of the impact its
choice of granularity has on the capital
charge and provide evidence that the
choice is reasonable (BCBS 196, par
167).
1.
2.
Confirm, and document, through the inspection of
developmental evidence and discussions with management
whether the bank considered external or industry data
availability.
3.
Enquire of management if the four data elements are available
per the Basel II event type and business line matrix.
4.
Where the bank’s ORCs differ from this matrix, enquire and
document from management the reasons.
166
1.
Refer above to attest procedures 1.01 to 1.04.
167
1.
Inspect
the
developmental
evidence
and
model
documentation to confirm and document the extent to which
both qualitative and quantitative factors were considered in
determining the level of granularity.
2.
For the quantitative factors considered, re-perform significant
statistical techniques and tests performed by the bank.
Compare the results to those of the bank.
168 &
232
A high number of ORCs may lead to
an unrealistically high capital charge
when no correlations are modelled and
capital charges for all ORCs are
Page 2 of 24
Findings
Management comment
No.
Criteria
BCBS
196
Basel
II
Banks
Act
Reg.
Attest procedures
summed together. On the other hand,
a bank modelling correlations that use
a high number of ORCs might have
difficulty finding statistical means to
validate correlation assumptions due
to minimal loss data for each ORC
(BCBS 196, par 168).
Dependence should not be
inappropriately affected by the choice
of granularity. For example, many
operational risk management
frameworks assume statistical
independence between losses within
the same ORC. To the extent that a
bank’s framework has only a few
ORCs, the impact of dependence may
be inappropriately minimised. In such
a situation, it may be preferable to
simply add capital estimates across
ORCs (BCBS 196, par 232).
1.07
Capital allocation to internal business
lines should be a factor when choosing
ORCs, as these ORCs may be used
as part of the capital allocation
process. When using an allocation
method that is very different in nature
from the choice of ORCs, the bank
should ensure that its choice of ORCs
and allocation method was reasonable
in the first place. Note that changes in
the ORCs need not always correspond
with changes in the capital allocation
method. For example, banks often
take continuous management actions
leading to changes in their business
units that may not lead to major
changes in their business processes or
risk profile. Such changes may not
justify changing the ORCs used for
capital modelling, even though they
must be incorporated in the capital
allocation process (BCBS 196, par
169).
2
Distributional assumptions: Building
calculation dataset
2.01
A bank should have a policy that
identifies when a loss or an event
recorded in the internal (or external)
loss event database is also to be
included in the calculation dataset.
This policy should provide a consistent
treatment for loss data across the
institution. Exceptions to the policy
should be limited and, in any case,
duly documented and properly
addressed to prevent undue reduction
of the capital charge (BCBS 196, par
178)
The building of a proper calculation
dataset from the available
internal/external data requires that a
bank develop policies and procedures
to address its several features (i.e.
perimeter of application, observation
period, reference date, de minimis
modelling thresholds and data
treatment) (BCBS 196, par 179)
A bank shall have in place duly
documented procedures in order to
assess the ongoing relevance of
historical data, which documented
procedures shall duly specify the
situations in which judgment, scaling
or other adjustments to internal loss
data may be used, including the extent
to which such judgment may be used
and the officials who are authorized to
make such decisions (Regulation
33(9)(d)(v)(B)(iv))
A bank’s internal processes relating to
the collection of loss data shall include
169
178 &
179
33(9)(
d)(v)(B
)(iv)
1.
Obtain and inspect model build, minutes of meetings and
other relevant documentation to confirm and document
whether the ORCs are used in the capital allocation process.
2.
Confirm, through the inspection of the bank’s documentation,
the consistency between capital allocation and level of
granularity. Where these differ document management’s
motivation.
1.
Confirm that the bank has a documented internal process for
the collection of loss data
2.
Confirm that the documented process includes an appropriate
loss threshold amount for the collection of internal loss data
Confirm that this threshold is in line with any minimum gross
loss threshold amount specified by the regulator
3.
Inspect the bank’s documentation for use of internal (and
external) operational risk data elements and document
whether there is a policy that identifies the requirements of the
use of the internal/external data in the calculation dataset.
4.
Inspect the policy and document that the bank’s internal loss
data are considered as direct inputs into the model. Where
there is less than 5 years of internal loss data (or 3 years
where the bank has just moved to AMA) document that the
bank has documented the alternative inputs used.
5.
Inspect the policy and document whether the process for
incorporating internal (or external) loss data has been
comprehensively documented, including exceptions and
limitations of the policy.
6.
Obtain the internal and external loss databases, confirm and
document that the inclusion/exclusion of these losses in the
calculation dataset is consistent with the bank’s policy by reperforming the filtering criteria on the loss database and
comparing it to the bank’s calculation dataset. Confirm and
document that any exceptions to the policy are motivated and
approved by senior management.
7.
Inspect the policies and procedures documentation and
document whether the data policies are regularly reviewed
internally and have also been subjected to independent review
(e.g. Audit, or Business Experts).
8.
Through inspection, ensure that the following is consistent
33(9)(
d)(v)(
C)
33(9)(
d)(v)(
D)
Page 3 of 24
Findings
Management comment
No.
Criteria
BCBS
196
Basel
II
Banks
Act
Reg.
Attest procedures
with the bank’s policy/documentation:
a. The data selection complies with any condition
specified in the AMA approval;
b. There is transparency around choice of observation
periods across ORCs;
c. The choice of reference date across ORCs;
d. The application of the de minimis modelling threshold
as defined for the ORC under consideration;
e. The treatment of data outliers, if any;
f. Any exceptions to the above or to areas of the bank’s
policy/documentation are motivated.
This test will be performed on the entire population except in
cases where performing the test on a sample basis would be
more efficient.
an appropriate de minimis gross loss
threshold amount for the collection of
internal loss data provided that in order
to ensure broadly consistent data
collection between banks that adopted
the advanced measurement approach
for the calculation of their respective
capital requirements relating
to
operational risk the Registrar may from
time to time specify a minimum gross
loss threshold amount. (Regulation
33(9)(d)(v)(C))
When a bank’s capital requirement in
respect of operational risk is based on
internal loss data, the said capital
requirement shall be based on a
minimum observation period of:
9.
Confirm that the bank has in place documented procedures for
assessing the ongoing relevance of historical data
10. Confirm that the ongoing relevance of historical data is
assessed by the bank on a regular (at least annual) basis
11. Confirm that the latest assessment performed is in line with
the documented procedures
(i) five years of data; or
(ii) when the bank originally adopts the
advanced measurement approach,
subject to such conditions as may be
specified in writing by the Registrar, a
minimum observation period of less
than five years of data, but in no case
less than three years of data,
irrespective whether the internal loss
data is used to calculate or validate the
bank’s measure of loss. (Regulation
33(9)(d)(v)(D))
2.02
2.03
The bank should base its internally
generated operational risk measures
on a minimum historical observation
period of five years (three years when
an institution first moves to an AMA).
For certain ORCs with low frequency
of events, an observation period
greater than five years may be
necessary to collect sufficient data to
generate reliable operational risk
measures and ensure that all material
losses are included in the calculation
dataset. If very long data series are
used, banks will need to consider the
heterogeneity arising from changes in
the risk profile through time. In such
cases, time trends or other
adjustments should be strongly
preferred to discarding older data.
Discarding older data should be
undertaken only as last resort for
ORCs where loss experience is
sparse. (BCBS 196, par 180).
The bank should base its internally
generated operational risk measures
on a minimum historical observation
period of five years (three years when
an institution first moves to an AMA).
For certain ORCs with low frequency
of events, an observation period
greater than five years may be
necessary to collect sufficient data to
generate reliable operational risk
measures and ensure that all material
losses are included in the calculation
dataset. If very long data series are
used, banks will need to consider the
heterogeneity arising from changes in
the risk profile through time. In such
cases, time trends or other
adjustments should be strongly
preferred to discarding older data.
Discarding older data should be
undertaken only as last resort for
ORCs where loss experience is
sparse. (BCBS 196, par 180).
180
1.
2.
180
Inspect the bank’s policy regarding the use of loss data and
document whether this policy mandates the use of at least 5
years of loss information once this data is available.
Inspect an extract from the loss data repository and assess
whether 5 years of loss data is currently available. When data
is available but has not been used in the capital model,
inspect whether reasons for not using available loss data have
been clearly explained within the model methodology of the
bank.
3.
Inspect the AMA modelling documentation (policy and
procedures) and document whether the bank included a
statement on heterogeneity arising from changes in the risk
profile (i.e. dealing with older data in cases where long data
series are used).
4.
Where heterogeneity may exist due to long data series,
confirm and document, by performing a walkthrough, that
appropriate time trends or other adjustments have been
considered prior to discarding old data.
5.
Perform a walkthrough and confirm, and document, that the
discarded internal data is documented, motivated and
approved by senior management.
6.
1.
Confirm and document that the documented procedures
specify situations in which judgment, scaling or other
adjustments to internal loss data may be used, including the
extent to which such judgment may be used and the officials
who are authorised to make such decisions.
Obtain and inspect the data set received from management
and confirm, for each ORC, that the bank has used a
minimum of five years of historical data (or three years if the
bank has recently moved (up to a period of 2 years) to AMA.
Document the period. (Refer to 2.01 procedure 3 when
testing this for individual ORCs)
2.
Re-perform any adjustments made to historical loss data.
3.
Refer to 2.02 procedure 6
Page 4 of 24
Findings
Management comment
No.
Criteria
BCBS
196
2.04
A bank may use one of the reference
dates (occurrence date, discovery
date, contingent liability date or
accounting date) for building the
calculation dataset, as long as material
loss data are not omitted. No other
dates are acceptable for building the
calculation dataset. (BCBS 196, par
181).
181
2.05
The discovery date or accounting date
are the most prudent choices for
developing a bank’s dataset for the
quantification of operational risk capital
related to that event. However, a bank
may use the occurrence date for
building the calculation dataset if the
bank has not constrained or limited the
observation period. (BCBS 196, par
182).
2.06
A bank should use a date no later than
date of reserve for including legal
related losses/exposures in the
calculation dataset. (BCBS 196, par
183).
2.07
A bank may establish a de minimis
modelling threshold for an ORC, so
that frequency and severity
distributions in each ORC are fitted to
the data only in excess of the
threshold. The de minimis modelling
threshold may differ across ORCs. The
choice of threshold for modelling
should not adversely impact the
credibility and accuracy of the
operational risk measures. (BCBS 196,
par 184).
Basel
II
Banks
Act
Reg.
Attest procedures
1.
Inspect the bank’s documentation and confirm, and document,
that the bank has included a reference date as per the
regulatory requirement for building the calculation dataset.
2.
Select a sample of ORCs and confirm that the bank is using
the appropriate reference date as stipulated in the policy and
that it is consistent across the ORCs.
3.
Verify through re-calculation that no material loss data are
omitted due to the selected reference date.
182
1.
Inspect the bank’s policy and confirm and document that the
bank’s motivation for its reference date is in line with the
bank’s policy.
183
1.
Inspect the bank’s measurement methodology, policy and
procedures documentation and confirm, and document, that
the bank has a comprehensive policy statement on the date of
reserve for including legal related losses/exposures in the
calculation dataset.
2.
Inspect the data set and confirm, and document, that the bank
has used a date no later than the date for reserve for including
legal related losses / exposures in the calculation dataset
Obtain and document the bank’s motivation and justification
for the choice of minimum modelling threshold with reference
to the impact on capital requirements.
184
33(9)(
d)(v)(B
)(iv)
1.
2.
Through the inspection of the bank’s developmental evidence:

Confirm that the documents incorporate processes
and governance requirements regarding changes in
the de minimis modelling threshold.

Confirm and document that the bank has assessed
the impact of different de minimis thresholds.

Confirm that changes in the de minimis modelling
thresholds are documented and approved by senior
management
1.
Inspect the bank’s modelling methodology, policy and
procedures documentation and confirm and document that the
identification and treatment (including any specific techniques)
of abandoned business lines and related events are
documented.
2.
Where a bank has identified data points related to abandoned
business lines within the calculation data, inspect the bank’s
documentation and confirm, and document, that the treatment
of these data points is consistent with the bank’s policy.
1.
Refer above to attest procedure 2.07.
1.
Obtain the loss dataset from management and apply the de
minimis threshold to the data. Compare the results to the
bank’s calculation dataset
2.
Perform a walkthrough and confirm, and document, that
losses above the set de minimis modelling threshold are all
included in the calculation dataset regardless of the amounts;
by comparing the loss dataset to the calculation dataset
Refer to 2.01 procedures for tests on regulation 33(9)(d)(v)(C)
Refer also Regulation 33(9)(d)(v)(B)(iv)
tested in 2.01 above
2.08
On an exceptional basis, a bank may
identify data points related to
abandoned business lines within the
calculation data. It may adopt specific
techniques for the treatment of these
data points to address an undesired
effect on capital measures. However, a
bank should justify and clearly
document the identification and
treatment of these data points and
provide estimates of the capital
requirements with and without this
treatment. (BCBS 196, par 185).
185
2.09
Use of de minimis modelling
thresholds that are much higher than
the data collection thresholds should
be limited and properly justified by
sensitivity analysis at various
thresholds. Moreover, changes in the
de minimis modelling thresholds, when
not embedded in the model engine
and driven by specific reasons (for
example discount rates), should be
limited in number and duly motivated
by the need to better capture the risk
profile of the ORC. (BCBS 196, par
186).
186
2.10
All operational losses above the set de
minimis modelling threshold should be
included in the calculation dataset and
used, whatever their amounts, for
generating the regulatory measures.
(BCBS 196, par 187).
187
Refer also regulation 33(9)(d)(v)(C) as
tested in 2.01 above
33(9)(
d)(v)(
C)
3.
Page 5 of 24
Findings
Management comment
No.
Criteria
BCBS
196
2.11
Losses caused by a common
operational loss event should be
grouped and entered into the
calculation dataset as a single loss,
unless a bank chooses to model
causality or dependence among those
losses in a different manner. A bank’s
internal loss data policy should
establish guidelines for deciding the
circumstances, types of data and
methodology for grouping data as
appropriate for their business, risk
management and capital charge
modelling needs. They should also
clarify and document their individual
judgments in applying these
guidelines. A bank’s policy about the
threshold and dates for single losses
should also be applied to grouped
losses. (BCBS 196, par 188).
188
A bank that groups small losses above
the threshold for modelling with no
causal relations for data collection and
registration purposes generally should
not include them in its calculation
dataset. (BCBS 196, par 189).
189
A bank should consider applying
appropriate adjustment rates on data
when inflation or deflation effects are
material. For example, when the
observation period for a specific ORC
is extensive (for example 15-20 years)
due to the infrequent occurrence of
loss events and the loss data series is
not stationary, adjusting loss amounts
due to discount effects could be the
solution to recover stationarity. (BCBS
196, par 190).
190
2.12
2.13
Basel
II
Banks
Act
Reg.
Attest procedures
1.
2.
Inspect bank’s policy regarding grouping. Perform a walk
through to confirm, and document, that the bank’s treatment is
in line with its policy.
Confirm appropriate condonation exist where grouping is not
applied.
1.
Determine, and document, by performing a walkthrough
whether small losses above the threshold for modelling with
no causal relations have been grouped for data collection and
registration purposes
2.
Discuss with management, and document, that grouped
losses with no causal relation have not been included in the
calculation data set. Perform a walk through to confirm that
these losses have been excluded.
Perform a walkthrough and confirm, and document, the bank’s
consideration of the impact of inflation or deflation.
1.
2.
Where it has been determined that the effects are material as
per the bank’s assessment, perform a walkthrough to
determine whether the bank has applied adjustment rates on
data according to their documented approach.
3.
Reperform the calculation of adjustment rates applied to allow
for the effect of inflation or deflation. Compare results to those
of the bank.
4.
2.14
A bank should not use loss net of
insurance recoveries as an input for its
AMA models. An approach using loss
net of recoveries and insurance
recoveries may prove especially
difficult in the calculation of the
maximum 20% capital requirements
reduction permitted for insurance
mitigation in the Basel III Framework.
(BCBS 196, par 191).
191
1.
Where adjustment rates have not been applied, obtain and
document the bank’s justification for their approach.
Through discussions with management confirm and document
that the bank has not used loss net of insurance recoveries as
an input into the AMA model
2.15
The recognition of insurance in
operational risk capital models is in an
early stage of development. A bank
should calculate the total operational
risk capital charge gross of insurance
recovery in order to determine the 20%
limit and isolate the bank’s
methodology for modelling insurance
mitigation. (BCBS 196, par 192)
192
1.
Refer below to attest procedure 14.05
3
Distributional assumptions: Identification of
probability distributions
3.01
A bank should follow a well specified,
documented and traceable process for
the selection, update and review of
probability distributions and the
estimate of their parameters This
process should result in consistent and
clear choices and be finalised to
properly capture the risk profile in the
tail.
193
1.
Inspect the bank’s developmental evidence, model
documentation, polices and minutes of meetings and confirm
and document that the bank has a documented process for;

Selecting the probability distributions,

Review of the probability distributions

Parameter estimation
3.02
Severity distributions play a crucial role
in AMA models. That the models are
often medium/heavy tailed implies that
the final outcome is significantly
impacted by the chosen distribution.
The choice of frequency distributions
has a lesser impact on the final
194
1.
Confirm, and document, through discussions with
management and through inspection of the bank’s
documented methodology, processes and minutes of
meetings that greater emphasis is placed on fitting severity
rather than frequency.
2.
Refer to 3.14
Page 6 of 24
Findings
Management comment
No.
Criteria
BCBS
196
Basel
II
Banks
Act
Reg.
Attest procedures
outcome.
3.03
The selection of probability
distributions should be consistent with
all elements of the AMA model. In
addition to statistical goodness of fit,
Dutta and Perry (2007) have proposed
the following criteria for assessing a
model’s suitability:

3.04
195
realistic (eg it generates a
loss distribution with a
realistic capital requirements
estimate, without the need to
implement “corrective
adjustments” such as caps),

well specified (eg the
characteristics of the fitted
data are similar to the loss
data and logically consistent),

flexible (eg the method is able
to reasonably accommodate
a wide variety of empirical
data) and

simple (eg it is easy to
implement and it is easy to
generate random numbers for
the purpose of loss
simulation).
The process of selecting the
probability distribution should be welldocumented, verifiable and lead to a
clear and consistent choice. To this
end, a bank should generally adhere to
the following:
196
1.
Re-perform the probability distribution selections based on the
bank’s policy and compare distribution selections to those
made by the bank.
2.
Re-perform the goodness of fits tests performed by the bank
and compare results to those of the bank. Re-performance of
goodness of fit tests must be conducted on both the data used
to perform the latest fit and any more recent loss data not yet
included in the model.
3.
Obtain and document management’s assessment of
alternative distributions. Re-perform the fitting of the
distribution and the goodness of fit test for the final distribution
selected by the bank. Where no alternatives have been
suggested, this must be noted.
Re-performance will be done on a sample basis.
1.
Inspect the developmental evidence and confirm, and
document, that probability distribution selection has been
adequately documented.
2.
Inspect the latest developmental evidence and all relevant
documentation, confirm that the utilized methodology which is
used to select the probability distributions are applied
consistently in subsequent input updates. Where there have
been changes, changes are documented and the impact of
the change has been considered by management.
3.
Inspect the developmental evidence and confirm, and
document, that the bank has performed EDA for each ORC
and that the results have been used in the distribution
selection process.
4.
Obtain and document the bank’s justification for the statistical
distribution selected.
(a) Exploratory Data Analysis (EDA)
for each ORC to better understand the
statistical profile of the data and select
the most appropriate distribution;
(b) Appropriate techniques for the
estimation of the distributional
parameters; and
(c) Appropriate diagnostic tools for
evaluating the quality of the fit of the
distributions to the data, giving
preference to those most sensitive to
the tail.
3.05
3.06
3.07
In order to examine the statistical
properties of each ORC (i.e.
homogeneity, independence,
stationarity), a bank should make use
of statistical tools which include, but
are not limited to, scatter plots, time
series autocorrelation plots, empirical
distribution plots, histograms and
regression analysis. Other tools, such
as p-p plots, q-q plots and meanexcess plots provide preliminary
evidence on the type and shape of the
probability distributions which better
represent the data. (BCBS 196, par
197).
197
The Range of Practice Paper reveals a
wide range of practices for the
estimate of the severity distributions,
with 31% of AMA banks applying a
single distribution to all the data and
nearly 50% using two separate
distributions for the body (or HFLI
region) and the tail (or LFHI region).
198
The operational risk data from a
severity perspective clearly illustrate
positive skewness and medium-heavy
tailedness (leptokurtosis). In statistical
terms, this may mean that not all the
199
Inspect the developmental evidence and minutes of meetings and
confirm, and document, that the bank used a range of appropriate
statistical tools to examine loss data and;
1.
Document the bank’s statistical technique.
2.
Re-perform the statistical tests performed, by the bank, on the
loss data and compare results to those of the bank.
1.
Inspect the developmental evidence and confirm, and
document, whether the bank has utilised splice-distributions.
Where splice-distributions are utilised, confirm the accuracy of
the bank’s results and selected splice distributions through reperformance and compare results to those of the bank.
2.
Where splice-distributions are utilised, re-perform the
goodness of fits tests performed and compare results to those
of the bank.
Re-performance will be done on a sample basis.
Inspect developmental evidence and confirm, and document,
that medium-heavy tailedness is identified in loss data and
considered, by management, in the distribution selection
process.
1.
Page 7 of 24
Findings
Management comment
No.
Criteria
BCBS
196
statistical moments of the severity
distribution exist; in many cases the
2nd moment (ie the standard
deviation) and higher moments,
although always empirically calculable,
are often enormous due to the relevant
dispersion of the data.
Basel
II
Banks
Act
Reg.
Attest procedures
2.
Inspect the developmental evidence and confirm, and
document, the bank has performed statistical tests to assess
medium-heavy tailedness of the data.
3.
Re-perform the statistical tests performed by the bank and
compare results to those of the bank.
3.08
A bank should pay particular attention
to the positive skewness and, above
all, leptokurtosis of the data when
selecting a severity distribution. In
particular, when the data are
medium/heavy tailed (therefore very
dispersed in the tail), the use of
empirical curves to estimate the tail
region is an unacceptable practice due
to the inability to extrapolate
information beyond the last observable
data point.
200
1.
Refer above to attest procedure 3.07.
3.09
In such cases the use of so-called subexponential distributions is highly
recommended. Subexponential
distributions, which sometimes have a
higher number of parameters than light
tailed curves, can better represent the
shape of the data in the tail (other than
their skewness in the body) by
allowing estimates of parameters that
do not depend on the higher order
statistical moments.
201
1.
Refer above to attest procedure 3.07.
3.10
When separate distributions for the
body and the tail are used, a bank
should carefully consider the choice of
the body-tail modelling threshold that
distinguishes the two regions. The
bank should provide documented
statistical support, supplemented as
appropriate by qualitative elements, for
the selected threshold, as the
threshold may significantly impact the
capital requirements. Ideally the
estimate of the body-tail modelling
threshold should be made conjunctly
with the parameters of the distribution;
however for practical reasons banks
tend to first identify the threshold and
then estimate the parameters. EDA
instruments like the hill plot and the
mean excess function plot can be
useful in the determination of the
threshold. A bank should employ
sound methods to connect the body
and tail distributions. In particular,
jumps in the probability mass function
when attaching the body and tail of the
distributions should be avoided, in
order to guarantee that the LFHI and
HFLI regions are mutually exclusive
and are properly reflected in the
aggregated distribution.
202
1.
Inspect the developmental evidence and confirm, and
document, that an inflection point analysis has been
performed by the bank that assesses the modelling threshold
between the two distributions.
2.
Inspect whether management have investigated the impact of
alternate thresholds between the tail and the body of the
population. If this has been done then review the statistical
tests and validations performed. Re-perform the statistical
tests performed by the bank. Compare results to those of the
bank.
When estimating the parameters of the
distribution, a bank should take into
account the incompleteness of the
calculation dataset in the model (eg
due to the presence of de minimis
modelling threshold(s) which may or
may not coincide with the data
collection threshold). The bank should
provide evidence that an incomplete
calculation dataset does not adversely
impact the credibility and accuracy of
the parameter estimates and capital
requirements.
203
1.
Obtain the bank’s developmental evidence and through
inspection, confirm, and document, that the choice of
thresholds above which data is captured and modelled is
justified and documented, by management, including:

Rationale in considering the thresholds used to
collect data;

Rationale in considering the modelling thresholds
used to determine the data set used in the model;

The application of a de minimis threshold

Whether the severity distribution parameter
estimation is conditional on this threshold

Whether the frequency distribution parameter takes
this threshold into account

Whether the conditional-fitting methodology is
statistically sound

Whether provision is made in the capital simulation
for the de minimis modelling threshold.
2.
Inspect the developmental evidence and confirm, and
document, that management has considered the impact of the
thresholds above which data is captured and modelled in the
model. (Refer to 2.02)
3.11
Page 8 of 24
Findings
Management comment
No.
3.12
Criteria
A bank should pay particular attention
to the estimate of the kurtosis-related
parameters, which describe the tail
region of the losses. Because of data
scarcity, the estimates can be highly
unstable. The bank should put in place
methodologies to reduce estimate
variability and provide measures of the
error around these estimates (eg
confidence intervals, p-values).
BCBS
196
204
Basel
II
Banks
Act
Reg.
Attest procedures
3.
If the minimum modeling threshold is significantly higher than
the minimum data collection threshold without qualitative
and/or quantitative justification, test appropriateness of
modelling threshold using goodness of fit tests on collected
data and final data used from modelling. Where quantitative
and/or qualitative justification is provided, document
appropriateness of the justification.
1.
Inspect the bank’s policy and documentation and confirm, and
document,
that a transparent and verifiable process is
documented to review and approve data adjustments such as
exclusions, scaling etc.
2.
Inspect evidence that any adjustments are documented and in
line with the bank’s policy.
3.
Re-perform the adjustments made by the bank and compare
results to those of the bank.
4.
Obtain the bank’s developmental evidence and through
inspection, confirm, and document the following:


3.13
Robust estimation methods (such as
alternatives to classical methods as
the Maximum Likelihood and the
Probability Weighted Moments),
proposed recently in operational risk
literature, are reasonably efficient
under small deviations from the
assumed model. These methods also
highlight which observations or
deviating substructures have the
greatest influence on the statistic to be
estimated. A bank may adopt
alternatives to classic estimators,
provided it can demonstrate that its
use does not underestimate risk in the
tail. These estimators may also be
used as a diagnostic technique for
evaluating the sensitivity of the capital
charge to the chosen parameter
estimation method.
205
3.14
A bank should assess the quality of fit
between the data and the selected
distribution. The tools typically adopted
for this purpose are graphical methods
(which visualise the difference
between the empirical and theoretical
functions) and quantitative methods,
based on goodness-of-fit tests. In
selecting these tools, a bank should
give preference to graphical methods
and goodness-of-fit tests that are more
sensitive to the tail than to the body of
the data (e.g. the Anderson Darling
upper tail test).
3.15
While diagnostic tools provide
information on the quality of fit
between the data and each
distribution, they do not always lead to
a clear choice of the best-fitting
distribution. Moreover, the results of
the goodness-of-fit tests are usually
Rationale for dealing with data outliers,
Whether the Quantile-Quantile plot error bands of the
severity distribution are produced and used in the
distribution selection process.
5.
Inspect the developmental evidence and confirm, and
document, that sensitivity analysis was conducted, by the
bank, to assess the variance within the tail region. Refer to
3.02.
6.
Inspect developmental evidence and confirm, and document,
that the bank has considered the fit parameters in determining
whether the estimator of kurtosis is appropriate.
Where the bank has used an alternative to the classical
estimators, inspect developmental evidence and confirm, and
document, that the statistical significance of the estimator has
been verified and documented and results have been
compared to classical methods.
1.
Document the classical methods considered by the bank.
Where classical methods have not been considered,
document the bank’s motivation and justification.
2.
Inspect the developmental evidence and confirm that the bank
has considered the fit parameters in determining whether the
estimator used is appropriate.
3.
Where the bank has used an alternative to the classical
estimators, inspect developmental evidence and confirm, and
document, that justification has been documented for any
variance specific to tail risk.
206
1.
Inspect the bank’s documented polices on the assessment of
quality of fit and confirm that:
a. The ranking of the goodness-of-fit test statistics have
been considered such goodness-of-fit tests could
include the Chi Square, Anderson-Darling,
Kolmogorov-Smirnov.
b. The statistical results support the method used to fit
the tail and that it accurately represents tail risk.
c. In assessing the graphical method used confirm that:
 The bank’s documented
policies/documentation address the graphical
method to be used.
 That the graphical method described in the
bank’s policy/documentation is repeatable.
 The graphical method used is in line with the
bank’s policy/documentation.
207
1.
Inspect the developmental evidence and confirm, and
document, that relative performance of the prescribed
statistics (such as the Likelihood Ratio, the Schwarz Bayesian
Criterion and the Violation Ratio) is used for each of the
distributions being considered for the fitting process.
2.
Inspect the developmental evidence and confirm, and
Page 9 of 24
Findings
Management comment
No.
Criteria
BCBS
196
sensitive to the sample size and the
number of parameters estimated. In
such cases, a bank should consider
selection methods that use the relative
performance of the distributions at
different confidence levels. Examples
of selection methods may include the
Likelihood Ratio, the Schwarz
Bayesian Criterion and the Violation
Ratio.
3.16
A bank should have a regular cycle to
verify assumptions underlying the
probability distributions they have
selected. These verifications may
follow the criteria and tests a bank’s
use in the selection of the probability
distribution. If assumptions are
invalidated, alternative methods should
be tested and implemented. However,
any change should be properly
justified. In particular, after suffering
one or more significant losses in an
ORC, a bank should not decide to
replace the probability distributions
used in that ORC with lighter-tailed
curves.
Basel
II
Banks
Act
Reg.
Attest procedures
document, that the distribution with the best relative
performance at different confidence levels is used and that
any deviations from this are documented and expressly
justified.
208
3.
Re-perform the statistical tests performed by the bank (such
as the Likelihood Ratio, the Schwarz Bayesian Criterion and
the Violation Ratio) and compare results to those of the bank.
4.
Where these statistical tests have not been performed by the
bank, this must be noted.
Inspect documented evidence and confirm, and document,
that the process the bank has in place to review and verify
assumptions underlying probability distributions used to
represent ORCs:

Is clearly documented, approved in line with policy
and guidelines and reviewed at least semi-annually.

Any updates to assumptions are thoroughly tested
and signed off by management prior to
implementation, including detailed justification for any
change.

Confirm through the review of developmental
evidence that the bank performs back-testing. Where
back-testing has not been performed, discuss
reasons with management.
1.
2.
3.
4
Distributional assumptions: Scenario Analysis
4.01
A bank should ensure that the loss
distribution(s) chosen to model
scenario analysis estimates
adequately represents the risk profile
of the ORCs. In doing so, banks
should also consider the potential
differences with an LDA in terms of
level of granularity and dependence
across the ORCs.
211
5
Distributional assumptions: Aggregate Loss
Distribution
5.01
The techniques to determine the
aggregated loss distributions should
ensure adequate levels of precision
and stability of the risk measures. The
risk measures should be monotonic,
reasonable and supplemented with
information on their level of accuracy.
5.02
Banks use several statistical
techniques to generate the aggregated
loss distributions from frequency and
severity curves and parameter
estimates. Given the type of
212
213
Inspect developmental evidence and confirm, and document,
that mean expected loss is compared by management to
historical data to assess potential for error.
Where back-testing has been performed, re-perform backtests done by the bank and compare results to those of the
bank.
1.
Inspect the bank’s policy and guidelines and confirm, and
document, that the bank has documented and approved
scenario analysis methodology.
2.
Inspect developmental evidence and confirm, and document,
that scenario analysis is used to assess the impact of
deviations from the correlation assumptions embedded in the
bank’s operational risk measurement framework, to evaluate
potential losses arising from multiple simultaneous operational
risk loss events.
3.
Inspect developmental evidence and confirm, and document,
that assessments are validated and re-assessed through
comparison to actual loss experience to ensure their
reasonableness.
4.
Inspect and confirm that documented policies and procedures
identify when an operational risk event becomes an
operational risk loss for the purpose of collection within the
operational risk loss database and when it is to be included in
the calculation data set.
5.
Inspect the bank’s documentation and process for fitting points
in the tail; reperform the fitting of the scenario analysis data to
a distribution.
1.
Inspect the developmental evidence and confirm, and
document, the techniques selected by management and
confirm and document that management considered
alternative techniques.
Where the bank has performed statistical tests to assess the
various aggregation techniques, re-perform tests performed by
the bank and compare results to those of the bank.
2.
Confirm, by re-performing statistical tests performed by the
bank on the risk measures in the build data, that risk measure
being used is monotonic.
3.
Inspect, and document, management’s developmental
evidence and confirm that the measure is estimated with a low
degree of error that is consistent with the bank’s guidelines.
The bank’s results must be measured against their own
policies. It must be noted by management if the bank’s
threshold is contravened.
Inspect the developmental evidence and confirm, and
document, that statistical techniques used to generate an
aggregate loss distribution result in a positively skewed loss
distribution; (methods used could include for example Monte
Carlo simulations, Fourier Transform-related methods, Panjer
1.
Page 10 of 24
Findings
Management comment
No.
Criteria
BCBS
196
distributions adopted in the context of
operational risk, it is especially difficult
to represent the aggregated loss
distributions by closed form curves. As
such, simulation, numerical or
approximation methods are necessary
to derive aggregated curves (e.g.
Monte Carlo simulations, Fourier
Transform-related methods, Panjer
algorithm and Single Loss
Approximations).
5.03
5.04
5.05
Basel
II
Banks
Act
Reg.
Attest procedures
algorithm and Single Loss Approximations)
A bank should adopt criteria that
mitigate sample and/or numerical
related errors and provide a measure
of the magnitude of these errors,
regardless of the techniques used to
aggregate frequency and severity
distributions.
214
Where Monte Carlo simulations are
used, the number of steps to be
performed is an important variable.
Good modelling practice suggests that
the number should be consistent with
the shape of the distributions and with
the confidence level to be achieved. In
particular, where the distribution of
losses is heavy tailed and measured at
a high confidence level, the number of
steps should be sufficiently large to
reduce sampling variability to an
acceptable level. In order to do this, a
bank can use either (i) a very large
number of iterations or (ii) a dynamic
number of iterations. The latter, which
is typically more accurate, allows the
simulation process to stop when the
marginal variation of the risk measure,
or some other dispersion index, is
close to zero.
215
If Fourier Transform or other numerical
methods are used, a bank should pay
attention to algorithm stability and error
propagation issues.
216
2.
Where Monte Carlo simulation has not been performed,
inspect developmental evidence and through discussions with
management, confirm, and document, management’s
assessment of alternative techniques prior to selecting a
technique.
3.
Re-perform the aggregations performed by the bank and
compare results to those of the bank
1.
Inspect developmental evidence and confirm, and document,
that management use and evidence techniques that minimise
sampling errors (for example clustering of sample data).
Re-perform the tests performed by the bank and compare
results to those of the bank.
2.
Review developmental evidence and confirm, and document,
that the measurement by management of the magnitude of
error is within defined limits (as defined by management) i.e.
the standard error of the sampling statistic is not larger than
expected.
1.
For banks that have performed Monte Carlo simulations,
confirm and document the accuracy of the simulations through
the re-performance of the simulations and compare results to
those of the bank.
Perform a walk-through to confirm, and document, that what
the bank has done is in line with their policy.
2.
Inspect developmental evidence and confirm, and document,
that a sufficient number of simulations have been performed in
line with the bank’s policy.
3.
If a dynamic approach is chosen inspect and document, that
this is justified by management with empirical evidence, for
example comparing results across different number of cycles
and verification and assessment as to whether results are
appropriate or significantly different in relation to dispersion
index.
4.
Confirm, and document, the accuracy of empirical evidence
simulations through re-performance of this alternative
methods performed by the bank Compare results to those of
the bank.
In the event that Fourier Transform or other numerical methods are
used:
1. Inspect evidence and document analysis relating to algorithm
stability; and
2.
Inspect developmental evidence and confirm, and document,
that error propagation has been minimised through
appropriate techniques, for example use of floating-point
numbers.
Accepted risk measures should be used to extract the risk
statistic and justification in support of this risk measure must
be documented by the bank;

If approach considers VaR only then inspect
developmental evidence and confirm, and document,
that management at a minimum includes an
assessment of the quality of data, quality of the
distribution and evidence of back-testing;

If approach considers Shortfall methods then inspect
developmental evidence and confirm, and document,
that management at a minimum includes assessment
justifying preference for a coherent risk measure
including a comparison to VaR and any concerns
relating to distribution fit or quality of data, or
instances where back-testing evidences concerns
that VaR is not sufficiently conservative.
5.06
The risk measure is a single statistic
extracted from the aggregated loss
distribution at the desired confidence
level. The most common and, so far,
most adopted measure in risk
management, including operational
risk, is the Value at Risk (VaR).
However, in certain applications and
fields, including risk management,
Shortfall measures (e.g. Expected
Shortfall, Median Shortfall) have also
gained acceptance in representing the
whole tail region and in providing a
coherent risk estimate (under a subadditively perspective).
217
1.
5.07
Whichever risk measure is adopted, a
bank should ensure that the measure
(and the overall AMA model) fulfils the
monotonic principle of risk, which can
be seen in the generation of higher
capital requirements when the
underlying risk profile increases.
218
1.
Refer to 5.01
5.08
It is also crucial that the risk measures
(while using conservative criteria and
assumptions for prudential purposes)
are realistic from a managerial and
economical perspective. In specific
cases, banks may adopt distributions
219
1.
In the event that a severity distribution that does not have a
defined mean is used in a specific ORC, inspect
developmental evidence to understand the bank’s approach
and inspect detailed evidence that supports this.
2.
At a minimum this evidence would include:
Page 11 of 24
Findings
Management comment
No.
Criteria
BCBS
196
Basel
II
Banks
Act
Reg.
Attest procedures

that envisage the non-existence of the
first moment (i.e. the mean), as this
would determine high capital
requirements and would not be easily
and clearly justifiable and applicable
within the firm.


5.09
5.10
A bank should recognise that the
estimated capital charge is inherently
uncertain due to the heaviness and
scarcity of operational risk losses in
the tail region. As such, the bank
should explicitly recognise this
variability in their estimates and
provide measures of the error around
these estimates.
A bank should also gather information
on the expected loss. Due to its high
sensitivity to extreme losses, the
arithmetic mean can cause an
inaccurate picture for the expected
losses. In light of this, the use of
statistics that are less influenced by
extreme losses (eg median, trimmed
mean) is recommended, especially in
the case of medium/heavy tailed
datasets.
6
Correlation and Dependence
6.01
Dependence assumptions should be
supported to the greatest extent
possible by an appropriate
combination of empirical data analysis
and expert judgment. It is important to
recognise that using internal and
external data to model dependence
presents challenges, as data
limitations observed in the univariate
context (modelling loss distributions for
single ORCs) are likely to be more
significant in the multivariate context
(modelling multiple ORCs). Using
judgment to model dependence
presents its own challenges, as
eliciting accurate but subjective
estimates is more difficult in the
multivariate context than in the
univariate context. As such, the
specification of dependence structures
represents one of the most significant
challenges in AMA modelling.
1.
220
Results confirming that the goodness-of-fit test
statistics are significant, such goodness-of-fit tests
could include the Chi Square, Anderson-Darling,
Kolmogorov-Smirnov;
Results confirming that back-testing evidence that
expected value from loss distribution corresponds to
actual loss experiences.
Evidence that benchmarking of the severity
distribution has been performed and that the results
have been considered.
Inspect the developmental evidence and confirm, and
document, that the degree of error in measuring estimated
capital in relation to relevant models used is documented and
validated by management; using an expected p value or
defined confidence intervals.
Re-perform the statistical tests performed by the bank and
compare results to those of the bank.
Inspect the developmental evidence and confirm that the bank
confirms that their estimation is consistent with their own
policies.
3.
Where the bank performs smoothing or averaging on the
model to eliminate variability, re-perform the smoothing and
compare results to those of the bank.
Where EL is used for capital set-off:

Inspect whether the arithmetic mean is used as the
measure of EL. Where this is the case obtain, and
document, the banks justification of the approach.

Confirm, through the inspection of developmental
evidence, that other measures (like median etc.) are
utilised for analytical EL estimations in cases where
high model sensitivity to large losses could inflate EL
measures such as the mean

Where analytical estimations have been performed,
re-perform these estimations.
1.
221
228
2.
669(d)
Reg
33 (9)
(e) (ii)
(B)
2.
Obtain reasons for differences between analytical EL
estimations and budgeted EL measures from management
and document
1.
In case the bank recognises diversification benefits for
regulatory capital, confirm through inspection of regulatory
approval, that regulatory approval was received to recognise
diversification benefits.
2.
Confirm and document through inspection of the Bank’s policy
document that a motivation for the inclusion of diversification
benefits in the capital estimation is available.
3.
Recalculate a sample of the correlation matrix using the
source data of the model development team.
4.
Confirm through inspection of the Bank’s policy
documentation, that expert judgement is incorporated into the
determination of the correlation matrix and/or the choice of
copula (if applicable). If expert judgement is incorporated,
confirm and document through inspection of the Bank’s policy
documents that documentary evidence contains the motivation
and procedure for incorporating judgement.
5.
Compare the method used to estimate correlation and the
choice of copula (if appropriate) to relevant benchmarks where
these benchmarks are available, (either to the auditors or to
the banks). The auditors should note where there are
discrepancies from these benchmarks.
Page 12 of 24
Findings
Management comment
No.
Criteria
BCBS
196
Basel
II
Banks
Act
Reg.
Attest procedures
6.02
Assumptions regarding dependence
should be conservative given the
uncertainties surrounding dependence
modelling for operational risk.
Consequently, the dependence
structures considered should not be
limited to those based on Normal or
Normal-like (eg T-Student distributions
with many degrees of freedom)
distributions, as normality may
underestimate the amount of
dependence between tail events.
229,
230
669(d)
Reg
33 (9)
(e) (ii)
(A)
1.
Confirm and document through inspection of the Banks
documentation, that sensitivity tests are performed in respect
of different correlation values, different copulas, and different
copula fitting parameters (where appropriate).
2.
Confirm through inspection of the Bank’s documentation and
code used (where applicable) in the modelling of operational
risk, that a degree of conservatism is present, for example:
a. Correlation values are floored at zero;
b. Rank correlations are used instead of linear
correlation;
c. A copula which includes tail dependence is
used, for example the t-copula instead of the
Gaussian copula.
d. Correlation values are adjusted upwards using,
for example expert judgement, or rounding up to
the nearest 5%.
e. In cases where data are not sufficient to
estimate a correlation matrix, perfect correlation
should be used (correlation values equals one).
1.
Through inspection of the Bank’s documentation, document
that the statistical method and/or logical arguments that is
used to confirm the independence of losses with ORCs. Note
that qualitative assessments can be considered based on
BCBC196: “Given that it could be challenging to prove
statistically the independence of losses within an ORC, sound
logical arguments may be used to evaluate the independence
of such losses. For example, losses arising from the same
root cause would not generally be considered independent.”
1.
Confirm and document through inspection of the Bank’s
documentation and code used (where applicable) that the
diversification benefit (percentage difference between
diversified and undiversified capital) is within the ranges
reported in relevant benchmarks, where these are available to
the auditors or to the bank.
2.
Confirm and document through inspection of the Bank’s
documentation and code used (where applicable) that the
number of ORCs is within the ranges reported in relevant
benchmarks, where these benchmarks are available either to
the auditors or from the banks.
1.
Confirm and document through inspection of the Bank’s
documentation and code used (where applicable) that the
bank can demonstrate at least the difference in diversified and
undiversified capital.
2.
Through an inspection and documentation of the sensitivity
analyses performed by the bank, confirm that the bank can
demonstrate the difference in capital with different correlation
values.
3.
Through an inspection and documentation of the sensitivity
analyses performed by the bank, confirm that the bank can
demonstrate the difference in capital with different choices of
copulas (if applicable).
1.
Confirm and document through inspection of the Bank’s
documentation and code used (where applicable) that the
correlation matrices used in the dependence modelling are
The degree of conservatism should
increase as the rigor of the
dependence model and the reliability
of the resulting capital requirements
estimates decrease. Accordingly,
models assuming statistical
independence across all loss events
would require a very high degree of
rigour. Such rigor may be difficult to
attain given the evolving nature of
dependence modelling for operational
risk. It is important to note that the
trade-off between rigor and
conservatism will function only within
certain bounds; supervisors would not
accept a high degree of conservatism
to compensate for an approach to
dependence that suffered from
fundamental deficiencies.
6.03
Losses within each ORC should be
independent of each other. If this is not
the case, either within-ORC
dependence should be modelled
explicitly or the input data should be
modified to achieve independence
across individual losses.
231
6.04
A high number of ORCs may lead to
an unrealistically high capital charge
when no correlations are modelled and
capital charges for all ORCs are
summed together. On the other hand,
a bank modelling correlations that use
a high number of ORCs might have
difficulty finding statistical means to
validate correlation assumptions due
to minimal loss data for each ORC.
168 &
232
669(c)
Dependence should not be
inappropriately affected by the choice
of granularity. For example, many
operational risk management
frameworks assume statistical
independence between losses within
the same ORC. To the extent that a
bank’s framework has only a few
ORCs, the impact of dependence may
be inappropriately minimised. In such
a situation, it may be preferable to
simply add capital estimates across
ORCs.
6.05
6.06
A bank should perform sensitivity
analyses and stress testing (eg
different parameter values and
different correlation models) on the
effect of alternative dependence
assumptions on its operational risk
capital charge estimate. A bank should
have a rigorous process in place
specifying the conditions under which
the results based on alternative
dependence assumptions would lead
to a revision of the operational risk
capital requirements estimate.
233
Given the evolving nature of
dependence modelling for operational
risk, it may be difficult to meaningfully
234
669(d)
669(d)
Page 13 of 24
Findings
Management comment
No.
Criteria
BCBS
196
Basel
II
differentiate the impact of dependence
at one bank versus another. One
would thus expect some degree of
cross-bank consistency in the overall
impact of dependence.
6.07
Banks
Act
Reg.
Attest procedures
calculated in a consistent way across the bank.
Subject to supervisory approval as
discussed in paragraph 669(d), the
incorporation of a well-reasoned
estimate of diversification benefits may
be factored in at the group-wide level
or at the banking subsidiary level.
However, any banking subsidiaries
whose host supervisors determine that
they must calculate stand-alone capital
requirements (see Part 1) may not
incorporate group-wide diversification
benefits in their AMA calculations (e.g.
where an internationally active banking
subsidiary is deemed to be significant,
the banking subsidiary may
incorporate the diversification benefits
of its own operations — those arising
at the sub-consolidated level — but
may not incorporate the diversification
benefits of the parent).
657,
669(d)
2.
Confirm and document through inspection of the Bank’s
documentation and code used (where applicable) that the
copula used in the dependence modelling is the same across
the bank.
3.
Confirm and document through inspection of the Bank’s
documentation and code used (where applicable) that any
expert-based judgemental adjustments made to correlation
values and copula fitting parameters (where applicable) are
done consistently across the bank.
4.
Confirm and document through inspection of the Bank’s
documentation and results of procedures performed by the
bank, that the resultant relative diversification benefit is similar
across the bank.
5.
In case of differences across the bank in any of the above
attest procedures, confirm and document through inspection
that an explanation for the presence of any apparent
inconsistency is available and has been documented.
1.
Through inspection of the Bank’s documentation and code
used (where applicable) document that the factoring in of
diversification benefits at a group-wide level or banking
subsidiary level is done consistent with the AMA approval
received.
Risk measures for different operational
risk estimates must be added for
purposes of calculating the regulatory
minimum capital requirement.
However, the bank may be permitted
to use internally determined
correlations in operational risk losses
across individual operational risk
estimates, provided it can demonstrate
to the satisfaction of the national
supervisor that its systems for
determining correlations are sound,
implemented with integrity, and take
into account the uncertainty
surrounding any such correlation
estimates (particularly in periods of
stress). The bank must validate its
correlation assumptions using
appropriate quantitative and qualitative
techniques.
7
Use of four data elements: Internal
Loss Data (ILD)
7.01
Refer section 2.01 to 2.14 for criteria
and attest procedures relating to
BCBS paragraphs 178 to 191
178
191
7.02
Refer 3.04
196a
Refer 3.04 procedures 4 and 5 relating to BCBS paragraph 196a
7.03
Refer 3.05 for criteria and attest
procedures
relating
to
BCBS
paragraph 197
197
Refer 3.05
7.04
Refer 5.10 for criteria and attest
procedures
relating
to
BCBS
paragraph 221
221
Refer 5.10
7.05
Inputs to the AMA model should be
based on data that represent or the
bank’s business risk profile and risk
management practices. Internal loss
data (ILD) is the only component of the
AMA model that records a bank's
247

-
Refer 2.01 – 2.14
Confirm that internal loss data (ILD) has been used in the
operational risk measurement system (ORMS):
 to assist in the estimation of loss frequencies
 to inform the severity distributions
 to serve as an input into scenario analysis
Page 14 of 24
Findings
Management comment
No.
Criteria
BCBS
196
Basel
II
Banks
Act
Reg.
actual loss experience. ILD should be
used in the operational risk
measurement system (ORMS) to
assist in the estimation of loss
frequencies, to inform the severity
distribution(s) to the extent possible
and to serve as an input into scenario
analysis as it provides a foundation for
the bank’s scenarios within its own risk
profile. Many banks have limited high
severity internal loss events to inform
the tail of the distribution(s) for their
capital charge modelling. It is therefore
necessary to consider the impact of
relevant external data and/or scenarios
for producing meaningful estimates of
capital requirements. (BCBS 196, par
247).
Attest procedures
If only ILD is used in a particular cell to calculate capital, confirm
and document that
-
-
the bank has considered relevant external data and/or
scenarios for producing estimates for the tail of the loss
distribution
the bank has considered the impact of relevant external data
and/or scenarios for producing estimates for the tail of the
loss distribution
7.06
A bank’s internal processes relating to
the collection of loss data shall include
an appropriate de minimis gross loss
threshold amount for the collection of
internal loss data provided that in order
to ensure broadly consistent data
collection between banks that adopted
the advanced measurement approach
for the calculation of their respective
capital requirements relating to
operational risk the Registrar may from
time to time specify a minimum gross
loss threshold amount. (Regulation
33(9)(d)(v)(C))
179,
184
and
187
33(9)(
d)(v)(
C)
Refer 2.01 and 7.01
7.07
A bank shall have in place duly
documented procedures in order to
assess the ongoing relevance of
historical data, which documented
procedures shall duly specify the
situations in which judgment, scaling
or other adjustments to internal loss
data may be used, including the extent
to which such judgment may be used
and the officials who are authorized to
make such decisions (Regulation
33(9)(d)(v)(B)(iv))
178
and
179
33(9)(
d)(v)(B
)(iv)
Refer 2.01 and 7.01
7.08
When a bank’s capital requirement in
respect of operational risk is based on
internal loss data, the said capital
requirement shall be based on a
minimum observation period of:
180
33(9)(
d)(v)(
D)
Refer 2.01 and 7.01
(i) five years of data; or
(ii) when the bank originally adopts the
advanced measurement approach,
subject to such conditions as may be
specified in writing by the Registrar, a
minimum observation period of less
than five years of data, but in no case
less than three years of data,
irrespective whether the internal loss
data is used to calculate or validate the
bank’s measure of loss. (Regulation
33(9)(d)(v)(D))
8
Use of four data elements: External
Data (ED)
8.01
ED provides information on large
actual losses that have not been
experienced by the bank, and is thus a
natural complement to ILD in
modelling loss severity. Supervisors
expect ED to be used in the estimation
of loss severity as ED contains
valuable information to inform the tail
of the loss distribution(s). ED is also an
essential input into scenario analysis
as it provides information on the size
of losses experienced in the industry.
Note that ED may have additional uses
beyond providing information on large
losses for modelling purposes. For
example, ED may be useful in
248
1.
Confirm and document that AMA measurement methodology,
policy documents and procedures:

2.
contain the process for considering external loss data
in the modelling process

contain details on the impact of ED on Capital, where
applicable

contain details that the process is repeatable and
commentary on the volatility caused by the use of
ED, where applicable

clearly distinguish between the use of consortium
and public domain databases

are approved
Confirm through inspection of model documentation that the
bank has considered the reputability of external data vendors
used.
Page 15 of 24
Findings
Management comment
No.
Criteria
BCBS
196
Basel
II
Banks
Act
Reg.
Attest procedures
assessing the riskiness of new
business lines, in benchmarking
analysis on recovery performance, and
in estimating competitors’ loss
experience.
8.02
8.03
While the ED can be a useful input into
the capital model, external losses may
not fit a particular bank’s risk profile
due to reporting bias. Reporting bias is
inherent in publicly-sourced ED and
therefore focuses on larger, more
remarkable losses. A bank should
address these biases in their
methodology to incorporate ED into
the capital model.
249
As ED may not necessarily fit a
particular bank’s risk profile, a bank
should have a defined process to
assess relevancy and to scale the loss
amounts as appropriate. A data
filtering process involves the selection
of relevant ED based on specific
criteria and is necessary to ensure that
the ED being used is relevant and
consistent with the risk profile of the
bank. To avoid bias in parameter
estimates, the filtering process should
result in consistent selection of data
regardless of loss amount. If a bank
permits exceptions to its selection
process, the bank should have a policy
providing criteria for exceptions and
documentation supporting the rationale
for any exceptions. A data scaling
process involves the adjustment of
loss amounts reported in external data
to fit a bank’s business activities and
risk profile. Any scaling process should
be systematic, statistically supported,
and should provide output that is
consistent with the bank’s risk profile.
250
1.
Confirm and document that the bank’s documentation
adequately describes how reporting biases in public domain
databases are considered
2.
If ED is used as a direct input in the capital modelling process,
document and re-perform any adjustments made for bias on a
sample of the data and compare the results with those
obtained by the bank.
3.
Where ED has been used as a direct input in the capital
modelling process, confirm and document whether the bank
has calculated the effect of bias adjustments on the capital
model.
1.
When a bank uses a scaling methodology confirm and
document that:




2.
the documentation describes the methodology as
well as the endogenous and exogenous factors that
are utilised
The scaling factors used have been tested for
appropriateness by management, either by statistical
methods or benchmarking.
Reperform the tests performed by the bank, for a
sample of ORCs. (Sample selected will be the 10
ORCs with the largest individual contribution to the
final capital number. Total sample selection should
contribute no less than 50% to the final capital
number)
Compare results to those of the bank.
When a bank uses a filtering methodology confirm and
document that:


The documentation is clear on the methodology for
excluding events and all exceptions are clearly
documented.
The filtering criteria have been tested for
appropriateness and re-perform the filtering
methodology documented by the bank to confirm that
the results are the same.
When a bank does not use ED directly in the modelling process
confirm and document that the bank has clearly indicated where in
the AMA methodology ED is utilized to inform model inputs and
that the bank demonstrates how the usage enables repeatable and
robust modelling.
8.04
To the extent that little or no relevant
ED exists for a bank, supervisors
would expect the model to rely more
heavily on the other data elements.
Limitations in relevant ED most
frequently arise for banks operating in
distinct geographic regions or in
specialised business lines.
9
Use of four data elements: Scenario
data
9.01
The AMA of a bank requires the use of
four data elements which are: internal
loss data (ILD); external data (ED);
scenario analysis (SA) and business
environment and internal control
factors (BEICFs). This section outlines
the Committee’s expectations with
respect to the use of these four data
elements to produce a credible and
robust estimate of the operational risk
capital charge.
9.02
A robust scenario analysis framework
is an important element of the ORMF.
This scenario process will necessarily
be informed by relevant ILD, ED and
suitable measures of BEICFs. While
there are a variety of integrated
scenario approaches, the level of
influence of scenario data within these
models differs significantly across
251
1.
Confirm and document that the methodology contains a
process for the more extended usage of other elements of the
AMA framework to compensate for a lack of ED.
235
1.
Confirm and document that four data elements are being
used; if not confirm and document that the documented bank
policy explicitly outlines the rationale for this.
252
1.
When a bank uses qualitative processes to inform scenario
analysis with ILD, BEICF and ED, confirm and document that



the bank’s process is documented
The methodology is repeatable, through a walkthrough of the methodology, with commentary offered
on whether governance is followed or not.
the methodology contains a process for addressing
biases
Page 16 of 24
Findings
Management comment
No.
Criteria
BCBS
196
banks.
Basel
II
Banks
Act
Reg.
Attest procedures
2.
When a bank uses quantitative and qualitative processes to
inform scenario analysis with ILD, BEICF and ED, confirm and
document through a walk-through that




9.03
9.04
The scenario process is qualitative by
nature and therefore the outputs from
a scenario process necessarily contain
significant uncertainties. This
uncertainty, together with the
uncertainty from the other elements,
should be reflected in the output of the
model producing a range for the
capital requirements estimate. Thus,
scenario uncertainties provide a
mechanism for estimating an
appropriate level of conservatism in
the choice of the final regulatory
capital charge. Because quantifying
the uncertainty arising from scenario
biases continues to pose significant
challenges, a bank should closely
observe the integrity of the modelling
process and engage closely with the
relevant supervisor.
253
Scenario data provides a forwardlooking view of potential operational
risk exposures. A robust governance
framework surrounding the scenario
process is essential to ensure the
integrity and consistency of the
estimates produced. Supervisors will
generally observe the following
elements in an established scenario
framework:
254
1.
Inspect and document that documentation for coverage on
how the scenario analyses outputs are used in the capital
model and contains justification for the chosen approach. For
a sample of ORCs, re-perform the model using the same
scenario results obtained and used by the bank. Compare
results to those obtained by the bank.(See Section 4)
2.
Perform a walk-through and document the rationale used by
the bank in getting to their scenarios. (See 9.02).
1.
Confirm and document that a policy for scenario analysis
approved at a senior governance committee exists.
2.
Confirm and document that the scenario analysis process and
outputs are subject to the bank’s governance processes and
structures, including various levels of review, challenge and
approval.
3.
Confirm and document that the scenario framework includes
and utilises:

A defined and repeatable process

Background preparation of the participants in the
scenario generation process

Qualified
and
experienced
facilitators
with
consistency in the facilitation process

Representatives of the business, subject matter
experts and the operational risk management
function as participants involved in the process

A structured process for the selection of data used in
developing scenario estimates

Documentation which provides reasoning and
evidence supporting the scenario output

An independent challenge process and oversight by
the operational risk management function to ensure
the appropriateness of scenario estimates

A process that is responsive to changes in both the
internal and external environment

Mechanisms for mitigating biases inherent in
scenario processes. Such biases include anchoring,
availability and motivational biases.
4.
Confirm and document that the bank uses scenario analysis
and external data to assess exposures to high impact events
and losses that may arise from multiple simultaneous
operational risk events.
5.
Confirm and document whether scenario analysis and
external data policies and implementations support the
assessment of high severity loss events, and that double
counting does not occur as a result of using two mechanisms
to assess large exposures.
(a) A clearly defined and repeatable
process;
(b) Good quality background
preparation of the participants in the
scenario generation process;
(c) Qualified and experienced
facilitators with consistency in the
facilitation process;
(d) The appropriate representatives of
the business, subject matter experts
and the corporate operational risk
management function as participants
involved in the process;
(e) A structured process for the
selection of data used in developing
scenario estimates;
(f) High quality documentation which
provides clear reasoning and evidence
supporting the scenario output;
(g) A robust independent challenge
process and oversight by the corporate
operational risk management function
to ensure the appropriateness of
scenario estimates;
(h) A process that is responsive to
changes in both the internal and
external environment; and
the process is documented
the methodology contains a process for addressing
biases
the methodology is sensitive to internal and external
changes
The methodology clearly indicates any filtering
criteria applied when including ILD, ED and BEICF,
and also demonstrates the appropriateness of the
filtering.
(i) Mechanisms for mitigating biases
inherent in scenario processes. Such
biases include anchoring, availability
and motivational biases.
10
Use of four data elements: BEICFs
Page 17 of 24
Findings
Management comment
No.
Criteria
BCBS
196
10.01
The AMA of a bank requires the use of
four data elements which are: internal
loss data (ILD); external data (ED);
scenario analysis (SA) and business
environment and internal control
factors (BEICFs). This section outlines
the Committee’s expectations with
respect to the use of these four data
elements to produce a credible and
robust estimate of the operational risk
capital charge.
235
1.
Confirm and document, through the review of developmental
evidence, that four data elements are being used; if not
documented bank policy should explicitly outline the rationale
for this. (See 9.01)
10.02
BEICFs are operational risk
management indicators that provide
forward-looking assessments of
business risk factors as well as a
bank’s internal control environment.
However, incorporating BEICFs
directly into the capital model poses
challenges given the subjectivity and
structure of BEICF tools. Banks
continue to investigate and refine
measures of BEICFs and explore
methods for incorporating them into
the capital model.
255
1.
When a bank uses BEICF directly in the model confirm by
inspection and document that:

the documentation is clear on the methodology
utilised

any weighting factors used have been tested for
appropriateness, either by statistical methods or
benchmarking

That this testing has been performed correctly,
through re-performance of quantitative statistical
testing used by the banks

The sensitivity in capital to changes in BEICFS is
documented

That the methodology for direct incorporation into the
model is repeatable.
2.
When a bank uses BEICF as an ex-post model adjustment
confirm by inspection and document that:

the process is documented

The methodology is repeatable, through reperformance of quantitative aspects of the
methodology, and a walk-through of qualitative
aspects.

the methodology produces capital adjustments that
are sensitive to changes in BEICFs

the methodology contains approved thresholds for
adjustments

Statistical methods and/or expert judgement that
were utilized in the setting of thresholds are
documented

Statistical methods and/or expert judgement that
were utilized in the setting of thresholds were
performed correctly, through re-performance of
quantitative aspects and a walk-through of qualitative
governance.

that the adjustments are back-tested against
scenarios and ILD

That the back testing has been performed correctly,
through reperformance

explicit regulatory approval was received to perform
ex-post adjustment for regulatory capital purposes
3.
When a bank does not use BEICF as a direct input or for
model adjustments confirm by inspection and document that:

The bank has documented the reason for not using
BEICF directly in the model. The bank has clearly
indicated where and how in the AMA methodology
BEICF is utilized to inform model inputs

The usage of BEICFs in other model input
components is repeatable. Confirm and document
the rationale used by the bank, and that these are
included appropriately in the scenario analysis where
needed. (See 10.03)
1.
When BEICFs are not used as direct model inputs or for expost adjustments, confirm by inspection and document that
BEICFs are used as inputs into the scenario analysis process.
2.
Confirm by inspection and document that the scenario
analysis policy details how BEICFs are used in scenario
analysis.
10.03
BEICFs are commonly used as an
indirect input into the quantification
framework and as an ex-post
adjustment to model output. Ex-post
adjustments serve as an important link
between the risk management and risk
measurement processes and may
result in an increase or decrease in the
AMA capital charge at the group-wide
or business-line level. Given the
subjective nature of BEICF
adjustments, a bank should have clear
policy guidelines that limit the
magnitude of either positive or
negative adjustments. It should also
have a policy to handle situations
where the adjustments actually exceed
these limits based on the current
BEICFs. BEICF adjustments should be
well-supported and the level of
supervisory scrutiny will increase with
256
Basel
II
Banks
Act
Reg.
Attest procedures
Page 18 of 24
Findings
Management comment
No.
Criteria
BCBS
196
Basel
II
Banks
Act
Reg.
Attest procedures
the size of the adjustment. Over time,
the direction and magnitude of
adjustments should be compared to
ILD, conditions in the business
environment and changes in the
effectiveness of controls to ensure
appropriateness. BEICFs should, at a
minimum, be used as an input in the
scenario analysis process.
11
Use of four data elements:
Combining elements
11.01
There are various ways that an AMA
model can be constructed to effectively
incorporate the four data elements. A
bank should carefully consider how the
data elements are combined and used
to ensure that the bank’s operational
risk capital charge is commensurate
with its level of risk exposure. A bank
should provide a clearly articulated
rationale for their modelling choices
and assumptions and conduct
sufficient research and analysis to
support their decisions. The approach
adopted should also encourage
ownership of the outcomes and be
readily understood by the business. It
is highly desirable that there is no
disconnect between the measurement
and the management of operational
risk within the bank. The Committee
recognises that operational risk
modelling continues to evolve and
encourages further investigation into
the combination of the four data
elements within AMA models.
257
The Range of Practice Paper
recognises that “[t]here are numerous
ways that the four data elements have
been combined in AMA capital models
and a bank should have a clear
understanding of the influence of each
of these elements in their capital
model”. In some cases it may not be
possible to:
258 &
259
11.02
1.
When combining the inputs into the capital modelling process
confirm by inspection and document that:

The methodology contains the process for combining
inputs

That the methodology is repeatable, through
performing a walk-through of the process employed,
and re-performance of any specific quantitative
methods used for combining the inputs.

That the methodology contains reference on how
modeller biases are limited and addressed

The sensitivity in capital results due to changes in
parameters that drive the data convolution process
have been documented.

The process of combining and weighting the data
elements is documented
2.
For a sample of ORCs, walk-through and document the
combination and weighting of data elements into the model as
documented by the bank, taking into account modeller biases.
Where there are quantitative aspects of the combination,
these should be re-performed.
3.
Confirm through discussions with management that the
Operational Risk capital numbers which are allocated to each
ORC drive the Operational Risk behaviour of business units.
1.
Confirm through inspection and document whether the role of
each of the four elements in the model are documented.
2.
Confirm through inspection and document that the impact on
capital due to data elements used directly in the model can be
explained and demonstrated, and that results are documented
by management.
3.
Confirm through inspection and document that the bank has
assessed the impact of the different data elements, either
through a using specific pieces of the data or assessing the
impact of removing specific pieces of data on the overall
capital value. Document and quantify these sensitivities. If
this is not possible, then a walk-through of the methods used
by the bank in developing their understanding of the impact
should be performed.
1.
Confirm through inspection and document that the capital
modelling methodology and all associated documentation
include sections that detail the methodologies used for the
combination of the four elements in the modelling process and
ensure that the applied methodology conforms to the
documented methodology via inspection
(a) Perform separate calculations for
each data element; or
(b) Precisely evaluate the effect of
gradually introducing the different
elements.
While in principle this may be a useful
mathematical approach, certain
approaches to modelling may not be
amenable to this style of
decomposition. However, regardless of
the modelling approach, a bank should
have a clear understanding of how
each of the four data elements
influences the capital charge.
11.03
A bank should avoid arbitrary
decisions if they combine the results
from different sub-models within an
AMA model. For example, in a model
where internal and external loss data
are modelled separately and then
combined, the blending of the output of
the two models should be based on a
logical and sound statistical
methodology. There is no reason to
expect that arbitrarily weighted partial
capital requirement estimates would
represent a bank’s requisite capital
requirements commensurate with its
operational risk profile. Any approach
using weighted capital charge
estimates needs to be defensible and
supported, for example by thorough
sensitivity analysis that considers the
260 &
261
Page 19 of 24
Findings
Management comment
No.
Criteria
BCBS
196
Basel
II
Banks
Act
Reg.
Attest procedures
impact of different weighting schemes.
The combination of data elements
within the capital model can provide
the opportunity for the development of
an integrated and self-consistent
modelling framework. However, there
are significant challenges that banks
will need to address when combining
data elements (eg combining scenario
data or ED directly with ILD). The
combination of data elements should
be based on a sound statistical
methodology. The Committee will
continue to monitor progress in the
development of robust techniques to
combine data elements.
12
Estimation of unexpected losses
12.01
Given the continuing evolution of
analytical approaches for operational
risk, the Committee is not specifying
the approach or distributional
assumptions used to generate the
operational risk measure for regulatory
capital purposes. However, a bank
must be able to demonstrate that its
approach captures potentially severe
‘tail’ loss events. Whatever approach is
used, a bank must demonstrate that its
operational risk measure meets a
soundness standard comparable to
that of the internal ratings-based
approach for credit risk, (i.e.
comparable to a one year holding
period and a 99.9th percentile
confidence interval).
667
12.02
Supervisors will require the bank to
calculate its regulatory capital
requirement as the sum of expected
loss (EL) and unexpected loss (UL),
unless the bank can demonstrate that
it is adequately capturing EL in its
internal business practices. That is, to
base the minimum regulatory capital
requirement on UL alone, the bank
must be able to demonstrate to the
satisfaction of its national supervisor
that it has measured and accounted
for its EL exposure.
669
(b)
13
13.01
Reg
33 (9)
(e) (i)
1.
Confirm and document through inspection of the Bank’s
documentation and results produced by the Bank that the UL
is the 99.9th percentile of the aggregated loss distribution.
1.
In cases where the EL is excluded from the solvency
requirement, confirm through inspection of the Bank’s
documentation that specific regulatory approval was received
to exclude EL from the solvency requirement.
2.
Confirm and document through inspection of the Bank’s
documentation and process for producing and communicating
results that the EL is calculated periodically in line with the
periodicity of capital estimation.
3.
Confirm and document through inspection of the Bank’s
documentation and code used (where applicable) that the EL
is calculated using appropriate methods, for example the
mean, median, or truncated mean of the loss distribution.
4.
Confirm and document through inspection of the Bank’s
documentation that an adequate motivation for the choice of
EL calculation method is in place.
5.
Confirm and document through inspection of the Bank’s
documentation and communications that the EL is periodically
compared to subsequent realised losses in order to assess
the accuracy of the EL calculation.
6.
Confirm and document through inspection of the Bank’s
documentation that if actual losses and EL are not comparable
over time, a sufficient explanation is available.
1.
Confirm and document through inspection of the Bank’s
documentation and code used (where applicable) that where
allocation of capital is done, it is done using a risk-sensitive
approach (as defined by management) that relates to the
business areas impacted by the allocation.
2.
Confirm and document through inspection of the Bank’s
documentation and code used (where applicable) that
allocation approaches are motivated and documented.
Risk sensitive capital allocation
As a minimum, a bank that wishes to
adopt the advanced measurement
approach for the calculation of the
bank’s capital requirement in respect
of operational risk shall demonstrate to
the satisfaction of the Registrar that
the bank’s measurement system is
capable of supporting the allocation of
economic capital for operational risk
across business lines in such a
manner that incentives are created to
improve the risk management
capabilities in each relevant business
line.
Reg
33 (9)
(d) (i)
(E)
Page 20 of 24
Findings
Management comment
No.
Criteria
14
Insurance modelling
14.01
A bank that adopted the advanced
measurement approach for the
calculation of the bank’s capital
requirement relating to operational risk
may recognise the risk mitigating
impact of insurance, provided that the
insurance provider
BCBS
196
Basel
II
Banks
Act
Reg.
Attest procedures
Reg
33 (9)
(f) (i)
1.
Confirm and document through inspection of publically
available information, that the insurance provider has a
minimum rating of A, or equivalent, in respect of its ability to
pay claims. Where not, confirm that the bank has received
condonation from the SARB through inspection of the relevant
SARB communication.
2.
Document the relationship between the Bank and its
insurance provider based on information made available by
management, to confirm that the insurance provider is
independent from the reporting bank, that is, a third party
entity or institution, provided that when a bank obtains
insurance through captives or affiliates the bank shall lay off
its risk exposure to an independent third-party entity or
institution.
1.
Confirm and document through inspection of the Bank’s
insurance policies that all insurance policies used in capital
mitigation have an initial term of at least one year, provided
that when an insurance policy has a residual term of less than
one year appropriate haircuts should be in place to reflect the
declining residual term of the policy.
2.
Confirm and document through inspection of the Bank’s
documentation and code used (where applicable) that
insurance policies with remaining term of 90 days or less are
excluded for capital mitigation purposes.
3.
Confirm and document through inspection of the Bank’s
insurance policies that insurance policies have a minimum
notice period for cancellation of 90 days.
4.
Confirm and document through inspection of the Bank’s
insurance policies that the insurance policies do not contain
any exclusions or limitations triggered by supervisory actions
or, in the case of a failed bank, that preclude the bank,
receiver liquidator from recovering for damages suffered or
expenses incurred by the bank, except when an event occurs
after the initiation of receivership or liquidation proceedings in
respect of the bank, provided that the insurance policy may
exclude any fine, penalty or punitive damages resulting from
supervisory actions.
1.
Confirm and document through inspection of the Bank’s
documentation that the method for including insurance
mitigation in the capital estimation is reasoned and
documented by management.
2.
Confirm and document through inspection of the Bank’s
documentation and code used (where applicable) that the
mapping of insurance cover to relevant model inputs or ORCs
is justified by management.
3.
Confirm and document through inspection of the Bank’s
documentation and code used (where applicable) that any
(A) Shall have a minimum rating
of A, or the equivalent
thereof, in respect of its ability
to pay claims;
(B) Shall be independent from
the reporting bank, that is, a
third party entity or institution,
provided that when a bank
obtains insurance through
captives or affiliates the bank
shall lay off its risk exposure
to an independent third-party
entity or institution, for
example, through reinsurance, provided that the
entity or institution that
provides the re-insurance
shall comply with the eligibility
criteria specified in this
paragraph (f)
14.02
A bank that adopted the advanced
measurement approach for the
calculation of the bank’s capital
requirement relating to operational risk
may recognise the risk mitigating
impact of insurance, provided that the
insurance policy –
Reg
33 (9)
(f) (ii)
(A) Shall have an initial term of
no less than one year,
provided that when an
insurance policy has a
residual term of less than one
year the bank shall make
provision for appropriate
haircuts that reflect the
declining residual term of the
policy, which haircut shall be
equal to 100 per cent in
respect of policies with a
residual term of 90 days or
less;
(B) Shall have a minimum notice
period for cancellation of 90
days;
(C) Shall not contain any
exclusions or limitations
triggered by supervisory
actions or, in the case of a
failed bank, that preclude the
bank, receiver or liquidator
from recovering for damages
suffered or expenses incurred
by the bank, except when an
event occurs after the
initiation of receivership or
liquidation proceeding in
respect of the bank, provided
that the insurance policy may
exclude any fine, penalty or
punitive damages resulting
from supervisory actions.
14.03
A bank that adopted the advanced
measurement approach for the
calculation of the bank’s capital
requirement relating to operational risk
may recognise the risk mitigating
impact of insurance, provided that the
bank’s calculations relating to risk
mitigation –
(A) Shall duly reflect the bank’s
insurance coverage
(B) Shall be consistent with the
Reg
33 (9)
(f) (iii)
Page 21 of 24
Findings
Management comment
No.
Criteria
BCBS
196
Basel
II
Banks
Act
Reg.
actual likelihood and impact
of loss used in the bank’s
overall determination of its
operational risk capital
14.04
Attest procedures
insurance deductibles and claim discounts are correctly
considered in the capital estimation.
A bank that adopted the advanced
measurement approach for the
calculation of the bank’s capital
requirement relating to operational risk
may recognise the risk mitigating
impact of insurance, provided that by
way of appropriate discounts or
haircuts, the bank’s methodology for
the recognition of insurance shall duly
capture –
Reg
33 (9)
(f) (vi)
4.
Confirm and document through inspection of the Bank’s
documentation and code used (where applicable) that the risk
mitigating impact of insurance is modelled consistent with the
likelihood and impact of losses used in the bank’s overall
determination of its operational risk capital.
1.
Confirm and document through inspection of the Bank’s
documentation and code used (where applicable) that
discounts (haircuts) are in place to recognize the following:
a. The likelihood that the insurance claim will be
repudiated by the insurer due to a contractual
mismatch between the claim and the cover;
b. The likelihood that the insurance claim will not
succeed due to the bank either not lodging the
claim or exceeding the maximum time period
required to notify claims to the insurer;
c. In case the claim is contractually valid, the
likelihood that the insurer will not be able to pay
the claim;
d. The time-value-of-money due to the time taken
to receive the claim payment from the insurer;
e. The residual term of the policy – if the residual
term is less than one year, a discount should be
applied which increases to 100% for a residual
term of less than 90 days.
2.
Confirm and document through inspection of the Bank’s
documentation that the use of discounts and any exceptions to
the use of the above discounts are motivated and
documented.
1.
In case the bank recognises the mitigating impact of insurance
in respect of regulatory capital, confirm and document through
inspection of the regulatory approval, that explicit regulatory
approval was received to recognise insurance mitigation.
2.
Confirm through inspection and document that the capital
mitigating impact of insurance does not exceed 20% of the
bank’s total Pillar I capital requirement in respect of
operational risk, calculated in terms of the advanced
measurement approach. This verification should be done
considering capital pre- and post-inclusion of the insurance
recoveries. The bank should quantify the impact of including
the insurance recoveries or not.
1.
Confirm and document through inspection of the Bank’s
documentation and code used if model documentation is
adequate to allow a suitably skilled person to replicate the
model independently with minimal interaction with the model
developer.
2.
Obtain and inspect the Bank’s documentation and confirm,
and document, that the following aspects are adequately
documented:
a. Choice and motivation of granularity in respect of
ORCs;
b. Approach to determine the data calculation set;
c. Approach to determine distributional
assumptions in respect of the four data
elements;
d. Approach to determine the aggregate loss
distribution;
e. Approach to model dependence;
f. Approach to perform risk-sensitive capital
allocation;
g. Approach to model insurance mitigation;
h. Approach to determine expected losses;
i. Independent model validation report.
1.
Inspect the bank’s documentation and confirm, and document,
that the validation activities cover the following relevant items:
a. Distribution assumptions;
b. Correlation assumptions;
c. Documentations;
d. The four elements of the AMA, namely:
o Internal loss data
o Scenario data
o External data
(A) The insurance policy’s
cancellation terms and
residual term;
(B) Any uncertainty of payment
(C) Any mismatches in protection
14.05
A bank that adopted the advanced
measurement approach for the
calculation of the bank’s capital
requirement relating to operational risk
may recognise the risk mitigating
impact of insurance, provided that the
bank’s recognition of operational risk
mitigation through insurance shall be
limited to 20 per cent of the bank’s
total capital requirement in respect of
operational risk, calculated in terms of
the advanced measurement approach.
15
Documentation
15.01
As a minimum, a bank that wishes to
adopt the advanced measurement
approach for the calculation of the
bank’s capital requirement in respect
of operational risk shall have in place a
duly documented and robust approach
for the measurement of the bank’s
exposure to operational risk, which
approach, amongst other things, shall
ensure that the bank has in place
rigorous procedures for the
development of a robust operational
risk model.
16
Validation Activities
16.01
The validation activity is designed to
provide a reasoned and well-informed
opinion of whether AMA models work
as predicted, and whether their results
(capital requirement estimates and
other information produced by the
ORMS) are suitable for their various
internal and supervisory purposes.
Reg
33 (9)
(f) (vii)
Reg
33 (9)
(d) (iii)
(A)
56
669 (e)
Reg
33 (8)
(b) (ii)
(B) (i)
(bb)
Page 22 of 24
Findings
Management comment
No.
Criteria
BCBS
196
Basel
II
Banks
Act
Reg.
Attest procedures
Validation activities should:
e.
(a) Have a broad scope, evaluating all
relevant items of the ORMS, such as:







16.02
Distributional assumptions;
Correlation assumptions;
Documentation;
The four elements of the AMA
Qualitative aspects (including
the internal controls, use test,
reporting, role of senior
management and
organisational aspects);
Technological environment
relating to the computational
processes; and
Procedures for the approval
and use of new and modified
estimation models or
methodologies (such
procedures should seek
explicit opinion from the
validation function in the
approval process);
(b) Evaluate the bank’s processes for
escalating issues identified during
validation reviews to ensure that:



f.
o BEICF
Evidence that the AMA model is not only used
for regulatory capital estimation i.e. use test;
Technological environment relating to the
computational processes.
56
666 (f)
Reg
33 (8)
(b) (ii)
(B) (i)
(bb)
1.
Inspect the bank’s documentation and confirm, and document,
the following regarding the escalating of issues identified
during validation:
a. Escalation processes are documented;
b. Issues raised during the validation are
considered and acted upon by senior
management;
c. Issues raised during the validation are escalated
to the bank’s governance committees.
Escalation processes are
sufficiently comprehensive;
All significant ORMS
concerns are appropriately
considered and acted upon
by senior management; and
All significant ORMS
concerns are escalated to
appropriate governance
committees;
16.03
(c) Evaluate the conceptual soundness
– including benchmarking and
outcome analysis – of the ORMS and
of the modelling output;
56
666 (f)
Reg
33 (8)
(b) (ii)
(B) (i)
(bb)
1.
Inspect the bank’s documentation and confirm, and document,
that the following elements of a soundness standard were
assessed by the bank:
a. The model development team’s staffed and skill
complement;
b. Appropriate modelling methods (as defined by
the bank) are used, e.g. LDA;
c. Appropriate model inputs (as defined by the
bank) are used;
d. Material aspects (as defined by the bank) of the
model are benchmarked to peers and/or with
other models;
e. Model results are available at the 99.9th
percentile of the aggregated loss distribution;
f. Model validation is performed independently of
model development;
g. The model approval process includes challenge
by technical specialists;
h. Model results behave intuitively sound in relation
to changes in model inputs and assumptions;
i. The impacts of uncertain model inputs and/or
assumptions are mitigated through conservative
choices for these inputs and/or assumptions.
16.04
(d) Reflect policies and procedures to
ensure that model validation efforts are
consistent with board and senior
management expectations.
56
666 (f)
Reg
33 (8)
(b) (ii)
(B) (i)
(bb)
1.
Inspect the bank’s developmental evidence and confirm, and
document, that model validation is done in accordance with
policies approved by senior management.
2.
Inspect the bank’s policies and confirm, and documents, that
these policies were approved according to the agreed
approval cycle, and by the appropriate governance
committee(s).
Reg
33 (8)
(b) (ii)
(B) (i)
(bb)
1.
Inspect the bank’s policies and procedures and confirm, and
document, that the policies and procedures include the
following critical elements of the validation process:
a. Responsibilities for model development and
validation;
b. Model documentation;
c. Validation procedures and frequency;
d. Audit oversight.
16.05
(e) Assess whether policies and
procedures are sufficiently
comprehensive to address critical
elements of the validation process.
These include independent review;
clearly defined responsibilities for
model development and validation;
model documentation; validation
procedures and frequency; and audit
oversight;
56
666 (f)
Page 23 of 24
Findings
Management comment
No.
Criteria
BCBS
196
16.06
(f) Confirm that the relationship
between the model’s inputs and
outputs are stable and that the
techniques underlying the model are
transparent and intuitive.
56
17
Model Governance
17.01
The development, use and
enhancement of the capital model
should occur within the ambit of a
formal governance process.
Basel
II
666 (f)
Banks
Act
Reg.
Attest procedures
Reg
33 (8)
(b) (ii)
(B) (i)
(bb)
1.
Inspect developmental evidence and confirm, and document,
that validation assessed the relationship between the model’s
inputs and outputs.
2.
Inspect developmental evidence and confirm, and document,
that validation assessed whether the techniques underlying
the model are transparent and intuitive.
Through inspection of the work performed by internal audit or a
similar independent function within the bank, confirm that the
following has been performed:
1.
2.
3.
4.
Inspect that the model methodology was subjected to
challenge, for example a Technical Committee comprising of
risk model specialists.
Inspect that the model methodology was formally approved at
a senior risk management committee.
Inspect that an approved model development policy is in
place. This policy should cover at least the following aspects:
a. Model development is part of a formalised
process which is independent from model
validation and internal audit.
b. The model should produce sufficiently predictive
realistic capital estimates supported, for
example, by motivation and/or evidence of the
following:
i. Capital estimates should be forwardlooking;
ii. Idiosyncratic risk should be captured
with adequate granularity;
iii. Model parameters should be stable;
iv. Results should be sensitive to changes
in risk position;
v. Should produce realistic capital
estimates;
vi. Should be technically sound;
vii. Should be explainable.
c. The continued appropriateness of the model
should be monitored periodically.
d. Model documentation should be sufficiently
detailed to allow an unambiguous understanding
of the model and its applicability.
Inspect if relevant policies require the use of conservatism in
cases where model assumptions and/or model inputs are
uncertain. Inspect that the conservatism was applied in cases
where model assumptions and/or model inputs are uncertain.
Where the above procedures have not been performed internally
by the bank, perform these procedures.
Page 24 of 24
Findings
Management comment