DEVELOPING MACRO-STRESS
TESTS
SESSION 9
1
MINDAUGAS LEIKA
MACROPRUDENTIAL
POLICY FRAMEWORK
I. Macroprudential policy definition, targets, policy
transmission channels and relationships with other policies
(Monday)
II. Institutional structure (Tuesday)
III. Policy tools (Tuesday)
2
IV. Risk identification and quantification: stress testing (This
lecture)
AGENDA
What is macro stress testing?
Macro stress testing framework
Macro ST process
Use of stress tests
3
Did STs fail?
FINANCIAL STABILITY
ANALYSIS
Qualitative judgment
Quantitative analysis
Risks and
vulnerabilities
Stress-testing
Transmission
mechanism
Sensitivity
analysis,
forecasts
4
Shocks
HOW CAN WE GROUP
RISKS:
A) Credit risk
B) Market risk
Usually
arises
gradually
C) Liquidity risk
Arise
instantly
5
D) Operational risk (e.g. failure of
SWIFT, software etc. BoE
quantifies this)
WHAT IS THE PURPOSE OF
MACRO STRESS TESTING?
 Provide quantitative and forward looking assessment of the
capital adequacy of the banking system*.
 Accountability to the public
 Decision-making support
 Measuring the impact of systemic risk
*Source: Bank of England (2013). A framework for stress testing the UK banking system.
6
 Macro prudential policy tool: addresses banking system
vulnerabilities (capital buffers, exposures to particular sector,
absence of diversification, capital planning, investor confidence
etc.)
WHY MACRO STRESS TESTING:
TWO CONCEPTS OF LOSSES
There are two concepts linked to risk mitigation techniques:
I Expected losses (loan loss provisions, loan impairment
charges);
II Unexpected losses (economic capital).
Expected losses are mean loss rate, i.e. amount that bank reasonably
expects to lose. Expected losses are usually covered by loan loss
provisions or loan impairment charges. It is called known part of losses.
Unexpected losses represent volatility of losses, i.e. unknown part.
Shareholders equity is used to absorb these losses. We have to
presume, that banks not only need capital to absorb these losses, but
also have to stay above minimum regulatory capital requirements
through the full business cycle.
7
The targeted level of unexpected losses depends on two factors:
minimum capital requirements and targeted rating.
EXAMPLE I
Example:
Bank has 100 bn in assets with 100% risk weight. Average
interest rate is 8%. Shareholders equity is 8 bn (CAR 8%).
Liabilities (deposits and bonds) is 92 bn. Average return on
liabilities is 4%. At the end of the year bank expects to receive 8
bn in return from its assets and pay to debt holders 3,68 bn.
Return to shareholders is 4,32 bn. However, if bank’s expected
losses are 3% (PDs), return to shareholders is lower (return to
debt holders is fixed!): 4,32-3=1,32 bn.
8
What happens, if during an economic downturn, PDs increase up
to 8% ? Bank’s losses are much higher than expected and equal
to 8 bn. In this case, bank’s income is equal to its losses: 8bn8bn=0. Bank’s payments to debt holders is fixed, hence bank
needs to tap its capital base to pay interest rate: 8-3,68=4,32 bn
capital left. That’s below minimum CAR of 8%. Bank needs to be
closed or recapitalized.
EXAMPLE II
How much additional capital bank needs to hold?
Bank provisions 3% for expected losses and needs additional reserves of 5
bn just to have zero profit. In this case return to its shareholders is 0.
To come up with the worst case scenario, and calculate additional
reserves, we need to perform a stress test and model expected and
unexpected losses.
Under Basel II IRB approach we have to model PDs, LGDs. EADs are
given. Losses are expressed as:
Expected losses=PDs x LGDs x EADs
Under Basel II STD approach non IFRS and Basel I we model loan loss
provisions (LLPs):
∆LLPs=∆NPLs x provisioning rate
9
Unexpected losses can be measured as a number of standard deviations
from expected losses (VaR concept).
FROM SHOCKS TO THEIR OUTCOME:
HOW TRANSMISSION MECHANISM
WORKS
Credit risk
Non- financial
corporate sector
Shocks originating
in Real sector
Real estate sector
Households
Transmission
channels:
exposure
Public sector
Banks
Shocks originating
in Financial sector
Insurance companies
Other institutions
Payment systems
Feedback
effects
Transmission
channels:
common
ownership,
exposure etc.
10
Market, liquidity,
counterparty, contagion
risks
Financial
institutions:
profit/loss,
capital
MACRO CREDIT RISK STRESS
TESTING MODEL
Real estate prices: collateral
value for LGD calculation
Macroeconomic forecasts
CB’s macro model
GDP, Housing prices, interest
rates, FX rate, unemployment
Short-term equations with AR(1) terms and/or ECM:
NPLs depencence on selected macro variables
calculated for 1 to 4 quarters
Long-term equations:
NPLs depencence on selected macro variables
calculated up to 3 years
Equations on a bank-by-bank basis
Banking sector data
Loan migration matrix
Projected additional provisions
On a bank-by-bank basis
NPLs, provisions, credit
growth
FX risk, concentration risk,
income/expense, duration gap models
Unexpected losses
Monte-carlo simulation
Projected net losses/profit
CAR
Number of banks that do
not meet minimum CAR
11
Banking sector data
Interest income, expenses,
credit growth, doeposits,
interest rate etc.
12
TOP-DOWN STRESS TESTING
FRAMEWORK
THREE TYPES OF MODELS FOR
MACRO STRESS TESTING
I Portfolio models (Credit Risk plus; Risk Metrics; Credit Portfolio
View etc.)
II Balance sheet models (Cihak, Boss et all. and modifications).
III Market data based models (CCA).
13
First type of models dominate in private sector, second and third
type dominate in regulatory institutions.
14
MACRO ST PROCESS
Increase capital
buffers
Reduce RWA
Reduce exposure
Review
concentration limits
etc.
15
Losses are
calculated using
either sensitivity or
scenario analysis or
both approaches
Losses in terms of
CAR are presented
Actions
Main risks are
identified and
scenarios
constructed.
III types of
scenarios:
I economic/industry
downturn
II market risk events
III liquidity crisis
Loss calculation and
mapping
Risk identification
MACRO ST PROCESS
UNDERSTANDING THE
INCENTIVES
 There are at least three stakeholders in the stress-testing
process: financial institutions, regulators and the
public/markets.
16
 Usually they have different incentives: regulators want more
data, more time, more extreme scenarios; financial institutions
want to provide less data, use in-house models, usually less
extreme scenarios. Regulators want to find the weakest
components of the banking system, whereas institutions want
to show resilience. Public wants “blood”- know institutions that
fail the test.
MACRO STRESS TESTING
STEPS
Determine the objective of the stress test
Design scenario
Perform stress test
Calculate stress losses
Report results
17
Determine actions
Pillar I
Pillar II
Pillar III
Minimum
capital
Supervisory
review
Market
discipline
Minimum capital
requirement; point in
time assessment
Individual capital
guidance: Pilar I
risks+additional (bank
specific) risks
Stress tests
ICAAP: Internal capital
adequacy assessment
process;
Calculation of
economic capital
18
BASEL II/III
SCENARIO DESIGN (1)
High probability
Low probability
High impact
Main focus
Watch
Low impact
Summary mention
Ignore
19
Shock matrix
SCENARIO DESIGN (2)
Risk mapping: from
systemic risks to
exogenous shocks
Scenario design
Shock calibration
Risk correlation
Macroeconomic models
Scenario output: macro
and financial variables
Adverse
scenario
20
Baseline
scenario
HISTORICAL EXPERIENCE
21
Credit risk depends on the state of economy (business cycle)
DEFINING THRESHOLDS
Expected
(mean losses)
E[x]=μ(x)
P(X)
Pass/Fail criteria and minimum
capital requirements
Confidence interval is identical to default
probability:
A
AA
BBB
AAA
0,07% 0,03% 0,01%
0,1%
Economic capital
Expected shortfall
Loss
22
Loan loss
provisions
APPLYING SHOCKS: NORMAL
AND SHOCKED PDF
P(X)
X2 represents shocked PDs, and as
it has higher variance, it bears
more risk than X1
Loan loss
provisions
Loan loss provisions
Economic capital
Economic capital
X2
Expected shortfall
Loss
23
X1
LOSSES AND BUSINESS CYCLE
X1=f(GDP↑, FX rate, Unemployment↓, Interest rates↓, Concentration↓
etc.)
X2=f(GDP↓, FX rate, Unemployment↑, Interest rates↑, Concentration↑
etc.)
P(X)
X1 represents upward trend
X2 represents downward trend
X1
Loan loss
provisions
Economic capital
Expected shortfall
Loss
Economic capital
24
Loan loss
provisions
X2
25
CALCULATING LOSSES
CALCULATING CAR
Current
Tier I and II
capital
(regulatory
capital)
Current RWA
for: credit,
market and
operational
risks
Satellite credit
growth model
Migration matrices
Loan loss
provisions;
Forecasted from
satellite credit
loss model
26
Net income before
loan loss provisions;
Forecasted from
satellite income
model
Loan loss
provisions;
Forecasted from
satellite credit
loss model
27
USE OF STRESS TESTS
THEORETICAL USE OF STRESS TESTS
What answers stress tests should provide:
 How much capital a bank needs to support its risk taking activities?
(Forward looking)
 Is the current level of capital adequate? (Present)
Lehman Brothers, Bear Stearns, Dexia, JP Morgan…. Did they do it
right?
28
Capital that is available vs. Capital that is needed vs. Capital that
regulators need.
ACTUAL USE OF STRESS TESTS DURING
THE CRISIS
 Stress tests popped out as a tool to address loss in public
confidence
 Confidence was boosted by disclosing individual banks’ results,
scenarios and data about exposures
29
 SCAP (US) vs. EBA (EU).
STS BEFORE AND AFTER CRISIS
After
Very little public disclosure, usually : “All
banks are adequately capitalized, however
challenges remain, thus we will be vigilant”
Comprehensive analysis, data available on
a bank by bank basis. Not all banks pass
tests, capital shortfalls are public
Static analysis
Dynamic analysis
Usually single shocks, VaR based
Macro based, multiple scenarios,
dependency among various risk factors,
CoVaR
In most cases solo, individual entity based
Consolidated at the parent (group) level
Simple models, usually for credit and
market risk separately
Comprehensive models: credit, market,
liquidity risks and lost income
Not necessarily linked to CAR
Linked to CAR
No macroprudential measures or capital
conservation plans
Macroprudential measures (system wide)
and capital conservation plans (individual)
30
Before
CAN STRESS TESTS DETECT
SYSTEMIC RISKS?
 In theory, macroprudential STs should unveil the sources of
systemic risk (see IMF (2012)
 In practice, sources indeed were identified correctly (e.g
housing market in the US, contagion from Greece in the EU
etc).
31
 Magnitude of shocks and subsequently their impact was
miscalculated
32
DID STS FAIL?
WHY STRESS TESTS CAN FAIL?
(1)
We can find many “wrongs”:
Wrong models: too complex
Wrong (absence of) data: where risks were “parked”?
Wrong scenarios: underestimation of tail risk events and contagion
effects
Wrong incentives: no need to rock the boat, public will not understand
Wrong scale: “shadow institutions” escaped
33
Wrong policy measures
WE CALCULATE ECONOMIC CAPITAL
USING 2 OR THREE STANDARD
DEVIATIONS…..
34
If we use a normal
distribution, two standard
deviations from the mean
means we still have almost 5
percent of observations
outside of our horizon (2,5
percent in each tail). This
means, we overestimate
earnings and underestimate
losses.
HOWEVER DURING THIS GLOBAL
FINANCIAL CRISIS VOLATILITY WAS
MUCH HIGHER…..
In August 2007, the Chief Financial Officer of Goldman Sachs,
David Viniar, commented to the Financial Times:
“We are seeing things that were 25-standard deviation moves,
several days in a row”.
As Andrew Haldane, executive director at the bank of England
noticed:
35
“Assuming a normal distribution, a 7.26-sigma daily loss would be
expected to occur once every 13.7 billion or so years. That is
roughly the estimated age of the universe. A 25-sigma event
would be expected to occur once every 6 x 10124 lives of the
universe.”
36
VOLATILITY DEPENDS ON TIME PERIOD
(SAMPLE FROM HALDANE (2009))
0.09
0.07
0.08
Default rates observed historically
Default rates during crisis
Model estimated historically
Model during crisis
0.06
Asset default rate
0.10
DRAFT: COMMENTS WELCOME
A HYPOTHETICAL EXAMPLE
OF A FACTOR WHOSE
RELATIONSHIP TO DEFAULT IS NOT CLEAR UNTIL
Figure 4 Hypothetical example of a factor whose relationship to default is not
A CRISIS PUSHES ITclear
TOuntilNEW
LEVELS
a crisis pushes
it to new levels
2
4
6
8
10
Hypothetical macro-economic factor
37
Source: S&P (2010)
First,
the underlying relationships between a macro-economic factor and, (e.g.) the default rate of an asset may be non-linear in a way that makes it hard to understand the true
relationship from historical data. Figure 4 shows an example of this type of “different”
behavior during a crisis. In this example, it is not until a key macro-economic factor be-
BIMODAL NATURE OF RATING
TRANSITIONS
38
Source: Moody’s (2013) Stress Testing of Credit Migration. A Macroeconomic approach.
WHY STRESS TESTS CAN FAIL?
(2)
39
 Underestimated probability of adverse outcomes (disaster
myopia)
 Reluctance to include severe scenarios
 Willingness to hold capital under less extreme scenario only
 Postponement of crisis
 Reverse engineering: scenarios are such, that bank never
violates minimum CAR
 Short time series in emerging market countries
 Data quality issues
 We usually never look beyond economic capital!
 Historical scenarios are based on historical data. We can not
test anything new using data from the past only
REDUCED FORM VS. FULL-SCALE
STRESS TESTS
 Most of the stress tests banks do are reduced form stress
tests
 Reduced form – Monte Carlo simulation
 Full scale – links with macro variables. Correlations
40
 Why reduced form? If probabilities are unknown, we face
uncertainty. In this case randomization is an answer.
WHY REDUCED FORM IS NOT
SUITABLE FOR MACRO STS
 Reduced form does depend on assumptions about distribution. Beta
distribution has fatter tails than the normal one
 Reduced form ST has very little connection with macro variables
 Is opaque
41
 Is good, once we deal with random, uncorrelated price movements
or volatilities. Is not suitable, once we deal with systemic events or
highly correlated movements
How to incorporate balance sheet adjustments into core models?
How to avoid modeling partial equilibrium situations only, i.e.
include feedback effects and adjustments in broader sectors of
economy?
How to model nonlinearities?
System’s stability is most vulnerable then nobody anticipates
shocks, i.e. risks are underpriced, real estate prices are at their
peaks, GDP and credit grows fast. How to model aggressive risk
taking and rapid build-up of imbalances?
How to model financial innovations and market liberalization
(historical data are not available at all or structural breaks
emerge)?
How to extend stress tests to other (non-bank) financial
institutions?
42
THE WAY AHEAD
IMPORTANCE FOR
MACROPRUDENTIAL POLICY
Based on Borio, Drehmann and Tstatsaronis (2012) objective of the
stress tests is to support crisis management and resolution. Drilling
down we can formulate this objective more precisely:
a) calculation of how much capital should be injected into the system to
prevent credit crunch;
b) Identification of weakest financial institutions;
c) signaling to the market about losses and restoring confidence in the
banking system;
d) Improve risk management practices, models and data collection;
43
e) Story telling: use stress tests to describe shocks, transmission
channels and possible impact on financial system and broader
economy.
SEVEN BEST PRACTICE
PRINCIPLES PROPOSED BY
THE IMF
1. Define appropriately the institutional perimeter for the
tests.
2. Identify all relevant channels of risk propagation.
3. Include all material risks and buffers.
4. Make use of the investors’ viewpoint in the design of
stress tests.
5. Focus on tail risks.
6. When communicating stress test results, speak
smarter, not just louder.
7. Beware of the “black swan.”
44
Source: Macrofinancial Stress Testing—Principles and Practices (2012)
WHAT TO DO?
 Do not constrain yourselves with historical experience and scenarios
(it is not contrary to the “this time is different” syndrome, i.e. one
should think that worst crisis might repeat again or my country is not
necessarily too much different from the ones that experienced crisis
earlier)
 Use judgmental adjustments in scenarios
 Use reverse stress testing more often to find break-even points
(especially important in liquidity stress-testing)
 Make it simple. Last CCAR (2013) emphasized simplicity. Simplicity
means no complicated “black box”: executives should be able to
understand and supervisors to verify
45
In the end, follow the advise by J.M. Keynes: “It is better to be roughly
right than precisely wrong”