Improving early warning indicators for
banking crises – satisfying policy
requirements
Mathias Drehmann and Mikael Juselius
Bank for International Settlements
“Understanding Macroprudential Regulation”
Norges Bank, Oslo, 29–30 November 2012
Restricted
CGFS report No 48
Operationalizing the selection and
application of macroprudential
instruments
Restricted
2
Operationalising macroprudential policies
 Report focusses on 3 high-level criteria that are key in
determining instrument selection and application in practice
 The ability to determine the appropriate timing for the
activation or deactivation of the instrument
 The effectiveness of the MPI in achieving the stated objective
 The efficiency of the instrument in terms of a cost-benefit
assessment
Restricted
3
Report ends with 9 questions and answers
1.
To what extent are vulnerabilities building up or crystallising?
2.
How (un)certain is the risk assessment?
3.
Is there a robust link between changes in the instrument and the
stated policy objective?
4.
How are expectations affected?
5.
What is the scope for leakages and arbitrage?
6.
How quickly and easily can an instrument be implemented?
7.
What are the costs of applying a macroprudential instrument?
8.
How uncertain are the effects of the policy instrument?
9.
What is the optimal mix of tools to address a given vulnerability?
Restricted
4
Report analysis three groups of macroprudential
instruments
 Capital-based tools (countercyclical capital buffers, sectoral
capital requirements and dynamic provisions)
 Liquidity-based tools (countercyclical liquidity requirements)
 Asset-side tools (loan-to- value (LTV) and debt-to-income (DTI)
ratio caps)
 For all tools report proposes ‘transmission maps’
Restricted
5
Transmission map for capital based tools
 Voluntary
buffers
Loan market
Leakages
to nonbanks
↑ lending
spreads
 dividend and
bonuses
Reprice
loans
 credit
demand
Undertake SEOs1
Arbitrage
away
 credit supply
 assets,
especially with
high RWA
Expectation channel
↑ Loss
Absorbency
Tighter risk
management
Increase resilience
Asset
prices
Impact on the credit cycle
Increase capital requirements or
provisions
Options to address
shortfall
Improving early warning indicators for
banking crises – satisfying policy
requirements
7
Introduction
 CGFS (2012): Policymakers need to be able to determine the
appropriate timing for the activation or deactivation of the
instrument
 In this paper we want to find reliable early warning indicators
(EWIs) for systemic banking crises
 What policy requirements do EWIs need to satisfy?
 Need to be evaluated with preference free methodology
 Need to have right timing
 Need to be stable
 Need to be robust
 Need to be understood by policymakers
Restricted
8
Implementing the framework
 We assess a broad range of indicators
 We find
 Credit-to-GDP gap best indicator for predicting crises 2-5 years
in advance
 Debt service ratios highly successful indicator for predicting
crises 1-2 years in advance
Restricted
9
How to evaluate the goodness of an EWI?
 To fully evaluate quality of a signal would need to know preferences
of policymakers, which are unknown (eg CGFS (2012))
 What are costs of acting on wrong signals (false positives)?
 What are the benefits of acting on correct signals (true
positives)?
→ Need to evaluate signalling quality independent of preferences
Restricted
10
The ROC curve
 Policymakers receive noisy signal S
 S higher → higher risk of a crisis
 At which threshold you policymakers act?
1
W1
Fully informative signal
Uninformative signal
Informative signal
W2
1
1
W2
True
positive
rate
True
positive
rate
True
positive
rate
W1
False positive rate
1
False positive rate
1
False positive rate
Restricted
1
11
Area under ROC curve as measure of signalling quality
 Area under the ROC curve (AUROC) provides summary measure
of the classification ability (eg Jorda and Taylor, 2011):
 AUROC 
1
 ROC ( FP )dFP
0
 AUROC=0.5 → uninformative indicator
 AUROC=1 → fully informative indicator
 AUROC ideal measure if preferences are not known
 Benefits
 Can be estimated non-paramterically
 Has convenient statistical properties
Restricted
12
Timing of ideal EWIs
 Ideal EWI needs to signal crisis early enough
 Likely to be 1-2 year lead-lag relationship (e.g. countercyclical
capital buffers)
 Policymakers tend to observe trends before reacting (e.g.
Bernanke, 2004)
 Ideal EWI signal crises not too early
 Introducing buffers too early may undermine effectiveness
(e.g. Caruana, 2010)
 We look at individual quarters within a 5 year horizon
Restricted
13
EWIs need to be stable and robust
 Policymakers adjust policy stance gradually
 Optimal for MP (Bernanke, 2004, Orphanides, 2003)
 Indictor should issue consistent signals
 Consistency of signal tied to persistency of underlying series
(eg Park and Phillips (2000))
 High degree of persistency problematic for statistical inference
 Non-parametric approach
 EWIs need to be robust to different samples and specifications
Restricted
14
Interpretability of EWI
 Evidence that practitioners value sensibility of forecasts more
than accuracy (Huss, 1987) adjust forecasts if the lack justifiable
explanations (Onka-Atay et al (2009)
 Purely statistical approaches are not suitable for policy
purposes and communication
 Our indicators reflect
 excessive leverage and asset price booms (Kindleberger, 2000,
and Minsky, 1982)
 non-core deposits (Hahm et al, 2012)
 the business cycle
Restricted
15
Analysing potential EWIs
 We construct and test a range of potential early warning
indicators building on Drehmann et al (2011)
 We select indicator variables from...
 Credit measures: Credit-to-GDP gap and real credit growth
 Asset prices: Real property and equity price gaps and real
property and equity price growth
 None-core bank liabilities (Hahm, Shin, and Shin (2012)):
 GDP growth
 History of financial crises
 ...and add one new measure:
 Debt service ratio (DSR) (Drehmann and Juselius (2012)):
interest payments and repayments on debt divided by income
Restricted
16
Analysing potential EWIs (II)
 We analyse quarterly time-series data from 27 countries.
 The sample starts in 1980 for most countries and series, and at
the earliest available date for the rest
 Use balanced sample
 We follow the dating of systemic banking crises in Laeven and
Valencia (2012)
 We ignore crises which are driven by cross-boarder exposures
 We adjust dating for some crisis after discussions with CBs
Restricted
17
Persistency
 Several of the variables display dynamics which are hard to
distinguish from I(2) process
 Indicators which have performed well in the past are more
persistent
→ Benefits of a non-parametric approach
Restricted
18
Behaviour around systemic crises
Property pr. gap
Equity pr. gap
GDP growth
-8
-4
0
4
8
12
-20 -16 -12
-4
0
4
8
10
0
-5
-20 -16 -12
-4
0
4
8
12
-20 -16 -12
Prop. price gr.
-8
-4
0
4
8
12
-20 -16 -12
Equity price gr.
-4
0
4
8
12
0
4
8
12
-20 -16 -12
-8
-4
0
4
8
12
-20 -16 -12
-8
-4
0
4
8
12
0
-2 0
-2 0
-1 0
-5 0
.5
0
0
0
0
1
20
10
20
50
1 .5
40
20
-8
his tory
1 00
40
Credit growth
-8
-1 0
-5 0
12
60
Non-core deposit ratio
-8
2
-20 -16 -12
-4 0
-5
-2 0
0
-2 0
0
0
0
5
20
50
5
20
10
40
40
1 00
Credit-to-GDP gap
15
DSR
-20 -16 -12
-8
-4
0
4
8
12
-20 -16 -12
-8
-4
0
4
8
12
-20 -16 -12
-8
-4
Restricted
19
ROC curves for 2 year forecast horizon
.6
.8
1
.6
.4
FP
.8
1
1
.8
1
.6
RO C
.4
.2
.4
0
.8
0
1
.6
.4
FP
.8
1
.6
.8
1
.6
.8
1
1
.8
1
RO C
.6
RO C
.4
.2
.2
.4
History
0
0
.2
FP
.8
1
.6
RO C
.4
.2
.6
.4
Equity price gr.
.2
0
.2
FP
0
0
.8
0
1
.8
1
.6
RO C
.4
FP
.8
Prop. price gr.
.2
.6
.4
.6
.4
FP
.8
1
.8
.6
RO C
.4
.2
0
.2
.2
Credit growth
Non-core deposi ts ratio
0
0
0
1
FP
.6
.4
.4
.2
.2
0
1
FP
0
.8
.2
.4
.2
0
0
.6
.4
.6
RO C
.6
RO C
.6
RO C
.4
.2
.2
.8
1
.8
1
.8
1
.8
.6
RO C
.4
.2
0
0
GDP growth
Equity pr. gap
Property pr. gap
Credit-to-GDP gap
DSR
0
.2
.6
.4
FP
.8
1
0
.2
.4
FP
Restricted
20
1
1
.8
.7
.5
.6
A URO C
.5
.6
A URO C
.7
-15
-5
.4
.3
Property pr. gap
0
-20
-15
Equity pr. gap
-5
0
-20
-15
-10
GDP growth
-5
0
-20
-15
.9
-10
-5
0
-5
0
Horizon
1
1
Horizon
.6
A URO C
.5
A URO C
.6
.7
.7
-15
-10
Horizon
-5
0
-20
-15
-10
Horizon
-5
0
.4
.3
.2
Prop. price gr.
-20
-15
-10
Horizon
Equity price gr.
-5
0
-20
-15
-10
Horizon
His tory
.2
.2
Credit growth
.2
.2
Non-core deposits ratio
-20
.3
.4
.3
.4
.3
.3
.4
.4
.5
.5
.6
A URO C
.7
.6
.5
.5
.6
A URO C
.7
.7
.8
.8
.9
.8
.9
.8
-10
Horizon
1
Horizon
1
Horizon
-10
1
-20
.2
Credit-to-GDP gap
0
.9
-5
.9
-10
.8
-15
.2
.2
DSR
-20
.2
.2
.3
.3
.3
.3
.4
.4
.4
.4
.5
.6
A URO C
.7
.6
.5
.5
.6
A URO C
.7
.7
.8
.8
.9
.9
1
.9
1
.9
.8
.8
.9
1
ROC curves over time
-5
0
-20
-15
-10
Horizon
Restricted
21
Credit to GDP gap and
property price gap
DSR and property price
gap
-20
-15
Horizon
-5
0
1
Credit/GDP gap
-20
-15
DSR
Credit\GDP gap and DSR
-10
Horizon
-5
DSR
.2
Credit\GDP gap and prop. gap
-10
.2
Property gap
.3
.4
.5
.6
A UR OC
.7
.8
.9
1
.9
.8
.7
.6
.5
.4
.3
.4
.5
.6
A UR OC
.7
.8
.9
1
Credit to GDP gap and
DSR
.3
Credit/GDP gap
.2
A UR OC
Combining variables
0
-20
-15
Property gap
DSR and prop. gap
-10
-5
0
Horizon
Restricted
22
Robustness checks
 Robust across samples
 Robust to different crisis dating
 Robust to balanced versus unbalanced samples
 Robust if partial ROC curves are used
Restricted
23
Conclusion
 We argue that EWIs need satisfy six policy requirements:
 Need to be evaluated with preferences free methodology
 Need to have right timing
 Need to be stable
 Need to be robust
 Need to be understood by policymakers
 Appliying this approch to data from 27 countries we find that:
 The DSR and the credit-to-GDP gap dominate other EWIs
 The DRS dominates at shorter horizons and the credit-to-GDP
gap dominates at longer ones
Restricted
24