Using Maths to Manage Risk in
the Financial Service Sector
Matthew Jones
Work experience across public & private sectors
including academia, pharmaceuticals and finance
Background
10/03:
PhD in Statistics
10/03 – 01/07:
Statistical Consultant (Pharmaceuticals)
02/07 – 03/12:
Senior Analyst, Manager, Senior Manager.
03/03 – 07/13:
University Lecturer – Computational Statistics
03/12 – Present: Senior Manager (Model Validation, Credit Cards Risk,
Retail Modelling, IRB Modelling)
Head of Retail Modelling
2
Background
Team
Pre-screening
Modelling
Questions
Team
Retail Modelling Team-Graduates academic background
 Majority of the team are
Mathematics graduates/postgraduates, or have a numerically
based degree
 It is more important that graduate
can combine analytical skills with
knowledge about the
financial/banking industry
 It is also important that graduates
are able to communicate complex
information in a simple manner.
3
Background
Team
Pre-screening
Modelling
Questions
Identifying the best candidates early – leveraging
testing to determine applicants that have requisite skills
Pre-screening
Pre-screening uses tests that can aid in identifying how well a candidate will perform in the
job, and helps hone in on the top candidates to progress to interview
Numerical Reasoning Tests
These typically consist of:
 Tables to identify relevant facts, and working out %’s and averages
 Using basic ratios, decimals and fractions
Financial institutions use them to:
 Determine if the candidate progresses to interview based on a cut-off
 Rank order multiple candidates
CV Vetting
Decisions on whether to progress are also made on information in the CV, such as
qualifications, any relevant work/project experience and fit into team culture
4
Background
Team
Pre-screening
Modelling
Questions
Pre-screening
High volume basic maths tests – candidates required
to perform basic analysis under time pressure
The tests:
 typically multiple choice, ~20 questions in ~20 minutes
 challenge the candidate to work smartly under time pressure
Practise tests are readily available online e.g SHL direct, Unversity of Kent etc.
5
Background
Team
Pre-screening
Modelling
Questions
Practice Test 1
One-to-One exam with candidate – provide information
to the candidate and question in face to face situation
QUESTION 1:
Calculate the Sales per Contact Rate for the whole week
6
Background
Team
Pre-screening
Modelling
Questions
Solution
One-to-One exam with candidate – provide information
to the candidate and question in face to face situation
Solution 1:
Sales per Contact rate for the week:
7
Background
Team
Pre-screening
Modelling
Questions
Practice Test 2
Observe communication style – a good opportunity to
see how effectively the candidate explains their work
QUESTION 2:
Contacts cost £1 each and Sales revenue is £35.
Which day is the most profitable?
8
Background
Team
Pre-screening
Modelling
Questions
Solution
Observe communication style – a good opportunity to see
how effectively the candidate explains their work
QUESTION 2:
Contacts cost £1 each and Sales revenue is £35.
Which day is the most profitable?
Monday (£1,100)
9
Background
Team
Pre-screening
Modelling
Questions
Business Acumen
Test applicants business savvy – opportunity to see how
practical and business minded the candidate is
QUESTION 3:
How could you increase profit?
Possible suggestions:
• Introduce cross training
• Conduct analysis to see why Monday has better targeting
• Pool resource into more profitable days etc.
10
Background
Team
Pre-screening
Modelling
Questions
Modelling
Modelling and Applications of Modelling:
• Retail Risk Modelling is an application of mathematical modelling in the
financial industry.
• Risk modelling is concerned with the quantification of risk.
• For the financial services industry, this could mean building a model that
helps in mitigating future losses from product offerings, holding enough
capital to ensure continuity of business during downturns in the economic
environment.
• Using the information available, models help the organisation to quantify
risks posed by its operations
11
Background
Team
Pre-screening
Modelling
Questions
Modelling
Mathematical / Statistical skills useful in Retail Risk Modelling:
• Functions
• Logarithms
• Distributions
• Mean, Median, Standard Deviation
• Percentages and Ratios
• Probability
• Regression
12
Background
Team
Pre-screening
Modelling
Questions
Retail Modelling
Retail Risk Modelling within the Financial sector
includes the following:
•
Decision Modelling

Application Scoring

Affordability

Customer Management
o
o
13
Limit Increases
Collections
Background
Team
Pre-screening
Modelling
Questions
Decision modelling
What’s involved in a model build ?
Current Account – Application Model Build
• Selecting a development sample /
• Exclusions
representative population
– Observation
– Performance
• Defining what we mean by ‘bad’
– Outcome (over how long a period)
– Severity (harsh or lenient)
• Data sources
– Application (Age, Marital Status, Income)
– Bureau (Income, Credit Searches, Debt)
– Cross-Holdings (Savings)
– Derived variables (Combine Data sources)
Scorecard segmentations
14
Background
Team
Pre-screening
Modelling
Questions
Decision modelling
What’s involved in a model build ?
• The outcome period of a bad definition is decided by
• The severity of a bad definition is decided by
the emergence analysis.
the Roll Rates.
Where should we set the severity of the
bad definition?
Does a plateau occur in the bad rates during the
outcome window?
15
Background
Team
Pre-screening
Hint: At what point are we more likely to
lose money because the customer
cannot pay back?
Modelling
Questions
What is in a scorecard?
Decision modelling
IV expresses the amount of diagnostic
information of a predictor variable for
separating Good from Bad accounts
It is important that a variable is informative
enough to separate Good from Bad accounts
but also important that it is intuitive
Application Scorecard
What would you score on this scorecard?
Any suggestions as to what not to include in a
Scorecard?
What other information may be used in a
current account scorecard?
Do you think the scoring is fair?
16
Background
Team
Pre-screening
Modelling
Questions
Modelling Methodology
Credit Risk Modelling- Credit Scorecards
Modelling Methodology
• Logistic regression is the standard methodology used in Decision Modelling
• To each account, the logit (log odds) of the account becoming a ‘bad’ is assigned
• To make this more palatable for the business, this log odds is converted to a score
Centre Score
• The score where the ratio of ‘Goods’ and ‘Bads’ is 1:1
• The centre score is where the bad rate is expected to be 50%
PDO (Points to Double Odds)
• The number of points to increase or decrease the ratio of Goods to Bads
• For example, we might have a centre score of 200 and a PDO of 20
17
Background
Team
Pre-screening
Modelling
Questions
How model is applied in the business?
Modelling
• A more sophisticated approach is to
Cut-off setting
•
3500
60%
3000
50%
40%
2000
30%
1500
20%
1000
400
10%
500
0
0%
200
220
240
260
280
300
320
340
360
380
Profit per Applicant
2500
Profitability by Score
Cumulative Bad Rate
Volume of Applications
•
look at profitability alongside bad rates.
This will be a balance of accept
rates and bad rates
We could set the new cut-off to
match the current accept rate or
bad rate
200
0
-200
160 180 200 220 240 260 280 300 320 340 360 380 400
-400
-600
-800
Score
-1000
Score
18
Background
Team
Pre-screening
Modelling
Questions
Modelling
Ongoing Monitoring of The model
Population Stability Index (PSI) 
 Expected % 
 Expected %  Observed% ln Observed% 
Bins or Bands
Population Stability helps to view how the current population has changed since the
model was built.
•
•
•
19
Background
Team
Pre-screening
A high Population Stability Index indicates
model instability
This indicates the data used to develop the
model is not representative of current
population
This could suggest a model rebuild or
recalibration is required.
Modelling
Questions
Modelling
Ongoing monitoring of the model
Gini by Month
80%
Gini quantifies how well a model discriminates between
Good and Bad accounts
70%
60%
50%
40%


Gini    G i 1  G i Bi 1  Bi   1
 i

30%
20%
Gini2in24
Gini2in18
Gini2in12
JUN13
MAR13
DEC12
SEP12
JUN12
MAR12
DEC11
SEP11
JUN11
MAR11
SEP10
0%
DEC10
10%
Thresho ld
Kolmogorov
Smirnov
KS Graph
100%
90%
80%
Test
70%
60%
50%
40%
30%
20%
10%
0%
200 220 240 260 280 300 320 340 360 380 400 420 440
Score
Cumulative Goods
20
Background
Cumulative Bads
Team
Pre-screening
Modelling
Questions
21
Background
Team
Pre-screening
Modelling
Questions