Best practice in data & scoring
Dr Paul Russell
Director Analytical Solutions
© Experian Limited 2007. All rights reserved. Experian and the marks used herein are service marks or registered trademarks of Experian Limited.
Other product and company names mentioned herein may be the trademarks of their respective owners. No part of this copyrighted work
may be reproduced, modified, or distributed in any form or manner without the prior written permission of Experian Limited.
Confidential and proprietary.
Agenda
 Some themes
 Analytics and the customer life cycle
 The role of scoring
 Building a scorecard
 Using scoring systems
 Risk management infrastructure
© Experian Limited 2007. All rights reserved.
Confidential and proprietary.
2
Themes
 Best practice is often discussed but almost never seen
 Do the simple things well
 Risk management is more than just a scorecard
 The same principles apply across the credit lifecycle
© Experian Limited 2007. All rights reserved.
Confidential and proprietary.
3
13 ways to grow bad debt
Credit process Target
step
population
Customer
acquisition
New
customers
Description
1. Identifying potential customers;
2. Selling credit products to new customers;
3. Identifying the credit risk of the customer and the proposed
transaction;
4. Identifying the risk of fraudulent application
5. Deciding whether to accept or decline the transaction;
Customer
management
Collections
Existing,
nondelinquent
customers
Existing,
delinquent
customers
6. Deciding, for accepted transactions, on the terms, e.g., credit
amount, pricing.
7. Reviewing the customers facilities (e.g., credit limits, price, etc.);
8. Cross-selling new products to the customers;
9. Ensuring good customers are retained;
10. Identify fraudulent transactions.
11. Identifying self-cure customers;
12. Rehabilitation of potentially good customers;
13. Work-out customers where relationship is broken.
© Experian Limited 2007. All rights reserved.
Confidential and proprietary.
4
Why is credit risk management important?
2,986
European consumer finance
business, Profit Before Tax and
Impairment Charges ($m)
Get it right and it can
support phenomenal
value creation
2,196
1,522
1,520
1,382
Profits
1,374
Impairment
charges
1,230
1,094
924
836
802
740
676
804
764
478
288
1998
340
1999
2000
2001
2002
2003
© Experian Limited 2007. All rights reserved.
Confidential and proprietary.
Source: Annual Reports
2004
2005
2006
5
5 core components
Component
Description
Data
Application data (for new customers)
Account behaviour data (for existing customers)
External data (e.g., credit bureaux)
Statistical Models
Risk models (PD, LGD), fraud models (application and
transaction fraud) and revenue models
Credit strategies
Business rules that translate the outcome of statistical models
in credit decisions (accept/decline, price, credit limits, etc.)
that maximise profit
Implementation tools
Software tools to automate the calculation of the above scores
and credit strategies on-line on high volumes, with a high
degree of flexibility to change credit strategies “on the fly”
Evaluation tools
Software tools to evaluate the performance of statistical
models and credit strategies, and accuracy of implementation
© Experian Limited 2007. All rights reserved.
Confidential and proprietary.
6
Agenda
 Some basic themes
 Analytics and the customer life cycle
 The role of scoring
 Building a scorecard
 Using scoring systems
 Risk management infrastructure
© Experian Limited 2007. All rights reserved.
Confidential and proprietary.
7
Analytics and the customer life cycle
Solicitation
Application
Customer
management
Collections
Debt
recovery
Population
Information
Analytics touches every part of the customer lifecycle
 Analytics touches every part of the customer life cycle
 Amount of information about the customer grows as the relationship
advances through the customer life cycle
© Experian Limited 2007. All rights reserved.
Confidential and proprietary.
8
Analytics and the customer life cycle
Solicitation
• Channel
preference
• Contact history
• Demographics
• Location
• Bureau data
• Action outcomes
• Costs
Application
• Channel
• Product
holdings
• Demographics
• Bureau data
• Previous
relationships
• Account
performance
• Costs
Customer
management
Collections
Debt
recovery
• Product holdings
• Usage
• Delinquency
• Customer
contacts
• Preferences
• Bureau data
• Actions taken
• Action outcomes
• Costs
• Action history
• Promises to
pay
• Promises
fulfilled
• Action
outcomes
• Bureau data
• Costs
• Action history
• Promises to
pay
• Bureau data
• Agents used
• Promises
fulfilled
• Litigation
outcomes
• Costs
© Experian Limited 2007. All rights reserved.
Confidential and proprietary.
9
Analytics and the customer life cycle
Define Goals
Agree objectives
Understand
results
Plan
Assess current
challenger
Review
Assess
Strategy
Review
Design
Monitor
Track progress
against
expectations
Implement
Build new strategy
Ensure operational
deployment
© Experian Limited 2007. All rights reserved.
Confidential and proprietary.
10
Agenda
 Some basic themes
 Analytics and the customer life cycle
 The role of scoring
 Building a scorecard
 Using scoring systems
 Risk management infrastructure
© Experian Limited 2007. All rights reserved.
Confidential and proprietary.
11
The role of scoring
 Credit scoring is a technique for predicting the future
 This prediction can be anything of importance to the business

Arrears

Fraud

Profit

Response

Account closure

Company failure

Etc.
 All scoring is based on one key assumption:

The past predicts the future
© Experian Limited 2007. All rights reserved.
Confidential and proprietary.
12
The role of scoring
Example Scorecard
How does scoring work?
Baseline Constant
•
Scorecards add and subtract
points to a baseline constant
according to individual’s or
account’s data
Applicant Age in Years
< 22
-50
22 - 25
-20
26 – 40
0
41 – 55
+30
> 55
•
Scorecards are easy to apply
and simple to understand
The resulting score gives a
prediction of future behaviour
0
Worst Status L6M (on all Accounts)
0
•
800
0
1-2
-45
3+
-100
Joint Applicant Present
•
Scores are used to rank
individuals to assign the best
actions
Y
+20
N
0
Etc.
…
© Experian Limited 2007. All rights reserved.
Confidential and proprietary.
Etc.
…
13
The role of scoring – application scorecard
•
•
Consider a scorecard built to predict whether a new applicant for a
credit product will default in the next 12 months
This scorecard is used when a new customer applies…
Scorecard
Application
Form Data
Score
External Data
(Bureau etc.)
Take most appropriate
action for each individual
© Experian Limited 2007. All rights reserved.
Confidential and proprietary.
14
Proportion of Applicants
The role of scoring - scores can drive actions
Application Score
Low Score /
High Risk
High Score /
Low Risk
Extremely Low Risk
Extremely High Risk
Reject
High Risk
Standard Risk
Reject or price to cover
the high expected loss
Accept on standard
terms
© Experian Limited 2007. All rights reserved.
Confidential and proprietary.
Consider for crosssell of other
products
15
The role of scoring - benefits
 Best use of data
 Objective
 Consistent
 Automation
 Control
 Reduced losses
© Experian Limited 2007. All rights reserved.
Confidential and proprietary.
16
Agenda
 Some basic themes
 Analytics and the customer life cycle
 The role of scoring
 Building a scorecard
 Using scoring systems
 Risk management infrastructure
© Experian Limited 2007. All rights reserved.
Confidential and proprietary.
17
Building a scorecard – 3 requirements
 Development sample – the historical data on which the scorecard will be built
 Outcome – what we are trying to predict
Some time later
NOW
The recent past
THEN
 Modelling methodology – the statistical tool that will help us form our scoring model
Outcome
Development
Sample
Statistical Model
© Experian Limited 2007. All rights reserved.
Confidential and proprietary.
Scorecard
18
 Representative
 Products
 Business cycle
 The future
THEN
Is my sample any good?
The recent past
Development
Sample
 Robust
 Volumes
 Mature
Is the outcome reliable?
© Experian Limited 2007. All rights reserved.
Confidential and proprietary.
19
•
•
This data can come from a number of sources
All relevant data should go into the development
sample
THEN
Building a scorecard – the development sample
The recent past
Development
Sample
Application
Form
Credit
Bureau
Data
Historical
Account
Behaviour
Other
Account
Information
Information
collected from
the applicant at
the application
point
Information on
the individual’s
other credit
commitments
Information on
the historical
behaviour on
the account
Information on
the historical
behaviour on
other accounts
with the same
lender
© Experian Limited 2007. All rights reserved.
Confidential and proprietary.
20
Building a scorecard – the outcome
•
•
Can be a continuous variable (profit, revenue, loss given default,
etc.)
More commonly it is dichotomous - yes/no




Will this applicant default?
Is this transaction fraudulent?
Will this company fail?
Etc.
NOW
This is the behaviour that we are trying to predict
Outcome
Good
THE FUTURE
Bad
Observation - Now
Outcome - Prediction
© Experian Limited 2007. All rights reserved.
Confidential and proprietary.
21
NOW
What are we trying to predict?
Consumer
Limited business
Non-limited business
Bad
Good
3 payments in
arrears
Not 3 payments in
arrears
Failed
Still going
Bankruptcy, court
judgements or
defaults
No bankruptcy,
court judgements
or defaults
© Experian Limited 2007. All rights reserved.
Confidential and proprietary.
Outcome
22
Building a scorecard – the statistical model
 Many statistical tools available
Statistical Model
 Data is the most important factor
Observation Data
Outcome
Statistical Model
Scorecard
Statistical tool needs to be:
 Powerful – to get the best prediction from the data
 Flexible – can handle varying data types and outcomes
 Interpretable – easy to understand and to overlay business intelligence
 Transparent – should be non-’black box’ for regulatory reasons and to ensure
understanding
© Experian Limited 2007. All rights reserved.
Confidential and proprietary.
23
Building a scorecard - the statistical model
Statistical Model
 Linear regression
Reality
 Logistic regression
 Artificial neural networks
 Etc
 Other things being equal the choice of algorithm
has relatively little impact on the ultimate power
of the model
x x
x
x
x
x x
x x
x
x
x
x
x x
x x
x
x x
x
Prediction
© Experian Limited 2007. All rights reserved.
Confidential and proprietary.
24
Building a scorecard – assessing the model
Does the model solve the business problem?
Statistical Model
 Discrimination – the power to polarise individuals between good and
bad - Gini statistic & Kolmogorov-Smirnov statistic
 Accuracy – how much of the variability of the outcome is explained by
the model
 Validation – ensures that over-modelling has not occurred or that an
anomalous sample has not been used
 Improvement – the new model should outperforms the existing model
© Experian Limited 2007. All rights reserved.
Confidential and proprietary.
25
Agenda
 Some basic themes
 Analytics and the customer life cycle
 The role of scoring
 Building a scorecard
 Using scoring systems
 Risk management infrastructure
© Experian Limited 2007. All rights reserved.
Confidential and proprietary.
26
Using scoring systems
SCORE
CARD
DATA
DECISIONS
STRATEGY
 The data feeds the scoring system, which is used to aid the decisioning
 The decisions a company makes determine its strategy
 It is the aims and strategy of the business that must be considered when deciding
how to use a scoring system, e.g.

Growing the market share

Reducing bad debt

Increasing automation

Maximising response for given marketing cost

Combating fraud
© Experian Limited 2007. All rights reserved.
Confidential and proprietary.
27
Proportion of Applicants
The role of scoring - scores can drive actions
Application Score
Low Score /
High Risk
High Score /
Low Risk
Extremely Low Risk
Extremely High Risk
Reject
High Risk
Standard Risk
Reject or price to cover
the high expected loss
Accept on standard
terms
© Experian Limited 2007. All rights reserved.
Confidential and proprietary.
Consider for crosssell of other
products
28
Using scoring systems - the score distribution
•
•
Score Band
# Goods
# Bads
GB Odds
% Applicants
≤ 400
500
500
1
9.8
401 – 550
700
350
2
10.3
551 – 650
815
163
5
9.6
651 – 700
1008
84
12
10.7
701 – 750
976
61
16
10.1
751 - 800
950
38
25
9.7
801 – 850
1000
25
40
10.0
851 – 900
1050
21
50
10.5
901 – 950
960
16
60
9.5
≥ 951
1000
10
100
9.9
TOTAL
8959
1268
7.1
100
Score distribution is obtained by applying the score to the development
sample
Gives us a prediction for new applicants falling into a given score range
© Experian Limited 2007. All rights reserved.
Confidential and proprietary.
29
Building a scorecard - the score distribution
Score Band
# Goods
# Bads
GB Odds
% Applicants
≤ 400
500
500
1
9.8
401 – 550
700
350
2
10.3
551 – 650
815
163
5
9.6
651 – 700
1008
84
12
10.7
701 – 750
976
61
16
10.1
751 - 800
950
38
25
9.7
801 – 850
1000
25
40
10.0
851 – 900
1050
21
50
10.5
901 – 950
960
16
60
9.5
≥ 951
1000
10
100
9.9
TOTAL
8959
1268
7.1
100
REJECT
REFER
ACCEPT
ACCEPT
WITH XSELL
Score + Policy Rules + Terms of Business = Strategy
© Experian Limited 2007. All rights reserved.
Confidential and proprietary.
30
Agenda
 Some basic themes
 Analytics and the customer life cycle
 The role of scoring
 Building a scorecard
 Using scoring systems
 Risk management infrastructure
© Experian Limited 2007. All rights reserved.
Confidential and proprietary.
31
Implementation – the Business Rules Engine
Data
Rules execution
(Decision Agent)
Rules Definition
(Strategy Design Studio)
Results
© Experian Limited 2007. All rights reserved.
Confidential and proprietary.
Deployed in: Origination
 Application processing
 Portfolio Management
 Customer level
decisioning
 Collections
 Authorisations
 Intelligent Messaging
 Event Management
 Basel II Stress testing
 …..
32
The unsecured lending origination process
Gather & validate
application data
Gather existing
customer
information
A full range of client
options and interfaces for
channel independence
and data accuracy
Invoke enrichment
strategy
Credit bureau links
Online links to gather data
about existing relationships
and customer behaviour
Get decision &
terms of business
Business-driven
scoring & decisionmaking
Detect application
fraud
Application screening
and data matching
Handle referrals
and manual
procedures
Comprehensive workflow
capabilities and provision of
relevant data for users
Get policy decision
& enrichment
strategy
Business-driven scoring
and decision-making
Implement final
decision
Automated account set-up.
Provision of hand-off files.
Letter and e-mail production
© Experian Limited 2007. All rights reserved.
Confidential and proprietary.
33
Defines
Business logic, Segmentation, Scorecards,
Strategies and Champion Challenger
Business logic, Segmentation, Scorecards,
Strategies and Champion Challenger
Active
History
HOST
e.g. Account
Management System,
Authorisation System
etc
Implements
Results
Analytical
Data Mart
Extract
Feedback
Variables
Decision
Engine
Strategy Implementation
Data Manager
Operational environment
Evaluation
Optimisation
Reporting
© Experian Limited 2007. All rights reserved.
Confidential and proprietary.
Rule
Definition
Strategic business
environment
34
Beyond scoring - strategy optimisation
There are disadvantages to traditional champion/challenger testing…
•
The time frame for observing results can be long
•
It can be hard to design the next step
•
The result can become a “semi-random walk”...
We want to get there with
the first challenger !
Value
Challenger
n
Using performance
data enables better
decisions, but is
also more complex
to combine all the
decision influences
to maximise value
Decision strategy
“deploy-learndeploy” process
The challenger
strategy proven in
one time period,
may no longer be
appropriate for
another time period
– things change
Challenger
1
Champion
Challenger
3
Challenger
2
Challenger
4
Time
© Experian Limited 2007. All rights reserved.
Confidential and proprietary.
Influence due to:
• Macro-economics?
• Use of intuition?
• Misunderstanding?
35
#35
Developments in analytics - strategy optimisation
The next step…
Incremental benefit
ROI
Some organisations
are still here
Scoring
X XXX
X
XX
XXX
X X XX
Manual
X
X
X X
X X X XX
X
X
X X
X
XX
X
X X XX X
•
Experience and
intuition
•
Trial and error
Optimised Strategies
Most are here
•
•
Elaborate Strategies
•
Allocates optimal
action for each
customer within
constraints
•
Segmentation
based on predictive
model dimensions:
e.g. risk and
revenue
•
Objective,
mathematical goal
maximisation
•
“Subjective”
judgment used to
manage trade-offs
XX
X X
X X
X X
single predictive
model e.g. credit
risk score
“Heuristic” cut-offs
assigned using
good:bad odds
Underlying decision complexity
© Experian Limited 2007. All rights reserved.
Confidential and proprietary.
36
#36
Stage 1: build the infrastructure
 Centralisation of credit decisioning
 Set-up of IT tools required to automate credit risk and
market management processes and the interaction
between front line and back office
 Development of decision support tools
 Development of credit / marketing databases
Automate the processes
© Experian Limited 2007. All rights reserved.
Confidential and proprietary.
37
Stage 2: fine-tune for performance
 Fine customer segmentation based on customer profile, product
holding and behavior data
 Advanced credit and marketing databases drive increased
sophistication in statistical models development
 Customer interactions for risk and marketing are proactively initiated
at all key points
 Strategies are designed at customer-level
Automate the decisions
© Experian Limited 2007. All rights reserved.
Confidential and proprietary.
38
Stage 3: optimize for excellence
 Infrastructure enables total proactive control of the business –
decision analytics becomes a way of life
 Risk and marketing strategies are centrally designed based on
advanced statistical techniques and drive customer profitability
 Decision analytics is well structured and integrated across business
functions including risk, marketing, sales, operations, finance
Optimize the decisions
© Experian Limited 2007. All rights reserved.
Confidential and proprietary.
39
The Road Map
Build the
infrastructure
Strategy
Fine-tune for
performance
Profit driven credit
strategy in place and
reviewed regularly
Credit policy in
place
How are credit policies
and strategies defined,
reviewed and
improved?
Processes
Include all available
data into the process.
Focus underwriter on
“key” review, not
second scorecard
How well do staff
understand all profit
drivers? What is the
degree of expertise in
credit scoring and
decision science?
Knowledge
Monthly review of
credit strategies.
Champion/challenger a
way of life
Processes regularly
reviewed and refined.
Little manual intervention
Processes well
defined and
automated
How well defined
and the processes,
and what is the
degree of
automation?
Optimize for
excellence
Tools
“a world-class consumer
finance company”
Processes fully
support profit-driven
strategy, and are
integrated across
functions
Education on
strategy review
process, fully
understanding the
use of MIS
Ongoing
knowledge
improvement Full suite of
scorecards,
ability to
optimize credit
strategies
Ability to review
and modify
credit strategies
‘on the fly’
Profit-driven
organisation
across
functions
Create strategy review
cross-functional team
Scorecards in place
for all critical
segments, decision
engine used to
Ensure clear assignment of
control terms of
responsibilities for risk
business. Generate
management functions
key KPI’s
What credit management
tools are used? How
How are credit risk,
flexible are they? How
marketing and finance
easy is it for business user
working together? How
to change processes and
Organisation
are operational and
strategies?
strategic decisions taken?
© Experian Limited 2007. All rights reserved.
Confidential and proprietary.
40
Conclusions
 It all starts with data
 Scorecards are important
 Strategy is more important
 Implementing the strategy properly is vital
 If you don’t monitor you’re wasting you time
 Risk management is a never-ending journey
© Experian Limited 2007. All rights reserved.
Confidential and proprietary.
41
Best practice in data & scoring
Dr Paul Russell
Director Analytical Solutions
© Experian Limited 2007. All rights reserved. Experian and the marks used herein are service marks or registered trademarks of Experian Limited.
Other product and company names mentioned herein may be the trademarks of their respective owners. No part of this copyrighted work
may be reproduced, modified, or distributed in any form or manner without the prior written permission of Experian Limited.
Confidential and proprietary.