2007 CAS PREDICTIVE
MODELING SEMINAR
PROJECT MANAGEMENT
FOR PREDICTIVE MODELS
BETH FITZGERALD, ISO
Accomplishing Business Goals
•
•
•
2
Project Management
Implementation
Future
Project Management
• Determine business processes that support
strategic goals
– Underwriting decisions
– Pricing decisions
• Develop project plan aligned with strategic
goals
– Model Building
– Technology Development
– Implementation Phases
• Determine project needs
• Monitor actual vs. planned costs/milestones
3
Project Needs
• Team Skills
– Data management
– Analytical/statistical
– Technology
– Business Knowledge
• Data
• Statistical Tools
• Computer Capacity
4
Prior to Modeling
• Formulate the Problem
• Evaluate Possible Data Sources
• Prepare the Data
• Explore the Data with Simple Modeling
Techniques
5
What percent of a model building project
is the data preparation and data
management?
 25%
 50%
 75%
 85%
6
Prepare the Data
• Do quality checks in level of detail needed
for project
• Understand how to prepare individual
variables for use in models
• Need to be practical about number of
classification categories models can
handle
• Need to decide on truncation and
bucketing of variables that are continuous
• Create new variables
7
Data Management Issues
• Matching additional internal policy
information to premium/loss data
– Different points in time
– Tracking & balancing audited exposures
• Different summarization keys – handling of
mid-term endorsements
• Address scrubbing
• Matching to external data for correct point in
time
• Significance of missing values within variable
8
Modeling Procedures and Diagnostics
• Basic modeling training – GLM, Data
Mining
• Decide on appropriate diagnostics
• Evaluate diagnostics
9
Modeling Process
Data Gathering
Data Linking
Data Cleansing
Evaluation
Business
Knowledge
Determine
Predictive Variables
10
Analyze
Variables
Modeling
Business Questions
• What goals are you trying to achieve?
• What results do you expect to see?
• How will you know if the results are
reasonable?
• How do you ensure sufficient knowledge
transfer to business staff?
11
Model Performance
Relative Loss Ratio Lift
Optimal Model
Loss Ratio Relativity
1.3
1.2
1.1
LR Relativity by Decile
1
0.9
0.8
0.7
1
2
3
4
5
6
7
8
Decile of Worst to Best Risk
12
9
10
Model Input/Output
• Model input considerations
– Access to data
– Robustness/quality of data
– Timeliness of refreshed data
• Design Model output for users
– Definition of output – expected loss ratio, pure
premium, loss ratio relativity?
– Provide support for output – reason codes
13
Business Implementation of Model
• Model usage determined by strategic goals
– Underwriting risk decision
– Pricing of risks
– Support of market growth
• Integration of Model into business workflow
decisions
– Consistency in underwriting/pricing decisions
– Compliance with regulations based on
implementation decisions
14
Implementation of Model
Workflows:
• Underwriting
–
–
New Business
Renewal business
• Rating
–
–
15
Pricing
Coverage Adjustment
Implementation of Model
• New Business decision options
–
–
–
–
Write risk
Request additional info on risk
Decline risk
Adjust price/coverage
• Consider model output alone or along with
other information available from application
• Model output needed within seconds for quick
decision
16
Implementation of Model
• Renewal decision options
– Automatic renewal
– Flag for non-renewal
– Adjust coverage level for risk
– Adjust pricing for risk
• Initial Year
– review all in-force policies on weekly or monthly
basis
• Subsequent years
– establish schedule for reevaluation based on
specific underwriting guidelines
17
Implementation of Model
Rating
• Model O/P represents relative loss ratio
factor
• Determine rating selections
• Determine rating process
– Modify application of IRPM plan
– Implement new rating factors based on
Model
– Tier risks into different insurers within
insurer group
18
Technology Development
• Incorporate business implementation decisions
• Decide on how Model will be accessible
electronically
– Web-based interface
– Integrated into existing workflow
– Batch processing
• Develop/Modify Systems
– Phase-in technology
• Model uses information from a third-party vendor
• Determine I/P and O/P criteria
19
Rollout Implementation of Model
• Prepare Announcement/Training
Material for Internal & External
Customers
• Coordinate Implementation Phases
• Monitor Feedback/Adjust
Implementation
• Monitor Results against Strategic
Goals
20
Future of Predictive Modeling
• More refined rating plans
– Industry-sourced or internally developed
– Combination of internally-developed & industry-
sourced risk component variables
• Ongoing updating and maintenance of Models
– Refresh data
– New data sources/variables
– New tools/techniques
– React to new market environments
21