Evaluating the Evaluation
Bank Evaluation Programs
Doug Potts, MAI, VP
Commerce Bank
A Few Questions
• True/False?
– Most loan transactions require appraisals.
• True/False?
– Evaluations are killing the appraisal profession.
• True/False?
– Reg B will eventually create more business for appraisers.
• True/False?
– Reg B will - in time - increase available consumer credit.
A Primer on Bank Lending
• Major Purposes
• New Loans (new collateral)
• Subsequent Transactions (existing collateral)
• Problem Loan Management (“exiting” collateral)
• Major Loan Categories
• Consumer purpose lending (GSE / HELOC)
• Commercial business lending (C & I)
• Commercial investment lending (CRE)
• Unclassified lending – the “junk drawer”
Bank Portfolios--Risks & Rewards
•
•
•
•
Where does most of a bank’s workload sit?
Do renewals generate new $ for added cost?
New loans generate growth in profits
80/20 rule with a vengeance
• Loan profitability sits in the 20%
• But the workload sits in the 80%
• Renewed loans generate $0 added profit
• But comprises the bulk of a bank portfolio
LOAN STRATIFICATION v PROFITABILITY
$4,000
6,000
$3,500
5,000
$3,000
# of Notes
$2,500
$2,000
3,000
$1,500
2,000
$1,000
1,000
$500
$-
-
<$100M
<$250M
<$500M
Loan Size Bracket
<$1MM
> $1MM
Earnings MM
4,000
Job Stratification by Function
Appr
Eval NM
Eval Valid
23%
48%
29%
The Point Is ….
• When you combine the 80/20 rule with the $0
profit rule ……
• You need a cost-effective solution to meet
• Underwriting need
• Regulatory requirements
• The key is “adequacy”
The Purpose of Evaluations
• Regulations require every loan get a value
• Only some deals require an appraisal.
• But they also provide for cost effectiveness
• Spread thin but broadly
• Load balance tool
• Balance risk & reward
• Risk sensitized
• For loan size (e.g. <$250M de minimis)
• For repayment reliance (CRE v C&I v Consumer)
• For portfolio concentration & workload (new v renewal)
Program Basics
• External Vendor Procured
• Proliferating Options
• Growing array of vendors
• Growing array of evaluation styles & products
• Coverage & Fulfillment
• Internally Developed
•
•
•
•
Licensed or Unlicensed Team Members
Qualifications & Training
Data Sourcing & Inspections
QA / QC Controls
Evaluation – Scope of Work
•
•
•
•
•
Interagency Appraisal & Evaluation Guidelines
Flexibility v rigidness
When & where to use
Defining the problem
Creating the solution
What will your regulator permit?
2010 IAG Requirements
•
•
•
•
•
Should contain sufficient information
Details analysis, assumptions, conclusions
Should be documented in the credit file or reproducible.
Identify the location of the property.
Provide a description of the property and its current and
projected use.
• Provide an estimate of the property’s market value in its
actual physical condition, use and zoning designation as of the
effective date of the evaluation (that is, the date that the
analysis was completed), with any limiting conditions.
• Describe the method(s) the institution used to confirm the
property’s actual physical condition and the extent to which
an inspection was performed.
2010 IAG Requirements #2
• Describe the analysis that was performed and the supporting
information that was used in valuing the property.
• Describe the supplemental information that was considered
when using an analytical method or technological tool.
• Indicate all source(s) of information used in the analysis, as
applicable, to value the property, including:
– External data sources (such as market sales databases and public tax
and land records);
– Property-specific data (such as previous sales data for the subject
property, tax assessment data, and comparable sales information);
– Evidence of a property inspection;
– Photos of the property;
– Description of the neighborhood; or
– Local market conditions.
2010 IAG Requirements #3
• Include information on the preparer when an evaluation is
performed by a person, such as the name and contact
information, and signature (electronic or other legally
permissible signature) of the preparer.
Let’s Re-Arrange the IAG Ideas
Evaluations have 5 major elements
• Define the problem to be solved
• Get needed research elements
• Make needed inspections as appropriate
• Selecting the right data
• Doing the math & calculations
Evaluation Scope Defining the
Problem
Calculating
an
Answer
Selecting
Appriorate
Data
Research
&
Descriptions
Inspection
&
Descriptions
Eval Content - Preliminaries
• Defining the Problem
•
•
•
•
•
Complexity
Effective date(s) – what is this?
Limiting conditions
Can the premise be prospective?
Can the zoning be prospective?
• Basic research & descriptions •
•
•
•
Property specific records
Neighborhood & market trends
Current & projected use
Current zoning designation
Eval Content – Inspection Elements
•
•
•
•
•
Methods to confirm physical condition
Describe extent of inspection
Evidence of inspection
Descriptions inspection details - granularity
Property photos
More on Inspections
• “Methods to confirm physical condition”
• “Describe extent of inspection”
• What about?
• Out of territory property
• Borrower or lender representation?
• Shelf life of the representation/inspection?
• Origination v Renewal?
• Flexible standard?
• Credit Quality? Pass v non-pass?
Eval Content –Selecting Data
•
•
•
•
“As applicable” data
External sales databases etc.
Local market conditions
Comparable sales
Eval Content – Calculate an Answer
• Provide value in actual physical condition
• Is that the only condition allowed?
• Describe analysis performed
• Describe supporting information
• Data sourcing & selection
• What valuation technique? What methods?
• Analytical method or technological tool
• Analysis performed & supporting information
• Supplemental information
Eval Content - Fingerprints
•
•
•
•
•
“When performed by a person”
Name / contact info
Actual signature
Electronic signature
Is the evaluator qualified?
• What about state restrictions?
Evaluation Scope Defining the
Problem
Calculating
an
Answer
Selecting
Appriorate
Data
Research
&
Descriptions
Inspection
&
Descriptions
Limitations
• Shelf life of evaluations
• Tailoring for loan-specific issues
• LTV adequacy
• Lease structures
• Environmental impacts & offsets
• As-is value v prospective value
Evaluation Options & Risk
What program to develop and use?
“You get to pick your problems, you don’t get to
pick no problems….”
• Internal Program
• External Vendor Program
• Appraisal Only – “no evaluations”
• Hybrid?
• What about BPO?
The Nexus -- Risk v Regulation
• Triangulating competing elements
•
•
•
•
Accuracy
Adequacy
Efficiency
Cost effectiveness
• Does an evaluation need to be accurate?
•
•
•
•
Is accuracy always cost effective?
Is precision always necessary?
What about low LTV transactions?
Using a shotgun to shoot a fly…
External v Internal Eval Programs
• Consider the business model -• Collateral class – conventional v oddball
• Market coverage – urban v tract suburb v rural
• External Model
• Consistency & uniform product – predictable
• Within its limits – highly reliable system
• Fulfillment failure risk
• Internal Model
• Flexibility & adaptability
• Precision subordinated to practicality of business
• Disclosure Risks
External Products
•
•
•
•
•
•
Appraiser-centric or broker-centric
Available options
How to compare them?
How to review them?
Fulfillment issues
E & O dynamics
Vendor List
• Product type – residential v commercial
• Geographic coverage
•
•
•
•
National
Urban coverage
Rural coverage
Appraiser- v broker-centric
• Branch from national appraisal firm
• Local vendor based approach
Some Options
Name
Type
Coverage
Best
Concentration
Performed by
Boxwood
Comml
Natl
Urban/Suburb
Appraiser
Mountainseed
Comml
Natl
Urban/Suburb
Appraiser
Clear Capital
Comml & Res
Natl
Urb/Sub/Rural
Broker
Collateral Analytics Res 1-4
Natl
Urban/Suburb
AVM
Corelogic
Comml & Res
Natl
Urb/Sub/Rural
Appr Brkr AVM
DataQuick
Res 1-4
Natl
Urban/Suburb
Appr Brkr AVM
First American
Comml & Res
Natl
Urban/Suburb
Appr Brkr AVM
Inside Valuation
Comml & Res
Natl
Urban/Suburb
Broker
ProTeck
Res 1-4
Natl
Urb/Sub/Rural
Appr Brkr AVM
Valligent
Res 1-4
Natl
Urban/Suburb
Appr
ValueNet
Res 1-4
Natl
Urban/Suburb
Appr AVM
Veros
Res 1-4
Natl
AVM
Comparison Methods
1334928.3
--
--
1004000579
2/21/2013
Y,N
121026-52344A
121026-52450A
10/26/2012
YY
11/19/2012
Sanitized, MN
2/12/2012
Y,Y
2/12/2012
Y,Y
12/23/2011
Y,Y
Y,Y,Y,Y,Y
Y,Y,Y,Y,Y
Y,Y,Y,Y,Y
Y,Y,Y,Y,Y
Y,Y,Y,Y,Y
Y,Y,Y,Y,Y
Y
N
Y - narrative
N
N
Y
N
N
N
N
N
N
Y
Y
Y
Y
Y
Y
N
N
N
N
N
N
Y
Y
Y
Y
Y
Y
N
N
Y
Y
Y
Y
Sep Rpt
Sep Rpt
Y
Y
Y
Y
Y - inf/eq/sup
N - no analysis
Y - ranking
Y - AVM
Y - AVM
Y - ranking
Y - comps
Y - comps
Y - comps
Y
Y
Y - comps
N
N
Y
Y
Y
Y
N
N
Y
Sep Rpt
Sep Rpt
N
N
N
Sep Rpt
Sep Rpt
Y
Y - MLS
Y
Y
Y
Y
Y - MLS
Y
Y
Y
Y
Y - MLS
Y
Y
Y
Y
Y - MLS
Y
Y
Y
Neighborhood Description
Local Market Conditions
Preparer Name & Signature
N
N
N
Sep Rpt
Sep Rpt
N
Y
Y
Y
Y
Y
N
Y
Y
N
Y
Y
Y
Order portal avail?
Y
Y
Y
BOK, Soverign,
Bank of West;
State Farm
Y
Y
Order #
Date
Property Location
Property Description
Property Current Use
Property Projected Use
Property Market Value Est "As-Is"
Property Zoning Designation
Effective Date of Value
130219-53537A
Address; assmt tax ID
Bldg size, bldg age, lot
size, bed, bath
Bldg Use Type (SFR, MFH,
Condo, Offc, Retail, Ind,
etc.)
Proposed construction or
renovation, change in
bldg use
Must state "value" not
"price"
Local zoning citation
"as is" = Date of Drafting
"as proposed" = Future
Date
Limiting Conditions
Physical Condition Confirmation
Describe Method of Analysis
Supporting Information - Sales
Supporting Information - Listings
Describe Supplemental Information
Indicate External Data Sources
Indicate Property Specific Data
Indicate Evidence of Property Inspect
Property Photos
Inspection or not; proof
of inspect
Unknown -- is a display
grid sufficient?
Unknown -- is a display
grid sufficient?
Unknown -- is a display
grid sufficient?
Sales history; assmt
Other banks using?
Independent 3rd party validation avail?
Validation Frequency?
Can product be upgrade to appraisal?
Compliance with C & R
Optional items:
PNC; Suntrust
No
No
NA
Monthly to Annual
Yes extra fee
Yes extra fee
Yes extra fee
Cert Appr Signs
Yes extra fee
non-appr revs
non-appr revs
Cert Appr Signs
Evaluation Review Needs
•
•
•
•
General v specific
Checklist only?
Part of the purpose is speed/efficiency
Drags on velocity
• Secondary data confirmations
• Secondary corroborating data
• Secondary inspections
Evaluation Review Questions
1.
2.
3.
4.
5.
6.
7.
8.
Adequately identifies the collateral property (e.g. legal description, tax number,
street address, etc.)?
Adequately describes the collateral’s current and (if applicable) projected use?
Provides an estimate of the collateral’s market value in “As is” condition & use &
(if applicable) zoning as of the effective date of the Alternate Valuation?
Indicates all source(s) used in the analysis, as applicable, to value the property
(e.g. external data sources, property specific data, photos, neighborhood
description, description of local market conditions)?
Describes the analysis that was performed and the supporting information that
was used in valuing the property?
Evidence of a sufficient property inspection (e.g. photographs, comments)?
Includes all other relevant information (described as needed in Comments
below)?
Report includes preparer name and signature?
A Primitive Example
Here’s an Interesting Wrinkle
This review intends to fulfill Regulation B requirements for
Residential 1- to 4-unit dwellings. Transactions secured by 1- to
4-unit dwellings that qualify for an evalution by law may be
valued for xxxx by outside providers. The valuation document
under review may be labeled “Appraisal;” but because of its
limited content, it is reviewed under “evaluation” content
standards outlined in the Interagency Appraisal & Evaluation
Guidelines (effective Dec-2010).
Vendor Management
• Fulfilling possibilities
•
•
•
•
•
List of serviced counties
How many active vendors
How many jobs by county last year
Avg fee by county
How many fulfillment failures by county
• Contracts – NDAs - Processing
• The follow-up process
• What changes can be incorporated to improve?
• Portals & ownership
To AMC or Not to AMC
•
•
•
•
•
•
•
Geographic coverage
Panel control
Customary & Reasonable Fees
Waterfall process - upgrades
Portals
Invoicing
Compliance challenges
• Would it just be cheaper to bring it in house?
Automated Valuation Modeling
•
•
•
•
•
Products & Options
Hit Rates
Confidence Intervals & FSD
AVM Cascades & Selection Tables
Program Benefits
• Speed – returns in moments
• Cost effective – even in a cascade/waterfall
AVM Validation - Challenges
•
•
•
•
•
•
•
Regulatory compliance – modelling policy
Sample spread
Sample size
Sample cost
Sample frequency
Validations methods applied to each event
Who owns the validation process?
Trends in Vendor Mgmt
• Fed management requirements are heavy
• And getting heavier
• Escalating compliance costs
• Many banks are considering moving to inhouse programs
• More product control
• Compliance is no worse than vendor management
• Cost is ≤ to vendor management
Internal Evaluation Risks
• Adequacy of Facts
• Sufficient documentation
• Potential inaccuracies
• Adequacy of Opinions
• Recognize all needed elements?
• Disregard elements?
• Value precision?
• Does adequate alone = misleading?
• Misleading to whom?
• Potential inaccuracies acceptable – big picture
• Risk is internal business decision
Disclosure Risks
• Does it matter if an opinion is not disclosed?
• Can a consumer recognize the difference between an
evaluation and appraisal?
• Consumer wounded ego.
• What about credit denial?
• Consumer uses a disclosed eval in unintended ways
•
•
•
•
•
Eval walks to another bank – portability
Divorce litigation or tax appeal
Price representations & negotiations
Personal financial statement
Estate planning or donation
What About Fed/State Compliance
• The enforcement referral
• The court subpoena
• The patchwork quilt of state regulators
• Mandatory v non-mandatory states
• Scope of practice Issues
• State enforcement risk to an internal program
• Built-in exemptions
• Fed requirements for qualified evaluators
• What about AI Ethics/Standards?
Comparing possible programs
•
•
•
•
Outside vendor appraisal
Appraisal-centric eval
Internal team eval – non licensed
Internal team eval –licensed
Vendor Appraisal – Conventional
Research
Intensity
Inspection
Intensity
Results
Precision
Delivered Cost
Fulfillment
Probability
Threat to State
Licensure
Vendor Appraisal-Centric Eval
Research
Intensity
Inspection
Intensity
Results Precision
Delivered Cost
Fulfillment
Probability
Threat to State
Licensure
Internal Eval – Licensed Staff
Research
Intensity
Inspection
Intensity
Results
Precision
Delivered Cost
Fulfillment
Probability
Threat to State
Licensure
Internal Eval – Unlicensed Staff
Research
Intensity
Inspection
Intensity
Results Precision
Delivered Cost
Fulfillment
Probability
Threat to State
Licensure
Can BPOs Help?
• Shifting risks from in-house to …..
• Out-house?
• The importance of broker/agent skill
• Local knowledge & customary thinking
• Can BPOs function as in-lieu evaluations?
• The IAG restriction?
• Work arounds?
• State brokers regulatory prohibitions
• NAR lobbying in the States
What is NAR Up To?
• Challenges from Zillow’s sometimes misleading zestimates
• RealtyAlliance v Local MLSs
• The RET server:
– Who owns the MLS data? What level of access?
• RPR – the NAR AVM – 600 MLS servers feed it.
– Not making money. Trying to grow
• New NAR rules allow broker developed AVMs in all markets
• Issuing rules to keep MLS data from being proprietary by local
MLS systems
• NAR pursuing inroads to allow brokers to prepare evaluations
for financial institutions without a potential listing.
What are Peers Doing? from CART
CART -- How many internal evaluations last
year?
2013
45
2014
CART - What does Bank permit for eval
support?
2013
40
# of Banks
# of Banks
35
30
25
20
15
10
5
1-100
101-1000
101-5000
>5000
0
Volume of Evals
CART -- do you use the tax value w adj factor?
50
45
40
2013
2014
35
# of Banks
0
30
25
20
15
10
5
0
Never
Up to
$250M
Up to
$500M
Up to
$1MM
Up to
$5MM
Over
$5MM
Other
2014
Dodd Frank & Reg B / ECOA
• Key New Definitions
• Borrower
• Transaction
• Collateral – what is a dwelling?
• Notification v disclosure
• Consumer purpose or
• Secured by 1-4 Unit Dwelling
• The Dwelling Trump Card
• Mission creep from law to rule-making
Reg B Notice & Disclosure
LOAN PURPOSE V COLLATERAL CLASS PROBLEM
LOAN PURPOSE
COLLATERAL
CLASS
Consumer
Business
Reg B Mandated
1-4u dwelling
Notice & Disclosure
No Notice & Disclosure
Reg B Exempt
Everything else
Notice / No disclosure
No Notice / No disclosure
Is This a Dwelling?
Reg B Impacts Community Banks
• Increases financing & liability risk
• Forces disclosure of potentially inaccurate but otherwise
adequate analysis
• Requires deal restructure – financing unsecured to avoid
documentation of value
• Potentially inaccurate but otherwise adequate can be
misconstrued or misused by borrower
• Alternative is to order more vendor product to deflect risk
• Limits credit access to community borrowers
• Inflexible rules don’t sync with the physical collateral world
• Added cost of vendor product
• Added delay of vendor product
Reg B Impacts Consumers
• Increases confusion
• Blurs distinctions from consumer to biz deals
• Treats all “valuations” equally for disclosure
• Increases cost
• Vendor products more costly than internal products
• Prioritizes precision over adequacy
• Increases delay
• Vendor products take additional time
• Risk of fulfillment failure
Homebuilder Impacts
• Reg B intended to give each of thousands of
individual consumers an individual appraisal.
but instead ….
• Reg B requires a handful of homebuilders to
receive thousands of appraisals
Hybrid Product – the BPO Eval
• Can it “shift the risk” ?
• How to qualify the valuation elements?
•
•
•
•
Opine price or value -- both?
IAG limitations
Review based qualification of BPO opinion?
Separate opinion using BPO elements only?
• Convert into bank-issued estimate of value
• Who inks it?
• Qualification v state compliance risk
• Wiggle words - disclaimers
• E & O need?
BPO Vendor – Evaluation Hybrid
Research Intensity
Inspection Intensity
Results Precision
Delivered Cost
Fulfillment Probability
Threat to State Licensure
Appraisers Re-enter the Eval Space
• USPAP & state restrictions
• Economics
• E & O coverage
Techtonic Shifts
•
•
•
•
•
•
•
AI Standards v USPAP
FRTs v RET? Where do evaluations fit?
Should we cage USPAP into FRT only?
HR 5148 (Leutkemeyer TILA High Risk Mortgage Bill)
Federal pre-emption – supremacy clause
See Gibbons v Ogden 22 US 1 (1824)
See Gade v National Solid Wastes Mgmt Assoc 505
US 88.98 (1992)
• See Florida Lime v Avocado Growers Inc v Paul 373
US 132, 142-43 (1962)
The Pre-Flight Evaluation
• The good-housekeeping seal of approval
• Post-eval changes to loan
– Impact on eval reliability
• Ordering a subsequent appraisal
– Valuation priority of place
Foreclosure Evaluations
• What is allowed – de minimis rules.
• Risks v benefits of use
• Can you manage the inspection?
• How to manage accounting needs
• Fair Value
• Liquidation / Distress Value
• Pre-foreclosure requirements
• Post-foreclosure surprises
The Forensic Evaluation
•
•
•
•
•
Multiple evaluations over time
Same asset
Allows a measure of market change
Retrospective analysis to test an appraisal
Showing editorial selection process
• What comps were uncovered but not used
Evaluation in Portfolio Monitoring
• The Nexus of Accuracy v Adequacy
• Existing collateral & the subsequent transaction
• Transaction cost v flat profitability shift
• Floor values on file as adequacy filter
• Against current balance? Increasing balance?
• Declining value? Physical deterioration?
• Methods?
• Owner occupied class
• Investment class
• Special cases
Our Friend IRV
Inc
Value
Rate
Capitalization Modeling
$9,000,000
$8,000,000
7.9% RO
7.0% RO
$7,000,000
$6,000,000
$5,000,000
$4,000,000
Annual Expenses
NOI
$3,000,000
$2,000,000
$1,000,000
$0
2011Q1
2012
2013
Appraisal Data
Prior Actual
Prior Actual
Forecast
IRV Example
Sales Trends
$70.00
$60.00
$50.00
Price $/sf
$40.00
$30.00
Orig Sales
$20.00
Orig Avg
Value Opinion @ Orig
Update Sales
$10.00
Update Avg
Suggested Current Value
$0.00
Date
Trend Tech Specs
APPRAISAL DATA IN FILE -- BUILDING SALES -- UNADJUSTED
Address
Sale Date
Sales Low
Sales High
Sales Median
Sales Avg
Sales COV
3/27/07
3/25/09
2/8/08
2/5/08
0.8%
Land Area
(sf)
Bldg Area
(sf)
Year Built
$/SF
$13,600,000 834,012
$21,600,000 2,178,000
$19,265,671 1,692,306
$18,722,866 1,494,233
16.2%
41.6%
330,190
500,000
403,960
408,151
16.6%
1981
2007
2007
1998.2
0.6%
$ 41.19
$ 53.50
$ 44.79
$ 46.02
10.3%
Sale Price
Com m ents
UPDATED BUILDING SALES
Address
Sales Low
Sales High
Sales Median
Sales Avg
Sales COV
Land Area
(sf)
Bldg Area
(sf)
Year Built
$/SF
12/1/09 $7,657,546 583,704
6/26/13 $23,531,680 3,242,171
10/25/11 $15,750,000 1,300,266
10/4/11 $15,672,307 1,606,602
1.6%
57.8%
76.0%
226,576
500,000
368,944
366,116
37.4%
1961
2009
1988.5
1986.75
1.2%
$ 28.12
$ 50.54
$ 41.15
$ 40.24
25.8%
Sale Date
Sale Price
Com m ents
Gas Station Validation
Gross
Profit
Multiplier
Value
Gas Station Performance
Multiplier Trending
Experiences
• How is Reg B working for you?
• How are evaluations working for you?