Benchmarking the Performance
of US Banks
R. Barr, SMU
T. Siems, Federal Reserve Bank of Dallas
S. Zimmel, SMU
Financial Industry Studies, Dec. 1998: www.dallasfed.org
1
Motivations and Goals

Motivations
 Safety
and soundness of banking system
 Protection of FDIC insurance fund
 Best allocation of examiner resources

Goals
 Prioritization
of on-site examinations
 Early-warning indicators of troubled banks
2
Objectives of the Research
Benchmark the U.S. banking system over
the last decade
 Assess performance with DEA-based model
 Isolate best- and worst-practice banks
 Support bank auditors by predicting trouble
 Evaluate DEA in large-scale benchmarking
role

3
Previous Work

Measuring bank management quality with
DEA
 Barr,

Seiford, Siems, 1993
Bank Failure Prediction Model
 DEA score
as input to logit forecasting model
 Barr and Siems, 1996

Technical report versions available at:
 www.smu.edu/~barr
4
Data Envelopment Analysis

A methodology for integrating and
analyzing benchmarking data that:
a multi-dimensional “gap analysis”
 Considers interactions, tradeoffs, substitutions
 Integrates all performance measures
 Gives an overall performance rating
 Suggests credible organizational goals,
benchmarking partners, ….
 Performs
5
Bank Performance Model
Inputs
Outputs
(Resources, Xs)
(Desired outcomes, Ys)
• Salary expense
•Earning assets
• Premises & fixed
assets
• Interest income
• Other noninterest
expense
• Noninterest
income
• Interest expense
• Purchased funds
6
Defining Efficiency

Efficiency = ratio of weighted sums of the
inputs and outputs (>0)
w Y
j
E
j
j
v X
i
i
i

Defines best practice in a DEA model
7
How DEA Works

Instead of using fixed weights for all units
under evaluation,
 DEA computes
a separate set of weights for
each bank
 Weights optimized to make that bank’s score
the best possible
 Constraints: no bank’s efficiency exceeds 1
when using the same weights
8
Formulating a DEA Model
There are many DEA models
 The basic idea in each is to choose a set of
weights for DMU k that:

Maximize Ek
such that : E p  1, for each bank p
all weights positive
9
Bank Output ($ Income)
Measuring Distance
60
Efficient frontier of best practice
50
40
z
f1
30
f
Inefficient bank
| f 1  z|
E
| f  z|
10
15
20
10
0
0
5
Bank Input ($Expenses)
10
Introducing Expert Judgment

Classic models may result in unreasonable
weight assignments for inputs & outputs
 
0 weights on unflattering dimensions
 Can overemphasize secondary factors

We added weight multipliers to the DEA
 Based
on survey of 12 FRB bank examiners
 Used response ranges to set UB/LBs on weights
11
Survey-Derived Constraints
Survey range
Survey average
Analytic Hierarchy
process weights
Inputs
Salary Expense
Premises/Fixed Assets
Other Noninterest Expense
Interest Expense
Purchased Funds
15.8% - 35.9%
3.1% - 15.7%
15.8% - 35.9%
17.2% - 42.8%
12.1% - 34.0%
23.10%
9.60%
22.70%
25.90%
18.80%
25.20%
11.40%
19.80%
23.50%
20.20%
Outputs
Earning Assets
Interest Income
Noninterest Income
40.9% - 69.5%
25.7% - 46.9%
10.2% - 20.2%
51.30%
34.30%
14.40%
52.50%
33.80%
13.70%
12
Banking Industry Test Data

End of year data for:
 1991
 1994
 1997
11,397 banks
10,224 banks
8,628 banks
Used constrained CCR-I model
 Run with large-scale specialized DEA
software

13
1991 Profiles by DEA E-Quartile
1991 data
1
DEA Efficiency Quartile
2
3
most efficient
least efficient
most to
least efficient
difference
4
INPUTS
Salary Expense / Total Assets
Premises and Fixed Assets / Total Assets
Other Noninterest Expense / Total Assets
Interest Expense / Total Assets
Purchased Funds / Total Assets
1.43%
1.00%
1.53%
4.71%
6.29%
1.54%
1.48%
1.62%
4.70%
8.17%
1.65%
1.76%
1.84%
4.66%
11.12%
1.83%
2.22%
2.41%
4.62%
16.07%
-0.40%
-1.22%
-0.87%
0.08%
-9.78%
*
*
*
*
*
92.68%
8.68%
0.95%
91.67%
8.71%
0.79%
90.59%
8.67%
0.89%
88.24%
8.55%
1.00%
4.44% *
0.13% *
-0.05%
OUTPUTS
Earning Assets / Total Assets
Interest Income / Total Assets
Noninterest Income / Total Assets
N
average efficiency score
lower boundary
upper boundary
2,850
0.7340
0.6334
1.0000
2,848
0.5982
0.5665
0.6334
2,849
0.5387
0.5092
0.5664
2,850
0.4611
0.0000
0.5091
0.2728
*
* Significant
at 0.01
(Values expressed as a percent of total bank assets)
14
1997 Profiles by DEA E-Quartile
1997 data
1
DEA Efficiency Quartile
2
3
most efficient
4
least efficient
most to
least efficient
difference
INPUTS
Salary Expense / Total Assets
Premises and Fixed Assets / Total Assets
Other Noninterest Expense / Total Assets
Interest Expense / Total Assets
Purchased Funds / Total Assets
1.67%
0.98%
1.85%
3.29%
10.46%
1.60%
1.55%
1.31%
3.30%
12.33%
1.64%
1.94%
1.50%
3.27%
13.63%
1.75%
2.44%
1.92%
3.15%
15.32%
-0.08%
-1.45% *
-0.07%
0.14% *
-4.85% *
92.99%
7.45%
1.80%
92.60%
7.41%
0.77%
91.83%
7.37%
0.84%
90.65%
7.33%
0.90%
2.33% *
0.13% ~
0.90% *
OUTPUTS
Earning Assets / Total Assets
Interest Income / Total Assets
Noninterest Income / Total Assets
N
average efficiency score
lower boundary
upper boundary
2,157
0.6685
0.4722
1.0000
2,157
0.4313
0.3982
0.4721
2,157
0.3717
0.3451
0.3981
2,157
0.3067
0.0000
0.3450
0.3617
*
15
Analysis of Results

1991 significant
differences, Q1-Q4:
 All
inputs, and most
outputs
 DEA scores


Noninterest income a
new focus for banks
 Fee
income
 Off-balance sheet
activities
Changed by 1997:
 Inputs:
Salary, other
non-interest (not sig.)
 Outputs: non-interest
income now signif.
16
Other Bank Performance Metrics
1
1991 data
DEA Efficiency Quartile
2
3
most efficient
Return on Average Assets
Equity / Total Assets
Total Loans / Total Assets
Non-performing Loans / Gross Loans
N
average efficiency score
lower boundary
upper boundary
1.23%
10.35%
48.95%
1.55%
2,850
0.7340
0.6334
1.0000
4
least efficient
1.00%
8.81%
53.34%
1.65%
2,848
0.5982
0.5665
0.6334
0.82%
8.25%
54.74%
1.96%
2,849
0.5387
0.5092
0.5664
0.01%
7.76%
56.56%
2.93%
2,850
0.4611
0.0000
0.5091
most to
least efficient
difference
1.22%
2.59%
-7.61%
-1.38%
*
*
*
*
0.2728
*
* Significant
at 0.01
17
Relationship with Other Metrics

Efficient banks:
 Greater
return on
assets
 Higher equity capital
 Fewer risky assets

1991 vs. 1997
 Not
comparable scores
 But underlying trends
of variables’
importance help
explain banking
industry changes
18
FRB Bank Examination Criteria
Capital adequacy
 Asset quality
 Management quality
 Earnings
 Liquidity

19
Bank Examiner Ratings



Confidential scores
from on-site visits
On each CAMEL
factor and overall
Values from 1 to 5
1 = sound in every
respect
2 = sound, modest
weaknesses
3 = weaknesses that give
cause for concern
4 = serious weaknesses
5 = critical weaknesses,
failure probable
20
CAMEL Ratings & DEA Scores


Compared CAMEL
ratings and DEA
efficiency scores
Included banks
examined recently:
1991: 7,487 banks
1994: 7,679 banks
1997: 4,494 banks

CAMEL rating groups
 Strong:
1 or 2 rating
 Weak: 3-5 rating

DEA-score groups
 Quintile,

by efficiency
If no relationship, each
group should contain
20% of each of the
other metric’s groups
21
Efficiency vs. CAMEL Ratings
Percentage of Banks in Efficiency Quintile
Efficiency Score Quintiles by CAMEL Rating
Combined Data 1991, 1994, 1997
100%
80%
60%
40%
20%
0%
1
2
3
4
5
CAMEL Rating
1st quintile (highest DEA scores)
2nd quintile
3rd quintile
4th quintile
5th quintile (lowest DEA scores)
22
“Strong” vs. “Weak” CAMELs
1991
Strong
Weak
Banks
Banks
1994
Strong
Weak
Banks
Banks
1997
Strong
Weak
Banks
Banks
INPUTS
Salary Expense / Total Assets
Premises and Fixed Assets / Total Assets
Other Noninterest Expense / Total Assets
Interest Expense / Total Assets
Purchased Funds / Total Assets
1.54%
1.53%
1.64%
4.64%
10.24%
1.83%
1.97%
2.41%
4.82%
11.39%
1.65%
1.65%
1.71%
2.61%
10.95%
2.23%
1.98%
2.92%
2.73%
10.79%
1.63%
1.72%
1.43%
3.25%
12.65%
2.04%
1.94%
2.26%
3.48%
14.83%
Earning Assets / Total Assets
Interest Income / Total Assets
Noninterest Income / Total Assets
91.65%
8.55%
0.83%
87.85%
8.88%
1.05%
91.92%
6.83%
0.93%
88.11%
7.32%
1.38%
92.18%
7.33%
0.85%
89.72%
7.88%
1.25%
DEA EFFICIENCY SCORE
0.5942
0.5235
0.6137
0.5532
0.4272
0.3751
Number of Institutions
5,641
1,846
7,188
491
4,273
221
OUTPUTS
23
In Summary
DEA useful in benchmarking in service
industry
 Can provide information for examiners, but
not perfect predictor
 Large-scale efficiency analyses can give
insight into industry dynamics and structure
changes

24