Mutual funds:
performance
evaluation
Worldwide TNA of
mutual funds
РЭШ
EFM 2006/7
2
Worldwide #
mutual funds
РЭШ
EFM 2006/7
3
Open-end mutual funds


Active vs passive (index) funds
Obliged to buy/sell shares at NAV
– Net Asset Value = Total Net Assets (TNA) per share


Part of the fund family (run by one management
company)
Management fee:
– Asset-based: proportional to TNA
– Performance-based: must be symmetric around the
benchmark
РЭШ
EFM 2006/7
4
MF categories
(by Morningstar)

Broad asset class:
– Domestic: equity vs bond vs money market vs hybrid
– International: foreign, world (global), Europe, Pacific, etc.

(Stated) investment objective
– Equity: aggressive growth, growth, growth&income, equityincome, income
– Bond: government, municipal, corporate
– Hybrid: balanced, asset allocation

(Estimated) investment style: 3x3 matrix
– Equity: large/mid/small-cap – value/blend/growth
– Bonds: high/medium/low credit quality –
short/intermediate/long duration
РЭШ
EFM 2006/7
5
TNA of US mutual funds
РЭШ
EFM 2006/7
6
# US mutual funds
РЭШ
EFM 2006/7
7
Benefits of investing via
MF

Low transaction costs
– Easy way to buy a diversified portfolio

Customer services
– Liquidity insurance
– Easy transfer across funds within the family

Professional management
– Selecting right stocks at right time?

The objective of the research:
– Check the validity of these claims
РЭШ
EFM 2006/7
8
Research questions







Why has it become one of the largest financial
intermediaries?
Why are there more mutual funds than stocks?
How to measure fund performance adjusted for
risk?
Does fund performance persist?
How do investors choose between funds?
Which incentives does it give to fund managers?
How accurately do categories divide funds?
EFM 2006/7
РЭШ
9
How to measure MF
performance?

Raw return, determined by
– Risk factors
– Factor exposures

Timing ability: changing beta at right time
– Selection (stock-picking) ability

Choosing right stocks (for same level of risk)
РЭШ
EFM 2006/7
10
How to measure MF
performance?

Risk-adjusted return:
– Difference between fund i’s return and benchmark
return
– Benchmark: passive portfolio with same risk as fund i

How to find a right benchmark?
– Return-based approach: estimate based on past returns
– Portfolio-based approach: construct a portfolio of
assets similar to those held by the fund
– Relative approach: compare to performance of other
funds
РЭШ
EFM 2006/7
11
Factor models

Regression of excess asset returns on factor
returns
Ri,t–RF,t = αi + Σkβi,kFk,t + εt,
–
–
–
–

Market model: RMRF
Fama-French: RMRF, SMB, HML
Carhart: RMRF, SMB, HML, MOM (1y momentum)
Elton-Gruber: RMRF, SMB, HML, excess bond index
return
Jensen’s alpha:
– Shows whether fund i outperforms passive portfolio of
K factors and RF
EFM 2006/7
РЭШ
12
Mean-variance spanning
tests

Test whether adding K new assets (MFs) to N old assets
leads to the shift of the MV frontier:
– Three cases possible: spanning, intersection, shift

Regression of new asset returns r (Kx1) on old asset
returns R (Nx1):
rt = α + BRt + εt
– Generalized Jensen’s alpha


Test for intersection: there exists η s.t. α-η(lN-BlK)=0
Test for spanning: α=0 and BlK=lN
– All additional assets can be written as portfolio of old assets
РЭШ
EFM 2006/7
13
Other absolute ordinal
measures



Sharpe ratio: (E(Ri)-RF)/σi
Treynor ratio: (E(Ri)-RF)/βi
Appraisal ratio: αi/σ(ε)i
– Called Treynor-Black ratio when alpha based on
market model
РЭШ
EFM 2006/7
14
Relative performance
measures




Use funds in the same category as a benchmark
Ordinal measures: difference with the mean or
median return in the fund’s category
Cardinal measures: category ranking based on
return/α/…
Drawbacks:
– There may be substantial differences in risk within the
category
– Survivor bias
– Bad incentives to managers (as in a tournament)
РЭШ
EFM 2006/7
15
How to measure
performance persistence?

Contingency tables:
– Sort funds by past and current performance

E.g., 2x2 (above/below median): winner-winner, WL, LW, LL
– Check whether actual frequencies are far from those under the null

Examine zero-investment portfolios formed on the basis
of past performance
– Sort funds into deciles by last-year return
– Test whether top-bottom portfolio has premium unexplained by
factor models

Cross-sectional regressions of current performance on
past performance
РЭШ
EFM 2006/7
16
Need to control for

Fund attrition
– Survivor bias

Cross-correlation in fund returns
– Fewer degrees of freedom will make s.e. larger

The measurement error (and mean
reversion)
– If measure both current and past performance
in the same way
РЭШ
EFM 2006/7
17
Brown and Goetzmann
(1995)
"Mutual fund performance persistence"
 Explore MF performance persistence
– Absolute vs relative benchmarks
– Explicitly model survivor bias
– Disaggregate on the annual basis
РЭШ
EFM 2006/7
18
Data

Common stock funds in 1976-1988
– Including dead funds
– Monthly return data

Table 1
– # funds: 372 in 1976, 829 in 1988
– Total assets rose more than 4 times
– MaxCap category became relatively less popular
РЭШ
EFM 2006/7
19
Average performance

Table 2
– VW mean MF return is below S&P500 return
by 0.4% p.a., though above index fund
– Dead funds heavily underperform living funds
– EW means exceed VW means
РЭШ
EFM 2006/7
20
Fund disappearance


Disappearance: termination or merging into
another fund
Table 3, determinants of prob(death)
– Lagged relative return: – Lagged relative new money: 
But insignificant in presence of past performance
– Relative size: – Expense ratio: +
– Age: РЭШ
EFM 2006/7
21
Performance persistence

Contingency tables:
– Sort funds by performance over the last year
and the current year
– Winner/loser = above/below median, 2x2
matrix
– Cross-product ratio: (WW*LL)/(WL*LW)=1
under the null
РЭШ
EFM 2006/7
22
Bootstrapping procedure

Necessary to control for fund attrition and
cross-correlation:
– Use de-meaned sample of fund monthly
returns in 1987-88
– For each year, select N funds without
replacement and randomize over time
– Assume that poorest performers after the first
year are eliminated
– Repeat 100 times
РЭШ
EFM 2006/7
23
Results

Table 4, odds ratio test for raw returns
relative to median
– 7 years: significant positive persistence
– 2 years: significant negative persistence
РЭШ
EFM 2006/7
24
Controlling for differences in
systematic risk

Use several risk-adjusted performance
measures:
– Jensen’s alpha from the market model
– One-index / three-index appraisal ratio
– Style-adjusted return

Table 6, odds ratio test for risk-adjusted
returns relative to median
– Similar results: 5-7 years +, 2 years - persistence
РЭШ
EFM 2006/7
25
Absolute benchmarks

Figure 1, frequencies of repeat losers
and winners wrt S&P500
– Repeat-losers dominate in the second
half of the sample period

Table 6, odds ratio test for alpha
relative to 0
– 5 years +, 2 years - persistence
РЭШ
EFM 2006/7
26
Investment implications

Table 7, performance of last-year return
octile portfolios
– Past winners perform better than past losers

Winner-loser portfolio generates significant
performance
– Idiosyncratic risk is the highest for past winners

Winner-loser portfolio return is mostly due to bad
performance of persistent losers
РЭШ
EFM 2006/7
27
Conclusions




Past performance is the strongest predictor of
fund attrition
Clear evidence of relative performance persistence
Performance persistence is strongly dependent on
the time period
Need to find common mgt strategies explaining
persistence and reversals
– Additional risk factor(s)
– Conditional approach
РЭШ
EFM 2006/7
28
Conclusions (cont.)


Chasing the winners is a risky strategy
Selling the losers makes sense
– Why don’t all shareholders of poorly
performing funds leave?

Disadvantaged clientele
– Arbitrageurs can’t short-sell losing MFs!
РЭШ
EFM 2006/7
29
Carhart (1997)
"On persistence in mutual fund performance"
 Survivor-bias free sample
 Examine portfolios ranked by lagged 1-year return
– The four-factor model: RMRF, SMB, HML, and 1-year
momentum…
– Explains most of the return unexplained by CAPM…
– Except for underperformance of the worst funds

Fama-MacBeth cross-sectional regressions of
alphas on current fund characteristics:
– Expense ratio, turnover, and load: negative effect
РЭШ
EFM 2006/7
30
Conditional
performance
evaluation
Plan for today

Up to now:
– Average performance

Jensen’s alpha: selection ability
– Differential performance


Performance persistence
Today:
– Conditional approach to performance evaluation


Timing ability
Use dynamic strategies based on public info as a benchmark
РЭШ
EFM 2006/7
32
Problems with the
unconditional approach

The market model (with excess returns):
ri,t = αi + βirM,t + εi,t
– What if β is correlated with the market return?
– If cov(β, rM)>0, the estimated α is downwardbiased!

How to measure timing ability?
РЭШ
EFM 2006/7
33
Market timing tests

Assume that βt = β0 + γf(RM-RF)
– Treynor-Mazuy: linear function, f(·)=RM-RF
– Merton-Henriksson: step function, f(·)=I{RM-RF>0}
– γ shows whether fund managers can time the market

Typical results for an average fund
– Negative alpha: no selection ability
– Negative gamma: no timing ability
РЭШ
EFM 2006/7
34
Problems with measuring
market timing

Benchmark assets may have option-like
characteristics
– Gamma is positive/negative for some stocks

Managers may have timing ability at higher
horizon
– Tests using monthly data have low power of identifying
market timing on a daily basis

Positive covariance between beta and market
return could result from using public info
EFM 2006/7
РЭШ
35
Ferson and Schadt (1996)
"Measuring Fund Strategy and
Performance in Changing Economic
Conditions"
 Evaluate MF performance using conditional
approach
– Both selection and timing ability
– Use dynamic strategies based on public info as
a benchmark

Consistent with SSFE
EFM 2006/7
РЭШ
36
Methodology

Conditional market model:
ri,t+1 = αi + βi,trM,t+1 + εi,t+1,
– where βi,t = β0i + β’1iZt (+ γif(rM,t+1))
– Zt are instruments


Estimation by OLS:
ri,t+1 = αi + (β0i+β’1iZt+γif(rM,t+1)) rM,t+1+εi,t+1
Extension: a four-factor model
– Large-cap (S&P-500) and small-cap stock returns,
government and corporate bond yields
РЭШ
EFM 2006/7
37
Data


Monthly returns of 67 (mostly equity) funds
in 1968-1990
Instruments (lagged, mean-adjusted):
–
–
–
–
–
30-day T-bill rate
Dividend yield
Term spread
Default spread
January dummy
РЭШ
EFM 2006/7
38
Results

Table 2, conditional vs unconditional
CAPM
– Market betas are related to conditional
information
 30-day
T-bill rate, dividend yield, and term
spread are significant
– Conditional alphas are higher than the
unconditional ones
РЭШ
EFM 2006/7
39
Results (cont.)

Table 3, cross-sectional distribution of tstats for cond. and uncond. alphas
– Unconditional approach: there are more
significantly negative alphas
– Conditional approach: # significantly negative /
positive alphas is similar
– Very similar results for one-factor and fourfactor models
РЭШ
EFM 2006/7
40
Results (cont.)

Table 4, conditional vs unconditional market
timing model for naïve strategies
– Naïve strategies:


Start with 65% large-cap, 13% small-cap, 20% gvt bonds, 2%
corporate bonds weights
Then: buy-and-hold / annual rebalancing / fixed weights
– Unconditional approach: positive alpha and negative
gamma for buy-and-hold strategy

Evidence of model misspecification
– Conditional approach: insignificant alpha and gamma
РЭШ
EFM 2006/7
41
Results (cont.)

Tables 5-6, conditional vs unconditional market
timing models for actual data
– Conditional approach: the significance of alpha and
gamma disappears for all categories but special
(concentrating on intl investments)

Table 7, cross-sectional distribution of t-stats for
cond. and uncond. gammas
– Fewer (significantly) negative gammas under the
conditional approach
– More (significantly) positive gammas under the
conditional approach, esp. for TM model
РЭШ
EFM 2006/7
42
Interpretation of the
results


Dynamic strategies based on instruments
contribute negatively to fund returns
Is it the active policy or mechanical effects?
– The underlying assets may have gammas different from
zero

Yet, we do not observe similar (α,β,γ) patters for the buy-andhold portfolio
– New money flows to funds increase their cash holdings
and lower betas

Edelen (1999): liquidity-motivated trading lowers both alpha
and gamma
РЭШ
EFM 2006/7
43
Conclusions

Conditioning on public information:
– Provides additional insights about fund
strategies
– Allows to estimate classical performance
measures more precisely

The average MF performance is no longer
inferior
– Both selection and timing ability
РЭШ
EFM 2006/7
44
Bollen and Busse (2001)
"On the timing ability of mutual fund managers"

Using daily returns in market timing tests
– Much higher power if managers time the market on a
daily basis

Traditional tests:
– 40% of funds have γ>0, 28% have γ<0


Cf: 33% +, 5% - based on monthly data
Compare fund γ’s with those for synthetic
portfolios (γB):
– 1/3 of funds have γ>γB, 1/3 have γ<γB
EFM 2006/7
РЭШ
45
Strategic behavior
Plan for today

Up to now:
– Average performance
Selection vs timing ability
 Unconditional vs conditional

– Differential performance


Performance persistence
Today:
– Strategic behavior of fund managers

Choice of risk in the annual tournaments
EFM 2006/7
РЭШ
47
The objective function of
MF manager

Career concerns
– High (low) performance leads to promotion (dismissal)
– High risk increases the probability of dismissal

Compensation
– Usually proportional to the fund’s size (and flows)
– Convex relation between flows and performance gives strong
incentives to win the MF tournament

Calendar-year performance is esp important
– Managers are usually evaluated at the end of the year
– Investors pay more attention to calendar year performance
РЭШ
EFM 2006/7
48
Chevalier and Ellison
(1997)
"Risk Taking by Mutual Funds as a Response to
Incentives"
 Estimate the shape of the flow-performance
relationship
– Separately for young and old funds


Estimate resulting risk-taking incentives
Examine the actual change in riskiness of funds’
portfolios
– On the basis of portfolio holdings in September and
December
РЭШ
EFM 2006/7
49
Data

449 growth and growth&income funds in 1982-92
– Monthly returns
– Annual TNA
– Portfolio holdings in September and December


About 92% of the portfolio matched to CRSP data
Excluding index, closed, primarily institutional,
merged in the current year, high expense ratio
(>4%), smallest (TNA<$10 mln) and youngest
(age < 2y) funds
РЭШ
EFM 2006/7
50
The flow-performance
relationship

Flowt = ΔTNAt/TNAt-1 – Rt
– Net relative growth in fund’s assets
Semi-parametric regression of annual flows on last-year
market-adjusted returns:
Flowi,t+1=ΣkγkAgeDkf(Ri,t-RM,t)+ΣkδkAgeDk+α1(Ri,t-1-RM,t-1)
+α2(Ri,t-2-RM,t-2)+α4IndFlowi,t+1+α5ln(TNA)i,t+εi,t+1

– f(Ri,t-RM,t) is a non-parametric function estimated separately for
young (2-5y) and old funds
– AgeDk are dummy variables for various age categories
– Fund’s size and growth in total TNA of equity funds are controls
РЭШ
EFM 2006/7
51
Results

Figures 1-2, Table 2: flow-performance
relationship for young and old funds
– Generally convex shape

Linearity is rejected, esp for old funds
– The sensitivity of flows to performance is
higher for young funds
– Flows rise with lagged performance up to 3
years, current category flows and fall with size
РЭШ
EFM 2006/7
52
Estimation of risk-taking
incentives

Assume:
– Fees are proportional to the fund’s assets
– Flows occur at the end of the year
– No agency problems between MF companies and their managers

In September of year t+1, the increase in expected end-ofyear flow due to a change in nonsystematic risk in the lastquarter return:
hk(rsep, σ, Δσ)=E[γk(f(Rsep+u)-f(Rsep+v))]
– After increasing nonsystematic risk by Δσ, the last-quarter return
distribution changes from u to v
– Take Δσ=0.5σ
РЭШ
EFM 2006/7
53
Results

Figure 3, risk incentives for 2y and 11y
funds
– Young funds with high (low) interim
performance have an incentive to decrease
(increase) risk to lock up the winning position
(catch up with top funds)

The risk incentives are reversed at the extreme
performance
– Insignificant pattern for old funds
РЭШ
EFM 2006/7
54
Actual risk-taking in response
to estimated risk incentives


Cross-sectional regressions of within-year
change in risk on risk incentive measure
Focus on the equity portion of funds’
portfolios (on average, about 90%
– Risk measures computed based on prior-year
daily stock data
РЭШ
EFM 2006/7
55
Actual risk-taking in response
to estimated risk incentives

Dependent variable: change between September
and December in
– St deviation of the market-adjusted return: ΔSD(Ri-RM)
– Unsystematic risk: ΔSD(Ri-βiRM)
– Systematic risk: Δ|βi-1|

Independent variables:
–
–
–
–
RiskIncentive: hk
Size: ln(TNA)
RiskIncentive*ln(TNA)
September risk level: to control for mean reversion
РЭШ
EFM 2006/7
56
Results

Table 4
– The higher risk incentives, the higher actual
change in total and unsystematic risk
– This effect becomes less important for larger
funds
– No evidence of mean reversion
РЭШ
EFM 2006/7
57
Actual risk-taking in response
to interim performance


Dependent variable: change between September
and December in total risk
Main independent variable:
– January-September market-adjusted return: Ri,sep-RM,sep

Assume that change in risk is a piecewise linear
function of interim performance
– 2 fitted kink points

Estimate separately for young and old funds
РЭШ
EFM 2006/7
58
Results

Table 5, Figure 4
– Generally negative relation between actual change in
total risk and interim performance
– Most slopes and kink points are not significant

Alternative approach to measure total risk:
– Using monthly returns: σ(Oct-Dec)-σ(Jan-Sep)


Very noisy, esp for last quarter (only 3 points!)
Table 6, Figure 5
– Generally positive (!) relation between actual change in
total risk and interim performance
РЭШ
EFM 2006/7
59
Conclusions




The flow-performance relationship is convex
This generates strategic risk-taking incentives
during the year
Mutual funds seem to respond to these incentives
The change in fund’s risk (measured via portfolio)
is negatively related to its interim performance
– Though contradictory evidence based on return-based
approach
РЭШ
EFM 2006/7
60
Brown, Harlow, and
Starks (1996)
"Of tournaments and temptations: An analysis
of managerial incentives in the MF industry"
 Contingency table approach:
– Sort funds by mid-year return and within-year change in
total risk

Risk-adjustment ratio based on monthly returns: σ(7:12)/σ(1:6)
– 2x2 matrix: return/RAR above/below median
– Each cell should have 25% of funds under the null

Find 27% frequency of high-return low-RAR
funds in 1980-1991
– Support the tournament hypothesis
РЭШ
EFM 2006/7
61
Busse (2001)
"Another look at mutual fund tournaments"
 Same contingency table approach using daily and
monthly data
– Disaggregate: annual tournaments

Control for cross-correlation and auto-correlation
in fund returns
– Compute p-values from bootstrap

No significant evidence for the tournament
hypothesis!
РЭШ
EFM 2006/7
62
Wermers (2000)
"MF performance: An empirical
decomposition into stock-picking talent,
style, transactions costs, and expenses"


Decompose fund’s return into several components
to analyze the value of active fund management
Portfolio-based approach:
– Using portfolio holdings data
РЭШ
EFM 2006/7
63
Methodology

Finding the benchmark: one of 125 portfolios
– In June of each year t, rank stocks by size (current ME)
and form 5 quintile portfolios
– Subdivide each of 5 size portfolios into 5 portfolios
based on BE/ME as of December of t-1
– Subdivide each of 25 size-BM portfolios into 5
portfolios based on past 12m return
– From July of t to June of t+1, compute monthly VW
returns of 125 portfolios
РЭШ
EFM 2006/7
64
Methodology (cont.)

Decomposing fund’s return: R = CS + CT + AS
– Characteristic selectivity: CS=Σjwj,t-1[Rj,t-Rt(bj,t-1)]



wj,t-1 is last-quarter weight of stock j in the fund’s portfolio
Rt(bj,t-1) is current return on the benchmark ptf matched to
stock j in quarter t-1
CS measures the fund’s return adjusted for 3 characteristics
– Characteristic timing: CT=Σj[wj,t-1Rt(bj,t-1)-wj,t-5Rt(bj,t-5)]

CT is higher if the fund increases the factor’s exposure when
its premium rises
– Average style: AS=Σjwj,t-5Rt(bj,t-5)

AS measures tendency to hold stocks with certain
characteristics
РЭШ
EFM 2006/7
65
Methodology (cont.)

Comparing with return-based approach:
– Potentially higher power: no need to estimate
factor loadings
– But: may be biased due to window-dressing
– But: only equity portion of fund’s portfolio
РЭШ
EFM 2006/7
66
Data

1788 diversified equity US funds in 1975-94
– CRSP: monthly returns, annual turnover,
expense ratios, and TNA
– CDA: quarterly portfolio holdings (only equity
portion)
– No survivor bias

CRSP files of US stocks
РЭШ
EFM 2006/7
67
Results

Table 5, decomposition of (equity portion of) MF
returns
–
–
–
–
–
–
–
–
Gross return: 15.8% p.a. > 14.3% VW-CRSP index
CS = 0.75%, significant
CT = 0.02%, insignificant
AS = 14.8%
Expense ratio = 0.79%, up from 65 to 93 b.p.
Transactions costs = 0.8%, down from 140 to 48 b.p.
Non-equity portion of the fund’s portfolio: 0.4%
Net return: 13.8% < 14.3% VW-CRSP index!
EFM 2006/7
РЭШ
68
Mutual funds: summary






Many funds hardly follow their stated objectives
On average, MFs do not earn positive
performance adjusted for risk and expenses
Bad performance persists
Money flows are concentrated among funds with
best performance
Poorly performing funds are not punished with
large outflows
Funds try to win annual tournaments by adjusting
risk
РЭШ
EFM 2006/7
69