META-ANALYSIS
Types of Meta-Analysis
• Qualitative
• Quantitative
2
Quantitative Meta-Analysis
Data Sources
• Primary – the ideal data source
– Raw data
• Secondary:
– The data and conclusions presented in study
reports / papers
• Tertiary:
– Distillations of the data and conclusions
presented by the secondary sources
3
Major Flaw in Existing
Meta-Analyses
• Depend on tertiary data (third-hand
information)
• Use extremely broad explanatory variable
definitions
• Generally have very low explanatory power
• Coefficients frequently have confidence
levels of less than 50%
4
Literature Example
Lipscomb, Mooney and Kilpatrick
“Do Survey Results Systematically Differ from
Hedonic Regression Results? Evidence From a
Residential Property Analysis”
- Forthcoming, Journal of Real Estate Literature
5
Issues
No comparison of results to
actual market results
6
Issues
Only peer-reviewed papers used
without a test for bias
resulting from
restricted data selection
7
From the Paper
Comparison of Descriptive Statistics and Meta-Analysis Results
Factor
8
Descriptive Statistics
Meta-Analysis
Water
-6.1%
-1.27%
Superfund
-11.4%
-1.78%
Air
-2.5%
-1.81%
Linear
-10.4%
-3.02%
Urban
-5.7%
-1.57%
From the Paper
Explanatory Coefficients Significant at 95% Confidence Level or Above
Variable
Constant
Coefficient
Standard Error
-4.893768000
2.376
-2.060
LITIG
0.306087800
0.133
2.290
URBANDIS
0.177918500
0.060
2.980
SUBURBAN
0.089769600
0.036
2.470
UNEMP
0.032650900
0.010
3.220
INFO
0.016814200
0.009
1.980
YEAROFDATA
0.002410900
0.001
2.010
N = 273; Adjusted R-Squared = 0.2453
9
T-statistic
From the Paper
Explanatory Coefficients Not Significantly at 95% Confidence Level
Variable
Coefficient
Standard Error
T-statistic
STUDY_SURV
0.104591900
0.055
1.910
LANDFILL
-0.188210500
0.101
-1.870
BELOW THIS POINT THE CONFIDENCE LEVEL IS LESS THAN 85%
INTERNATIONAL
-0.064073700
0.045
-1.410
REALVAL
-0.000000268
0.000
-1.210
LOTSOFOBS
-0.021452100
0.019
-1.120
NATIONAL
-0.079998900
0.096
-0.830
LINEAR
-0.030182400
0.039
-0.770
BELOW THIS POINT THE CONFIDENCE LEVEL IS THAN 50%
URBAN
-0.015673300
0.027
-0.590
SUPERSITE
0.017774100
0.032
0.550
LNDIST
-0.001526800
0.003
-0.530
MIXED
0.016710200
0.033
0.510
WATER
-0.012708100
0.025
-0.510
AIRCAFO
-0.018120600
0.036
-0.500
REGIONAL
0.031982300
0.065
0.490
POSITIVE
-0.006227500
0.021
-0.300
NUKEMANUF
0.019563400
0.071
0.280
LNSAMPLE
-0.001300300
0.005
-0.270
61% of coefficients less than 85% confidence
43% of coefficients less than 50% confidence
10
Base Studies
A number of the base studies used in
the meta-analyses to date can be
shown to be flawed
11
Summary
Quantitative Meta-analysis:
Hedonic analysis of a regression model
BUT
Meta-analysis has significantly greater issues
• Third-hand data
• Low explanatory
power
• Low statistical
significance of
coefficients
12
• Overly-broad variable
definitions
• Virtually complete
lack of confirmation
META-ANALYSIS
IS NOT AND SHOULD NOT BE
AN ACCEPTED
APPRAISAL METHOD
13
CONTINGENT VALUATION
• CV
• CVM
• POTENTIAL BUYER SURVEYS
14
DEFINITION
• “According to the World Bank Institute (2002,
p. 2), the CVM is a stated preference method
used to estimate ‘‘the value that a person
places on a good, usually one that is not sold
in markets, such as environmental quality or
good health.”
Lipscomb, Clifford A., Max Kummerow, Will Spiess, Sarah Kilpatrick, John A. Kilpatrick,
“Contingent Valuation and Real Estate Damage Estimation,” Journal of Real Estate Literature,
Volume 19, Number 2, 2011.
15
REASONS FOR DOING CVM
• real estate is a market good
• Argument for CVM based on the idea that
those who purchased are ill-informed,
imprudent, or plain foolish, and the prices
paid do not fully reflect the impact of the
issue
• a CVM survey must be performed to
determine how the market should have
responded.
16
Dr. Kinnard’s “dictum” has been stated as:
“The best information may be found in
transactions that did not occur.”
The correct statement is:
Possibly the best information might be found in
failed sales of impaired property.
17
COURT RULINGS
• The courts do not agree
• Melody Baker et al v Motorola, Inc, et al, Superior Court of the
State of Arizona in and for the County of Maricopa, CV 9202603
• Circuit Court for Baltimore County Case # 03-C-07-003809 IN
THE COURT OF APPEALS OF MARYLAND No. 15 September
Term, 2012 EXXON MOBIL CORPORATION v. THOMAS M.
ALBRIGHT, et al.
18
WHAT IS BEING MEASURED
• COMPENSATING SURPLUS:
• “That amount that would be required to make
the property owner indifferent between
remaining in the property with the issue, and
moving to a property without the issue.”
•
19
Mundy, Bill and David McLean, “The Addition of Contingent Valuation and Conjoint Analysis
to the Required Body of Knowledge for the Estimation of Environmental Damages to Real
Property,” Journal of Real Estate Practice and Education, Volume 1, Number 1, 1998.
{Referencing “The straightforward discussion of compensating surplus can be found in
Mitchell and Carson (1990), p. 24.”}
WRONG RESPONDENTS
• All of the surveys to date use respondents
chosen at random and not qualified with
respect to specialized knowledge of the issue.
• Surveys are normally less than ten minutes.
• Respondents do not have time to fully
consider the issues.
• Respondents do not have time to consult
other players
20
LACK OF CONFIRMATION
• CVM proponents have not tested the accuracy
and reliability of their predictions.
• The research by Roddewig and Frey noted
error rates of 200% when compared to actual
market performance
21
TRIMMED MEAN
• “This method {marginal bidder analysis}
eliminates extreme bids and gives more
weight to those bids closer to the middle of
the bid distribution. This is a form of trimmed
mean analysis, which eliminates outlier
observations.”
• Lipscomb, Clifford A., Max Kummerow, Will Spiess, Sarah Kilpatrick, John
A. Kilpatrick, ibid.
22
TRIMMED MEAN
• The buyers and sellers have no interest in
taking the “middle of the bid distribution” as a
sale price. The seller wishes to maximize his
price and only the top few bidders would be
considered, all others bids being put in the
circular file where the real market participants
actually put them.
23
NO BASELINE
• Simons CVM studies of leaking gas stations
provide an indicated discount.
• HOWEVER, no base line to let us determine
the discount for a gas station without a leak.
• This is a common CVM problem.
24
NOT AN APPROVED METHOD
• Not in USPAP or Advisory Opinions or Guide
notes
• Not endorsed in AI training materials
• Plenty of market data
• Intended only for non-market data and then
for amounts generally less than $50
25