Effectiveness of the Visual Analog Scale for
the Measurement of Business Expectations
Anna Stangl
Ifo Institute for Economic Research
Brussels, 12. October 2009
Visual Analog Scale (VAS) in Web Surveys
3-Category Scale.
Visual Analog Scale (VAS)
1. Present business situation
2. Business expectations
Ifo Economic Climate Index
2 / 21
Motivation: Measurement of Expectations
Qualitative measurement of economic expectations: category-rating scales
• limited, coarse data
• information loss in the neutral category
• no information on dispersion
• strong assumptions for modeling
• easy to apply
• reliable measurement
Quantitative measurement of economic expectations: point forecasts
• interval-scale measurement
• information on dispersion and the
shape of the distribution
• costly and time-demanding
• prone to inaccuracies
• not available in business surveys
Probabilistic expectations
• theoretically appealing
• information on dispersion and the
shape of the distribution
• information on uncertainty
• applicable in surveys of professionals
• general tendency of respondents to
be optimistic / further drawbacks
• not available in business surveys
3 / 21
Categorical Scale
This country's overall economic situation at present?
vs.
Visual Analog Scale
This country's overall economic situation at present?
Economic confidence
I.
Economic confidence
I.
a
b
bad
satisfactory
good
II.
bad
satisfactory
good
Score range: 1 to 100
Score range: 1 to 3
- Symmetric properties of the
indifference interval are assumed
- Information loss due to the central
tendency of responses
- Enables scores between categories
- Shows direction of change and
magnitude
- Delivers dispersion and uncertainty
measures
4 / 21
Categorical Scale
vs.
Visual Analog Scale
3-Category Scale
bad
Visual Analog Scale
bad
satisfactory good


satisfactory
good

Economic Expectations
25
80
Economic Expectations
67.62
10
%
40
% 15
60
20
Uncertainty
20
5
21.96
0
10.42
0
-50
Present Economic Situation
0
Visual Analog Scale
50
15
60
Present Economic Situation
%
%
40
10
55.21
20
5
28.38
0
16.4
0
-50
0
Visual Analog Scale
50
5 / 21
Data
Internet Business survey in manufacturing, Germany
Number of responses
Month
1
2
3
4
5
6
7
8
9
10
11
12
Total
2005
1,175
1,212
1,217
1,198
2006
1,393
1,384
1,447
1,430
1,434
1,345
1,374
1,296
1,369
1,359
1,386
1,422
2007
1,443
1,388
1,460
1,438
1,509
1,447
1,396
1,334
1,394
1,429
1,414
1,461
2008
1,476
1,519
1,498
1,521
1,512
1,516
1,562
1,543
1,542
1,571
1,541
1,541
2009
1,560
1,553
1,563
1,602
1,619
1,619
1,632
1,558
1,580
Total
5,872
5,844
5,968
5,991
6,074
5,927
5,964
5,731
7,060
5,571
5,558
5,622
4,802
16,639
17,113
18,342
14,286
71,182
6 / 21
Observation period
7 / 21
Scale Reliability Tests
Scale reliability:

VAS

3-Cat.
Parallel-form reliability
bad
satisfactory
good
bad
satisfactory
good
1.

Test-retest reliability

Internal consistency

Inter-rater reliability

2.
1. Step: Common factor of „econ.
situation“ items extracted

2. Step: Correlation of the common
factor with VAS / 3-Cat.

8 / 21
3-Category and VAS Business Expectations
and Production Index
3-Category Business Climate
VAS Business Climate
9 / 21
Measures of Uncertainty and Heterogeneity
Direct survey-based measures: probability distribution of a point forecast
• uncertainty of the individual forecaster
• responses are comparable
• internal consistency and accuracy
• surveys of professionals
• not available in business & consumer
surveys
Time-series based measure of uncertainty: forecast errors
• proxy for uncertainty
• readily available
• accuracy
• not available in business and consumer
surveys
Dispersion as a proxy for uncertainty (Zarnowith and Labros, 1987)
• readily available
• proxy for uncertainty
• important in its own right as measure
of heterogeneity
• quantitative point-forecasts
• not available in business and consumer
surveys
10 / 21
Measuring Uncertainty and Heterogeneity with VAS
20
0
5
Bruine de Bruin et al., 2000
% 15
“Epistemic uncertainty”
10
1.
25
Economic Expectations
-50
3.
25
20
% 15
10
Zarnowith and Labros, 1987, Bomberger, 1996, Batchelor and Dua, 1996,
Giordani and Soderlind, 2003, Rich and Tracy, 2006, Mitchel et al., 2005,
Lahiri and Liu, 2006, Doepke and Fritsche, 2006, Boero et al., 2007…
Economic Expectations
5
Dispersion of business expectations (Heterogeneity)
50
0
2.
0
Visual Analog Scale
-50
0
Visual Analog Scale
50
Kurtosis
Doepke and Fritsche, 2006: A significant kurtosis above 3 indicates
that the forecasters are very close to each other.
4.
Skewness
Doepke and Fritsche, 2006: If the distribution of economic expectations is
significantly skewed, consensus among forecasters is rejected
11 / 21
Dispersion of VAS Business Expectations
and the Production Index
Growth fall /
turning point
Correlation: -0.84
12 / 21
Measuring Uncertainty with VAS
0
5
10
% 15
20
25
Economic Expectations
-50
0
Visual Analog Scale
50
20
25
Economic Expectations
0
5
10
% 15
Correlation: 0.80
-50
0
Visual Analog Scale
50
20
25
Economic Expectations
0
5
10
% 15
Correlation: 0.72
-50
0
Visual Analog Scale
50
13 / 21
Skewness of the VAS Present Business Situation
Skewness
14 / 21
Skewness of the VAS Business Expectations
Skewness
15 / 21
Skewness of the VAS Business Expectations and
Production Index
16 / 21
Correlation with Production Index in Manufacturing
Correlation of the VAS Indicators
with the Production Index in the German Manufacturing Sector
Lead in months
0
1
2
3
VAS Climate
0.94
0.95
0.93
0.89
3-Cat. Climate
0.92
0.95
0.95
0.93
VAS Expectations
0.90
0.94
0.95
0.94
3-Cat. Expectations
0.74
0.84
0.90
0.93
VAS Situation
0.94
0.92
0.88
0.83
3-Cat. Situation
0.93
0.91
0.87
0.81
Skewness of VAS situation
0.93
0.93
0.91
0.88
Skewness of VAS expectations
0.65
0.72
0.77
0.82
Epistemic uncertainty
0.56
0.62
0.66
0.61
Standard deviation of VAS expectations
0.84
0.82
0.78
0.76
Kurtosis of VAS expectations
0.90
0.89
0.88
0.86





17 / 21
Summary: Effectiveness of the VAS
25
0
5
10
% 15
20

-50
% 15
20
25
Economic Expectations
-50
3.
Skewness of the VAS distribution
- Cyclical
- More pronounced in VAS business situation
0
Visual Analog Scale
50

10
“Epistemic uncertainty” (Bruine et al. definition)
- Increases around turning points
- Proxy for uncertainty
Economic Expectations
5
2.
Dispersion and Kurtosis of business expectations
- Contra-cyclical
- Proxy for uncertainty and heterogeneity
0
1.
0
Visual Analog Scale
50

18 / 21
The Present Study Adds to the Literature:
• Dispersion as proxy of uncertainty was previously calculated only from point
forecasts
Presents a meaningful dispersion measure based on qualitative
expectations
• There was no direct uncertainty measure derived from qualitative responses
Presents a meaningful measure of “epistemic” uncertainty without
eliciting probabilistic forecasts
• Business expectations were assumed to be normally distributed
Demonstrates systematically variable skewness of business
expectations over the business cycle
19 / 21
General Summary
• VAS is easy to apply and does not require any quantitative information
• VAS presents a direct, almost interval scale measure of economic expectations
• VAS delivers reliable and valid information
• VAS is highly efficient, delivering a variety of valid economic indicators
- Dispersion measures
- Uncertainty proxies
- Distributional shape
- Enables differentiation between uncertainty and heterogeneity
• Assumptions of the three-category expectations are systematically violated
VAS is an effective instrument for the measurement of business
expectations
20 / 21
Research outlook
•
Gathering longer time-series
•
Improving the business cycle forecasts with the information on uncertainty
and heterogeneity of expectations (out-of-sample forecasts)
•
VAS measurement of consumer expectations
•
Application of VAS batteries in surveys, analysis of drop-outs
21 / 21
Email: stangl@ifo.de
22 / 21