Quantifying and measuring revisions in time series estimates

advertisement



13th Meeting of the National Statistics Methodology Advisory Committee

Quantifying and Measuring Revisions in Time Series Estimates

Gary Brown and Duncan Elliott

Executive summary

Public and private organisations make crucial decisions based on their interpretations of movements and revisions in published ONS time series. We present a way of decomposing these movements and revisions to provide more detailed and transparent information on which to base these decisions.

The extra information will create a framework for ONS to further explore time series revisions, and to better quality assure its own processes and outputs to improve targeting of development resource.

Aims of paper

To discuss a new approach developed to:

1. help users better understand movements and revisions and their impact on different time series estimates

2. provide a framework for wider research into revisions

Requested actions from the committee

We would like to hear the opinions of NSMAC members on the general approach undertaken in the paper, and their suggestions for improving upon it.

We would also like the NSMAC to comment on the main issues for discussion listed below.

Main issues for discussion

1. Do the NSMAC members consider these representations of movements and revisions useful?

Should this decomposition approach be part of other ONS revisions systems? Do they have any suggestions for improvement?

2. Do the NSMAC members consider correlations and mean square revision (MSR) ratios useful measures for explaining revisions? What others do they suggest?

3. What do the NSMAC members think of the general decomposition approach? When considering revisions to original vs revisions to SA, is it useful to simulate first? What other options should

ONS consider?

1

NSMAC (13): Time Series

Quantifying and Measuring Revisions in Time Series Estimates

1. Introduction

ONS time series estimates provide crucial information for economic and social decision making and policy setting. For monthly and quarterly time series, although the original, seasonally adjusted and trend series are estimated, in general only the original and seasonally adjusted are published on a regular basis.

Revisions to series are an important part of the ONS production process, as reflected by continued research and development work in ONS, both singly and in partnership with the Organisation for

Economic Co-operation and Development. This paper summarises key issues from Elliott et al. (2007), which essentially covered the same ground but with more theory and explanation of ONS processes.

There is also considerable media interest in the size and nature of revisions. This constitutes a clear need to publish transparent revisions policies and detailed information about revisions.

 “…the Office for National Statistics (ONS) will have to raise its game, and cut the number and size of its revisions. These are so frequent that even the Bank of England says it does not believe the official statistics.” (Daily Telegraph, 29/11/2005)

 “After the ONS last week unveiled far-reaching revisions to a raft of official data, the Statistics

Commission said that it was alarmed by the flaws in figures that are crucial for setting economic policy.” (The Times, 6/7/2004)

Revisions can occur for three basic reasons: revisions to existing (original) data; data for new periods; seasonal adjustment (SA) model changes. Table 1 summarises the impact of each type on published series.

Series\Revision type

Original

Seasonally adjusted

Trend

Revised data

Revision

Revision

Revision

New data

No revision

Revision

Revision

SA model changes

No revision

Revision

Revision

Table 1: Impact of revisions on published series, by revision type

2. Framework

We assume a simple multiplicative time series decomposition (ie excluding trading days/holiday effects)

Y t|

 = C t|

 S t|

 I t|

 where:

Y t|

 is the original series at time point t given data up to and including time point

C t|

 is the trend component

S t|  is the seasonal component

I t|

 is the irregular component

(1) 

ONS uses the X-11 framework to calculate the seasonally adjusted series, SA t| 

SA t|  = Y t|  /S t|  (2)

2

NSMAC (13): Time Series

2.1 Revisions

We denote a revision of Y t|

 , after period m, as

 m) Y t

= Y t|  +m

- Y t|     (3)

2.2 Growth rates and revisions

We write the growth rate of the original series from time t – j to t as

Y

(t,j)| 



Y t-j|  = [Y t| 

Y t-j| 



Y t-j|   which can be decomposed into component growth rates as

   (4)

Y

(t,j)|





Y t-j|

 = [C t|

 S t|

 I t|



C t-j|

 S t-j|

 I t-j|



C t-j|

 S t-j|

 I t-j|

   

=

C

(t,j)|



C t-j|

 +

S

(t,j)|



S t-j|

 +

I

(t,j)|





I t-j|

 + a

(t,j)|

   (5) where a

(t,j)|

 represents second and third order terms (ie [

C

(t,j)|

S

(t,j)|

 ]



C t-j|

 S t-j|

 ] etc). Revisions can be similarly decomposed. Most importantly, the revision to the growth rate of the seasonally adjusted series as the sum of revisions to growth rates of trend and irregular series is given by

 m)



SA

(t,j)|





SA t-j|

 ] =

 m) [

C

(t,j)|



C t-j|

 ] +

 m) [

I

(t,j)|





I t-j|

 ] +

 m) [b

(t,j)|

 ]    (6) where b

(t,j)|

 represents second and third order interaction terms.

2.3 Revisions in aggregation structures

For a simple aggregation structure, where Y

1t|

 and Y

2t|

 sum to Y t|

 , the aggregate series can either be directly or indirectly seasonally adjusted (ie by summing the seasonally adjusted component series

SA(Y

1t|

 ) and SA(Y

2t|

 )). Direct and indirect seasonal adjustment will generally produce different results, although they will be equal when a linear filter is used to approximate an X-11 type seasonal adjustment

SA t|

 =

 w t,i

Y t+i|

 ;i = k, … k

   (7)  where w t,i

is the weight for time point t+i.

2.3.1 Original series

The expected revision of the aggregate series is the sum of revisions of the components, with variance var(

 m) Y t

) = var(

 m) Y

1t

) + var (

 m) Y

2t

) + 2cov(

 m) Y

1t

,

 m) Y

2t

) (8)

From (8), the mean square revision (MSR) of the aggregate series depends upon the MSR of the components and the covariance between their mean revisions. Assuming the means of the component revisions have expected values of zero, but that they are correlated (with coefficient ‘r’), then var(

 m) Y t

) = var(

 m) Y

1t

) + var (

 m) Y

2t

) + 2r[var(

 m) Y

1t

)var(

 m) Y

2t

)] 0.5

(9)

From (9), the MSR of the aggregate series is smaller or larger than the sum of the components MSRs depending upon the correlation of the revisions to the components.

3

NSMAC (13): Time Series

2.3.2 Seasonally adjusted series

The variance of the revision of the seasonally adjusted aggregate series is var[

 m) SA(Y t

)] = var[

 m) SA(Y

1t

)] + var[

 m) SA(Y

2t

)] + 2cov[

 m) SA(Y

1t

),



 m) SA(Y

2t|

 )] (10)

If there is a revision to time point t, no revision to any other, and we approximate seasonal adjustment by a linear filter then, as direct and indirect seasonal adjusted series are equal, var[

 m) SA(Y t+i|  )] = (w t-i,i

) 2 var(

 m) Y t|  ) ;i = k, … k (11)

If there are no revisions, but an additional data point is added to each component series var[

 m) SA(Y  -i

)] = (w  -i,i+1

) 2 var(

 m) Y  ) ;i = 0, … k (12)

3. Results

3.1 Presenting movements and revisions

Information on movements in the original or seasonally adjusted series are not always enough, although this is what ONS currently emphasizes in published estimates. The decomposition of movements into components provides a more complete picture for users, and enables better ONS quality assurance.

Figures 1 to 3 and 6 to 8 are constructed as follows.

Box-and-whiskers. The box is defined by the upper and lower quartiles; the line in the box is the median. The upper and lower whiskers extend to most extreme data points which are not more than 1.5 times the interquartile range distant from the respective quartiles.

Red points. The most recent data point.

Example 1: Month on month decomposition, point in time

Table 2 shows decomposed movements in the UK Index of Production (August – September 2006), original and seasonally adjusted series, using equation (5).

Series

Decomposition

Original

Seasonally adjusted

Overall

-5.17

Overall

-5.22

Trend

First order terms only

-0.15

Seasonal

-3.87

0.12 0.12 -0.15

Table 2: IoP Aug-Sept 2006 movement decomposition

Irregular Prior

0.27

0.27

-1.47

Movement in the original series is driven by “Seasonal Factor” and “Prior” (a prior adjustment for trading days). The difference between Order 1 and 4, ie the interactions a

(t,j)|

 , is negligible. For the seasonally adjusted estimate, the interaction is zero.

Example 2: Month on month decomposition, over time

Figure 1 shows the decomposition of movements over time. The main contributor over time is the

“Seasonal Factor”, and in 2006 (red points), an upward movement in “Irregular”, though countered by a downward movement in “Trend”, causes an upward movement in “Original”.

4

NSMAC (13): Time Series

40

4 0

30

3 0

20

2 0

10

1 0

0

0 o r i g i n a l t r e n d s e a s o n a l

Original Trend Seasonal

Factor i r r e g u l a r p r i o r s i n t e r a c t i o n

Irregular Priors Interaction terms

Figure 1: Jan-Feb movements over time. Recent = 2006.

Example 3: Year on year decomposition, over time

Figure 2 shows a larger than usual fall (4%) in the original series. The decomposition shows this is mainly due to the priors (ie prior adjustments for trading day factors), in addition to a fall in the trend.

However, the key piece of information (missing) here is that the previous year was a leap year.

-2

- 2

0

0

-4

- 4

4

4

2

2

-6 - 6 o r i g i n a l t r e n d s e a s o n a l

Original Trend Seasonal

Factor i r r e g u l a r p r i o r s i n t e r a c t i o n

Irregular Priors Interaction terms

Figure 2: Year on year Feb movements. Recent = 2005.

Example 4: Decomposition of revisions, to movements

Figure 3 shows a relatively high revision to the movement in the seasonally adjusted series is due to a large revision in the irregular component.

5

0

- 5

Irregular Interaction

Adj Series

Figure 3: Revisions to Sept - Oct growth. Recent = 2006.

5

NSMAC (13): Time Series

QUESTION 1: Do the NSMAC members consider these representations of movements and revisions useful?

Should this decomposition approach be part of other ONS revisions systems? Do they have any suggestions for improvement?

3.2 Understanding revisions

Example 5: Correlation of decomposition of revisions, due to new data

For revisions due only to an additional time point, we used 235 monthly time series to estimate revisions by components ( ie

1) Y  for

= 71, 7 2, …, 131).

Figure 4 shows evidence of negative correlation between revisions to the irregular and trend growth rates (the end point effect). There is also some evidence of negative correlation between revisions to the irregular and seasonal factor growth rates. No other evidence is seen, although it would be expected due to the X-12-ARIMA/seasonal adjustment process, where the product of the components is always equal to the original series.

- 1 0 trend

0 5 1 0 - 1 0 trend

0 5 1 0 - 1 0 5 1 0

1 0 - 1 0 - 5 0 5 seasonal factor

1 0 - 1 5 - 5 0 irregular

5 1 5 - 1 0 - 5 0 5 seasonal factor

Figure 4: Relationships between revisions & components

Example 6: Correlation of decomposition of revisions with forecast error

One major source of revision at the current end of a series is the quality of the ARIMA forecast.

Figure 5 shows evidence of correlations between revisions to growth rates of the components against the actual forecast error – positive for trend and negative for irregular.

- 1 0 0 - 5 0 0 5 0

F o r e c a s t E r r o r

1 0 0 - 1 0 0 - 5 0 0 5 0

F o r e c a s t E r r o r

1 0 0

- 1 0 0 - 5 0 0 5 0 1 0 0 - 1 0 0 - 5 0 0 5 0 1 0 0

F o r e c a s t E r r o r F o r e c a s t E r r o r

Figure5: Forecast error vs component growth revisions

Example 7: Simulation of seasonally adjusted and original SMR ratios

We simulated a two source revision (due to (1) i.i.d. N(0,1) revisions to the original series, and (2) an additional time point) examining the impact on the seasonally adjusted series using an X-11 process.

6

NSMAC (13): Time Series

Our simulation comprised 1000 replicates of: quarterly time series (length 100), 3x3 seasonal filter, 5term Henderson trend filter, end weight = 1, no outlier correction.

Our measurement statistic is the ratio of the mean square revision (MSR) of the seasonally adjusted series to the MSR of the original series, between the last and the current period.

Figure 6 shows revisions of levels in seasonally adjusted series are generally lower than original series, except for the current end – where they can be higher. The general result is due to the end point problem and use of asymmetric filter weights, ie seasonal adjustment smoothes the time series.

The change at the current end is due to implied forecast error.

These results demonstrate that when revisions occur in the original estimate in the middle of the series then we can expect revisions to the seasonally adjusted estimate to be lower.

Figures 7 and 8 show quarter on quarter and year on year differences, respectively, of the same MSR ratio, and have a similar pattern (and similar interpretation) as Figure 6.

QUESTION 2: Do the NSMAC members consider correlations and mean square revision (MSR) ratios useful measures for explaining revisions? What others do they suggest?

MSR Ratio between seasonally adjusted and orginal

over different last periods

1 4 8 12 100 last periods

Figure 6: MSR ratio (levels) for different last periods

MSR Ratio of QoQ between seasonally adjusted and orginal

over different last periods

1 4 8 12 100 last periods

Figure 7: MSR ratio (growths) for different last periods

7

NSMAC (13): Time Series

MSR Ratio of YoY between seasonally adjusted and orginal

over different last periods

1 4 8 12 100

4. Discussion last periods

Figure 8: MSR ratio (growths) for different last periods

We have presented an initial framework for providing detailed information on components to producers and users of statistics, to give a clearer understanding of movements and revisions in a time series.

The framework improves the transparency of the seasonal adjustment process. In particular it is of considerable use when series are affected by outliers, level shifts, trading day or holiday effects that may not be immediately obvious in the original or seasonally adjusted series. It also allows us to further explore the sources and impacts of revisions.

This work could be expanded to look at the impact of revisions as part of aggregation structures, and the effects of changing particular parameters in the seasonal adjustment process to understand how these can affect revisions.

QUESTION 3: What do the NSMAC members think of the general decomposition approach? When considering revisions to original vs revisions to SA, is it useful to simulate first? What other options should ONS consider?

Acknowledgements

The authors acknowledge the earlier work of Duncan Elliott, Craig McLaren and Mark Zhang (Australian

Bureau of Statistics) upon which this paper is based.

Reference

Elliott, D., McLaren, C.H., Zhang, X. (2007)

Quantifying and Measuring Revisions in Times-Series Estimates , (to appear in proceedings) 2007

Joint Statistical Meetings, 29 July – 2 August 2007, Salt Lake City, Utah.

8

NSMAC (13): Time Series

Download