A Measure of Forecast Accuracy
Prepared by:
Tyler Hedin

Agenda
• Tracking Signal Defined
– Tracking Signal and Forecasting
• Application, Advantages & Disadvantages
• How it works
– Step by step formula
• Company XYZ Example
• Exercise
• Summary
• Readings list
• Useful websites
• Appendix A

•
A measure that indicates whether the forecast average is keeping pace with any genuine upward or downward changes in demand*

• Continuous control indicator
• Monitor effectiveness of forecasting method
• Provide control limits

• Evaluates forecasting method
• Indicator of change in demand patterns
• Used in conjunction with anything dependent on future demand
– Sales
– Inventory

• Unbiased
• Versatile
– Can be used with any type of forecasting method (time series, regression line, etc.)

• Could wrongfully flag perfect forecasts
– Unlikely
• Small differences in the same direction could cause signal to go outside of control limits

How it Works – Forecast Error
• Difference between actual demand and forecast
Week
1
2
3
Actual
Demand
21
25
22
Forecasted
Demand
19
22
24
Forecast
Error
2
3
-2

• Express the forecast errors as absolute values
Week
1
2
3
Actual
Demand
21
25
22
Forecasted
Demand
19
22
24
Forecast
Error
2
3
-2
Absolute
Value
2
3
2

• Keep a continuous running sum of the forecast errors
• Do not add absolute values
Week
1
2
3
Actual
Demand
21
25
22
Forecasted
Demand
19
22
24
Forecast
Error
2
3
-2
Absolute
Value
2
3
2
Running
Sum
2
5
3

• Divide the summed absolute values by the number of periods to calculate MAD.
Week
1
2
3
Actual
Demand
21
25
22
Forecasted
Demand
19
22
24
Forecast
Error
2
3
-2
Absolute
Value
2
3
2
Running
Sum
2
5
3
MAD
2.00
2.50
2.33

• Tracking signal is mathematically defined as the sum of the forecast errors divided by the mean absolute deviation
Tracking signal =

(D t
– F
MAD t
)

How it Works – Tracking Signal
• Divide the running sum of forecast errors by the corresponding MAD value
Week
1
2
3
Actual
Demand
21
25
22
Forecasted
Demand
19
22
24
Forecast
Error
2
3
-2
Absolute
Value
2
3
-2
Running
Sum
2
5
3
MAD
2.00
2.50
2.33
Tracking
Signal
1.00
2.00
1.29

• Ratio of cumulative error to average deviation
• 0.8 σ ~ 1.25 MAD
• Limits are usually between 2 to 5 standard deviations

• Company XYZ has implemented a linear regression method to forecast sales. Actual sales for the months of January 2005 through January 2006 are given in Table 1 along with their corresponding forecasts.

MONTH
January-05
February-05
March-05
April-05
May-05
June-05
July-05
August-05
September-05
October-05
November-05
December-05
January-06
10
11
12
13
7
8
9
SALES (in thousands)
PERIOD
1
DEMAND
$37
2
3
4
5
6
$40
$41
$37
$45
$50
$43
$47
$56
$52
$55
$54
$55
FORECAST
37.35
38.97
40.60
42.23
43.85
45.48
47.10
48.73
50.36
51.98
53.61
55.23
56.86

• Company XYZ would like to employ a tracking signal to measure the performance of its forecasting method.

TRACKING
MONTH
January-05
February-05
March-05
April-05
May-05
June-05
July-05
August-05
September-05
October-05
November-05
December-05
January-06
PERIOD DEMAND FORECAST ERROR ABS DVN RUNNING SUM MAD
1
2
37
40
37.35
38.97
-0.35
1.03
0.35
1.03
-0.35
0.68
0.35
0.69
SIGNAL
-1.00
-1.00
0.99
0.99
1.82
3 41 40.60
0.40
0.40
1.08
0.59
1.82
4 37 42.23
-5.23
5.23
-4.15
1.75
-2.37
-2.37
5 45 43.85
1.15
1.15
-3.00
1.63
-1.84
-1.84
6 50 45.48
4.52
4.52
1.52
2.11
0.72
0.72
7 43 47.10
-4.10
4.10
-2.58
2.40
-1.08
-1.08
8
9
10
11
12
13
47
56
52
55
54
55
48.73
50.36
51.98
53.61
55.23
56.86
-1.73
5.64
0.02
1.39
-1.23
-1.86
1.73
5.64
0.02
1.39
1.23
1.86
-4.31
1.33
1.35
2.74
1.51
-0.35
2.31
-1.86
-1.86
2.68
0.50
0.50
2.42
0.56
0.56
2.32
1.18
1.18
2.23
2.20
0.68
0.68
-0.16
-0.16

• Your employer, Jones & Associates, has been using a linear regression method to forecast sales for 2006. After nine months have passed and actual sales data have been collected, your boss asks you to develop a tracking signal to measure the accuracy of the forecasts. The data for actual sales and forecasted sales is in Table 3.

MONTH
January-06
February-06
March-06
April-06
May-06
June-06
July-06
August-06
September-06
7
8
5
6
9
PERIOD
SALES
DEMAND
1
2
$3,769
$3,912
3
4
$4,212
$4,861
$4,672
$4,937
$5,346
$5,783
$6,021
FORECAST
3664.18
3953.92
4243.65
4533.39
4823.13
5112.87
5402.61
5692.35
5982.08

• A tracking signal statistically determines if a forecasting method is out-of-control.
– As long as tracking signal stays within 3 standard deviations, probability of forecast error caused by random variation is high
• Used by companies to track changes in demand patterns
• Calculated by dividing the most recent sum of forecast errors by the most recent estimate of MAD
• A tracking signal outside of established limits indicates that a forecasting method should be modified.
• Compatible with any forecasting method

• Chase, R. B. et al. (2004).
Operations Management for
Competitive Advantage 10 th edition . McGraw-Hill Higher
Education.
• Duncan, Robert M. (1992). Quality Forecasting Drives
Quality Inventory at GE. Industrial Engineer, January edition.
• Hanke, J.E. & Wichern, D. W. (2004).
Business
Forecasting . Prentice Hall.
• Lawrence, F. B. (1999). Closing the logistics loop: A tutorial. Production & Inventory Management Journal ,
40(1).

• http://www.bestforecastingsoftware.com
• http://www.IdeaWins.com
• http://www.lehigh.edu/~rhs2/IBE098/forecating.ppt
• http://is.ba.ttu.edu/faculty/ch11.ppt
• http://www.microsoft.com/dynamics/intro/default.mspx

MONTH
January-05
February-05
March-05
April-05
May-05
June-05
July-05
August-05
September-05
October-05
November-05
December-05
January-06
9
10
11
12
13
PERIOD DEMAND FORECAST ERROR ABS DVN RUNNING SUM MAD
1 37 37.35
-0.35
0.35
-0.35
0.35
2
3
4
40
41
37
38.97
40.60
42.23
1.03
0.40
-5.23
1.03
0.40
5.23
0.68
1.08
-4.15
0.69
0.59
1.75
5
6
7
8
45
50
43
47
43.85
45.48
47.10
48.73
1.15
4.52
-4.10
-1.73
1.15
4.52
4.10
1.73
-3.00
1.52
-2.58
-4.31
1.63
2.11
2.40
2.31
56
52
55
54
55
50.36
51.98
53.61
55.23
56.86
5.64
0.02
1.39
-1.23
-1.86
5.64
0.02
1.39
1.23
1.86
1.33
1.35
2.74
1.51
-0.35
2.68
2.42
2.32
2.23
2.20
TRACKING
SIGNAL
-1.00
0.99
1.82
-2.37
-1.84
0.72
-1.08
-1.86
0.50
0.56
1.18
0.68
-0.16