Tutorial #4: Model Evaluation & Selection
*In using Stata 7.0 and Stata 8.0, some commands may differ. In the case that there is a
difference, it will be noted as V8, for version 8.0 commands and V7 for version 7.0 commands.
*To change from the Stata 8.0 to Stata 7.0, type “version 7.0” in the command line. Likewise, to
change back to Stata 8.0, type “version 8.0”
4.1 Model Diagnostics
To conduct a Box-Pearce statistic test on the residuals:
First run the ARMA model, then generate the residuals.
. predict e, resid
The command to run the test is:
. wntestq e
An example:
Generate a random sample of 100 observations:
. set seed 1974
. set obs 100
Create the time trend and set the time variable:
. gen t=_n
. tsset t
Generating niid (0,1) errors:
. gen u=invnorm(uniform())
Generate an AR(2) process with the data AR(2) yt= p*yt-1 + q*yt-2 + et
. gen y=u if t<3
. replace y=0.1*L.y + 0.3*L2.y + u if t>2
Estimate ARMA(1,1):
arima y, arima(1,0,1) bfgs
Implement portmanteau-test:
. predict e, resid
. wntestq e
Compare these results to an ARMA(2,1):
. arima y, arima(2,0,1) bfgs
. predict e2, resid
. wntestq e2
Now, redo these same estimates with a large sample n=200.
4.2 Forecasting
To forecast using the sample generated in the previous example, specify the following command:
. fore d.y t, beg(1) end(100)
Note: kind() does not need to be specified in this case, given that the time variable is a time trend,
and not the yearly, where kind(y) would be specified, or monthly type with kind(m).
Generate the root mean squared error (RMSE):
. generate err = y - fore
. summarize err
The standard deviation in this summary table is the RMSE.