MMSE Forecasting Regression Models with ARIMA Errors

What Are MMSE Forecasts?

An objective of time series analysis is generating forecasts for responses over a future time horizon. That is, you can generate predictions fory_{T+ 1},y_{T+ 2}、……y_{T+ h}given the following:

An observed seriesy₁,y₂、……y_T
A forecast horizonh
Nonstochastic predictorsx₁,x₂、……x_T、……x_T+h, wherex_kis anr-vector containing the measurements ofrpredictors observed at timek
A regression model with ARIMA errors

$\begin{array}{l} y_{t} = c + X_{t} β + u_{t} \\ Η (L) u_{t} = Ν (L) ε_{t}, \end{array}$

where H(L) and N(L) are compound autoregressive and moving average lag operator polynomials (possibly containing integration), respectively.

Let ${\hat{y}}_{t + 1}$ denote a forecast for the process at timet+ 1, conditional on the history of the process up to timet(H_t), and assume that the predictors are fixed. The minimum mean square error (MMSE) forecast is the forecast ${\hat{y}}_{t + 1}$ that minimizes expected square loss,

$E {(y_{t + 1} - {\hat{y}}_{t + 1} | H_{t})}^{2} .$

Minimizing this loss function yields the MMSE forecast,

${\hat{y}}_{t + 1} = E (y_{t + 1} | H_{t}) .$

How forecast Generates MMSE Forecasts

forecastgenerates MMSE forecasts recursively. When you callforecast, you must specify aregARIMAmodel (Mdl) and the forecast horizon. You can also specify presample observations (Y0), predictors (X0), innovations (E0), and conditional disturbances (U0) using name-value pair arguments.

To begin forecastingy_tstarting at timeT+ 1, use the last few observations ofy_tandX_tas presample responses and predictors to initialize the forecast. Alternatively, you can specify presample unconditional disturbances or innovations.

However, when you specify presample data:

If you provide presample predictor data (X0), then you must also provide predictor forecasts (XF). It is best practice to setX0to the same predictor matrix that estimates the parameters. If you do not provide presample and future predictors, thenforecastignores the regression component in the model.
If the error process inMdlcontains a seasonal or nonseasonal autoregressive component, or seasonal or nonseasonal integration, thenforecastrequires a minimum ofPpresample unconditional disturbances to initialize the forecast. The propertyPofMdlstoresP.
If the error process inMdlcontains a seasonal or nonseasonal moving average component, thenforecastrequires a minimum ofQpresample innovations to initialize the forecast. The propertyQofMdlstoresQ.
If you provide a sufficient amount of presample unconditional disturbances, thenforecastignoresY0andX0. If you also do not provideE0, but provide enough presample unconditional disturbances, thenforecastinfers the required amount of presample innovations from the ARIMA error model andU0.
If you provide a sufficient amount of presample responses and predictors (and do not provideU0), thenforecastuses the regression model to infer the presample unconditional disturbances.
If you do not provide presample observations, thenforecastsets the required amount of presample unconditional disturbances and innovations to 0.
If you provide an insufficient amount of presample observations, thenforecastreturns an error.

Consider generating forecasts from a regression model with ARMA(3,2) errors:

$\begin{matrix} y_{t} = c + X_{t} β + u_{t} \\ (1 - a_{1} L - a_{2} L^{2} - a_{3} L^{3}) u_{t} = (1 + b_{1} L + b_{2} L^{2}) ε_{t} \\ or \\ a (L) u_{t} = b (L) ε_{t}, \end{matrix}$

wherea(L) andB(L) are lag operator polynomials. The largest AR lag is 3, the largest MA lag is 2. This model does not contain any seasonal lags nor integration. Therefore,P= 3 andQ= 2. To forecast this model, you need three presample responses and predictors, or three presample unconditional disturbances, and two presample innovations.

Given presample unconditional disturbances $(u_{T - 2}, u_{T - 1}, u_{T}),$ presample innovations $(ε_{T - 1}, ε_{T}),$ and future predictors $(X_{T + 1}, X_{T + 2}, ...),$ you can forecast the model as follows:

$\begin{array}{l} {\hat{u}}_{T + 1} = a_{1} u_{T} + a_{2} u_{T - 1} + a_{3} u_{T - 2} + b_{1} ε_{T} + b_{2} ε_{T - 1} \\ {\hat{y}}_{T + 1} = c + X_{T + 1} β + {\hat{u}}_{T + 1} . \end{array}$
$\begin{array}{l} {\hat{u}}_{T + 2} = a_{1} {\hat{u}}_{T + 1} + a_{2} u_{T} + a_{3} u_{T - 1} + b_{2} ε_{T} \\ {\hat{y}}_{T + 2} = c + X_{T + 2} β + {\hat{u}}_{T + 2} . \end{array}$
$\begin{array}{l} {\hat{u}}_{T + 3} = a_{1} {\hat{u}}_{T + 2} + a_{2} {\hat{u}}_{T + 1} + a_{3} u_{T} \\ {\hat{y}}_{T + 3} = c + X_{T + 3} β + {\hat{u}}_{T + 3} . \end{array}$

...

Note that:

Future innovations take on their unconditional mean, 0.
For stationary error processes, such as this one:
- 的forecasted unconditional disturbances converge to their unconditional mean,
  
  $E (u_{t}) = \frac{b (L)}{a (L)} E (ε_{t}) = 0.$
- c+X_tβgoverns the long-term behavior of the forecasted responses.

Forecast Error

的forecast error for ans-step ahead forecast of a regression model with ARIMA errors is

$\begin{matrix} MSE = E {(y_{T + s} - {\hat{y}}_{T + s} | H_{T + s - 1})}^{2} \\ = E {(c + X_{T + s} β + u_{T + s} - c - X_{t + s} β - {\hat{u}}_{T + s} | H_{T + s - 1})}^{2} \\ = E {(u_{T + s} - {\hat{u}}_{T + s} | H_{T + s - 1})}^{2} \\ = \frac{Ν (L)}{Η (L)} E (ε_{t}^{2} | H_{T + s - 1}) \\ = ψ (L) σ^{2}, \end{matrix}$

where the dividendψ(L)是一个无限延迟算子多项式σ²is the innovation variance.

If the error process is stationary, then the coefficients ofψ(L) are absolutely summable. Therefore, the MSE (mean square error) converges to the unconditional variance of the process[1].

If the error process is not stationary, then the MSE grows with increasings.

References

[1] Box, G. E. P., G. M. Jenkins, and G. C. Reinsel.Time Series Analysis: Forecasting and Control. 3rd ed. Englewood Cliffs, NJ: Prentice Hall, 1994.

Related Examples

More About

Monte Carlo Forecasting of regARIMA Models