Introduction
The ARIMA (autoregressive integrated moving average) models are also known as Box–Jenkins models. ARIMA models are applied in some cases where data show evidence of non-stationarity, where an initial differencing step (corresponding to the “integrated” part of the model) can be applied one or more times to eliminate the non-stationarity.
The AR part of ARIMA indicates that the evolving variable of interest is regressed on its own lagged (i.e., prior) values. The MA part indicates that the regression error is actually a linear combination of error terms whose values occurred contemporaneously and at various times in the past. The I (for “integrated”) indicates that the data values have been replaced with the difference between their values and the previous values (and this differencing process may have been performed more than once).
When two out of the three model parameters are zeros, the model may be referred to based on the non-zero parameter, dropping “AR”, “I” or “MA” from the acronym describing the model. For example, \(ARIMA (1,0,0)\) is \(AR(1)\), \(ARIMA(0,1,0)\) is \(I(1)\), and \(ARIMA(0,0,1)\) is \(MA(1)\).
AR(p)
An autoregressive (AR) model is a representation of a type of random process. The AR model specifies that the output variable depends linearly on its own previous values and on a stochastic term. In general, the AR model of order \(p\), i.e. \(AR(p)\), is defined as \[y_t = c + \sum_{i=1}^p \phi_i y_{t-i} +\epsilon_t,\] where \(\phi_1,\cdots,\phi_p\) are the parameters, \(c\) is a constant, and \(\epsilon_t\) is white noise often assumed following \(N(0,\sigma^2)\).
For AR(1), \(|\phi|<1\) is necessary for the process to be stationary, such that \[E(y_t) = E(y_{t-1}) = \mu,\] \[\text{var}(y_t) = \text{var}(y_{t-1})=\frac{\sigma^2}{1-\phi^2} = \sigma_y^2,\] assuming \(y_{t}\)’s and \(\epsilon_t\)’s are independent from each other. The condition \(|\phi|<1\) appears in the variance term so that \(\sigma_y^2\) is finite and positive. Note that \(\text{var}(y_t)\) in AR(1) is larger than in AR(0) i.e. regular linear models without autoregressions.
I(d)
Differencing in statistics is a transformation applied to time-series data in order to make it stationary. Differencing removes the changes in the level of a time series, eliminating trend and seasonality and consequently stabilizing the mean of the time series. The differenced data is then used for the estimation of an ARMA model.
The I(1) model of first-order differencing can be written as \[D(y_t) = y_t - y_{t-1} = \epsilon_t,\] and the I(2) model of second-order differencing can be written as \[D^2(y_t) = D(y_t) - D(y_{t-1}) = y_t - 2y_{t-1}+y_{t-2} = \epsilon_t,\] where \(D()\) is the operator of differencing and \(D^d(y_t) = D^{d-1}(y_t) - D^{d-1}(y_{t-1})\).
Another method of differencing data is seasonal differencing, which involves computing the difference between an observation and the corresponding observation in the previous period. For example, \[y'_t = y_t - y_{t-s},\] where \(s\) is the duration of season. We denote it as \(D_s(y_t)\).
MA(q)
The moving average model of order \(q\), i.e. \(MA(q)\), is given as \[y_t = \mu + \epsilon_t + \sum_{i=1}^q\theta_i\epsilon_{t-i},\] where \(\theta_1,\cdots,\theta_q\) are the parameters, \(\mu\) is the expectation of \(y_t\), and \(\epsilon_t, \epsilon_{t-1},\cdots\) are white noise error terms.
ARIMA(p,d,q)
Non-seasonal ARIMA
Non-seasonal ARIMA models are generally denoted \(ARIMA(p,d,q)\) where parameters \(p, d, q\) are non-negative integers, \(p\) is the order (number of time lags) of the autoregressive model, \(d\) is the degree of differencing (the number of times the data have had past values subtracted), and \(q\) is the order of the moving-average model. In general, an \(ARIMA(p,d, q)\) model is given as \[D^d(y_t) = \delta + \sum_{i=1}^p\phi_i D^d(y_{t-i}) + \epsilon_t + \sum_{i=1}^q \theta_i \epsilon_{t-i},\] where \(D^d(y_t)\) is the \(d\)-order difference of \(y_t\).
Seasonal ARIMA
Seasonal ARIMA models are usually denoted \(ARIMA(p,d,q)\times(P,D,Q)_s\), where \(s\) refers to the time span of repeating seasonal pattern, and the uppercase \(P,D,Q\) refer to the autoregressive, differencing, and moving average terms for the seasonal part of the ARIMA model.
The non-seasonal components are the same as in non-seasonal ARIMA. As for the seasonal components, we have, for example,
Seasonal AR: \(y_t = c + \sum_{i=1}^P\psi_i y_{t-si}\)
Seasonal MA: \(y_t = \mu+ \epsilon_t + \sum_{i=1}^Q\eta_i\epsilon_{t-si}\)
Seasonal I: \(D^D_s(y_t) = D^{D-1}_s(y_t)- D_s^{D-1}(y_{t-s})\)
Examples
Some well-known special cases arise naturally or are mathematically equivalent to other popular forecasting models. For example:
An \(ARIMA(0,1,0)\) model (or \(I(1)\) model) is given by \(X_{t}=X_{t-1}+\epsilon_{t}\), which is simply a random walk.
An \(ARIMA(0,1,0)\) with a constant, given by \(X_{t}=c+X_{t-1}+\epsilon_{t}\), which is a random walk with drift.
An \(ARIMA(0,0,0)\) model is a white noise model.
An \(ARIMA(0,1,2)\) model is a Damped Holt’s model.
An \(ARIMA(0,1,1)\) model without constant is a basic exponential smoothing model.
An \(ARIMA(0,2,2)\) model is given by \(X_{t}=2X_{t-1}-X_{t-2}+(\alpha +\beta -2)\epsilon_{t-1}+(1-\alpha )\epsilon_{t-2}+\epsilon_{t}\), which is equivalent to Holt’s linear method with additive errors, or second-order exponential smoothing.
