Solved – Conditional Maximum Likelihood Estimation for ARMA(p,q)

Question

I'm making a study on STARIMA (Space-Time Autoregressive Integrated Moving Average) Time Series. And one problem I have is the parameter estimation since there is no software yet that has a STARIMA function, so far from what I have searched. Now, I want to compute it manually using estimation procedure. And I prefer Conditional Maximum Likelihood Estimation as it is used in STARIMA. I want to start with the univariate time series, the ARMA(p,q). I wish to estimate first the parameters of ARMA(p,q), then I'll just apply the procedure in STARMA.

According to William W.S. Wei author of Time Series Analaysis: Univariate and Multivariate Methods that (Original text from his book):

---

For the general stationary ARMA(p,q) model$$Z_t^*=\phi_1Z_{t-1}^*+\cdots+\phi_pZ_{t-p}^*+a_t-\theta_1a_{t-1}-\cdots-\theta_qa_{t-q}\quad(7.2.1)$$where $Z_t^*=Z-\mu$ and $\{a_t\}$ are i.i.d. $N(0,\sigma_a^2)$ white noise, the joint probability density of $\mathbf{a}=(a_1,a_2,\cdots,a_n)'$ is given by$$P(\mathbf{a}|\phi,\mu,\theta,\sigma_a^2)=(2\pi\sigma_a^2)^{-n/2}exp\left[-\frac{1}{2\sigma_a^2}\sum_{t=1}^na_t^2\right].\quad(7.2.2)$$Rewriting $(7.2.1)$ as$$a_t=\theta_1a_{t-1}+\cdots+\theta_qa_{t-q}+Z_t^*-\phi Z_{t-1}^*-\cdots-\phi_pZ_{t-p}^*\quad(7.2.3)$$we can write down the likelihood function of the parameters $(\phi,\mu,\theta,\sigma_a^2)$. Let $\mathbf{Z}=(Z_1,Z_2,\cdots,Z_n)'$ and assume the initial conditions $\mathbf{Z_*}=(Z_{1-p},\cdots,Z_{-1},Z_0)'$ and $\mathbf{a_*}=(a_{1-q},\cdots,a_{-1},a_0)$. The conditional log-likelihood function$$\ln L_*(\phi,\mu,\theta,\sigma_a^2)=-\frac{n}{2}\ln 2\pi\sigma_a^2-\frac{S_*(\phi,\mu,\theta)}{2\sigma_a^2}\quad(7.2.4)$$where$$S_*(\phi,\mu,\theta)=\sum_{t=1}^na_t^2(\phi,\mu,\theta|\mathbf{Z_*},\mathbf{a_*},\mathbf{Z})\quad(7.2.5)$$is the conditional sum of squares function. The quantities of $\widehat{\phi},\widehat{\mu}$ and $\widehat{\theta}$, which maximize Equation $(7.2.4)$, are called the conditional maximum likelihood estimators.

---

Now, I don't have idea how to find the quantities $\widehat{\phi},\widehat{\mu}$ and $\widehat{\theta}$ that maximizes $(7.2.4)$. Is this the same with the usual Maximum Likelihood Estimation like for parameters of say Geometric Distribution? If so, how will I obtain the formula of each parameters ($\widehat{\phi},\widehat{\mu}$ and $\widehat{\theta}$)? I know its tedious to perform MLE, but just give me an idea how to set up the likelihood function, and I'll just perform the usual MLE for parameters of ARMA(p,q). Well, if you could give me other source for parameter estimation of STARIMA then that would be great.

Thanks in advance!

Answer 1

Conditional maximum likelihood works the same way as unconditional maximum likelihood. You can use the same techniques to maximize the conditional loglikelihood. The only difference is that the likelihood function is constructed conditional on past values of the series and in this case apparently some of the noise components.