For a linear model, the noncentered form is
$Y_i=\beta_0+\beta_1x_{i1}+…+\beta_kx_{ik}+\epsilon_i$ —(a)
the centered form is
$Y_i=\alpha+\beta_1(x_{i1}-\bar{x}_1)+…+\beta_k(x_{ik}-\bar{x}_k)+\epsilon_i$ —(b)
And we know that the matrix form of the centered form is
$Y=[j_n,X_c] (\alpha,\beta^*)'+\epsilon$ where $X_c=(I-\frac{1}{n}J_n)X$, $X=[X_1,…,X_k]$.
We could show that the estimators for $\alpha$ and $\beta^*$ are $\hat{\alpha}=\bar{Y}$ and $\hat{\beta}^*=(X_c'X_c)^{-1}X_c'Y$
My question is:
How to show $\hat{\beta}^*$ is the same as the last three elements of $\hat{\beta}=(X_a'X_a)^{-1}X_a'Y$, where $X_a=[j_n,X]$. What is the relationship between $\hat{\alpha}$ and the first element of $\hat{\beta}$?
Best Answer
The key to answering the question is to use the inverse of a 2x2 block matrix. The following formula is useful. Suppose $\boldsymbol{A}$ and $\boldsymbol{D}$ are invertible, then \begin{eqnarray*} \begin{pmatrix} \boldsymbol{A} & \boldsymbol{B} \\ \boldsymbol{C} & \boldsymbol{D} \end{pmatrix}^{-1} &=& \begin{pmatrix} \left(\boldsymbol{A}-\boldsymbol{B}\boldsymbol{D}^{-1}\boldsymbol{C}\right)^{-1} & 0 \\ 0 & \left(\boldsymbol{D}-\boldsymbol{C}\boldsymbol{A}^{-1}\boldsymbol{B}\right)^{-1} \end{pmatrix} \begin{pmatrix} \boldsymbol{I} & -\boldsymbol{B}\boldsymbol{D}^{-1} \\ -\boldsymbol{C}\boldsymbol{A}^{-1} & \boldsymbol{I} \end{pmatrix}. \end{eqnarray*} Consider the partition $\boldsymbol{X}_a = \left[\boldsymbol{j}_n, \boldsymbol{X}\right]$. Likewise, we shall partition the regression parameters into an intercept term and slope terms \begin{eqnarray*} \boldsymbol{\beta} = \begin{pmatrix} \beta_0 \\ \boldsymbol{\beta}^{\ast} \end{pmatrix}. \end{eqnarray*}
Clearly, $\widehat{\boldsymbol{\beta}} = \left(\boldsymbol{X}_a^{\prime} \boldsymbol{X}_a\right)^{-1}\boldsymbol{X}_a^{\prime} \boldsymbol{y}$. Therefore, \begin{eqnarray*} \begin{pmatrix} \widehat{\beta}_0 \\ \widehat{\boldsymbol{\beta}}^{\ast} \end{pmatrix} &=& \begin{pmatrix} n & \boldsymbol{j}_n^{\prime} \boldsymbol{X} \\ \boldsymbol{X}^{\prime} \boldsymbol{j}_n & \boldsymbol{X}^{\prime}\boldsymbol{X} \end{pmatrix}^{-1} \begin{pmatrix} \boldsymbol{j}_n^{\prime} \boldsymbol{y} \\ \boldsymbol{X}^{\prime}\boldsymbol{y} \end{pmatrix} \\ &=& \begin{pmatrix} \left(n -\boldsymbol{j}_n^{\prime} \boldsymbol{X} \left(\boldsymbol{X}^{\prime} \boldsymbol{X}\right)^{-1}\boldsymbol{X}^{\prime} \boldsymbol{j}_n\right)^{-1} & 0 \\ 0 & \left(\boldsymbol{X}^{\prime} \left[\boldsymbol{I}_n - \frac{\boldsymbol{J}_n}{n}\right]\boldsymbol{X}\right)^{-1} \end{pmatrix} \begin{pmatrix} \boldsymbol{j}_n^{\prime}\left[\boldsymbol{I}_n - \boldsymbol{X} \left(\boldsymbol{X}^{\prime} \boldsymbol{X}\right)^{-1}\boldsymbol{X}^{\prime}\right]\boldsymbol{y} \\ \boldsymbol{X}^{\prime}\left[\boldsymbol{I}_n - \frac{\boldsymbol{J}_n}{n}\right]\boldsymbol{y} \end{pmatrix} \end{eqnarray*}