Fixed Effects Dummies vs Estimator – Differences in Panel Data Models

categorical-encodingfixed-effects-modelleast squareslinear modelpanel data

I started to read about panel regression models. However, I am a bit confused about the different model specifications in the fixed effects model:

Does a fixed effects panel regression always mean that I introduce dummy variables for the cross-sections (e.g., for each country in my sample) and then run e.g. an OLS estimation?

What is the difference between adding fixed effects dummies to the regression model and the fixed effects estimator?

Thanks for your help!

Best Answer

To see equality, let us first derive the FE estimator.

Define the residual-maker matrix \begin{align*} \underset{(M\times M)}{\mathbf{Q}}&:=\mathbf{I}_M-\mathbf{1}_M(\mathbf{1}_M'\mathbf{1}_M)^{-1}\mathbf{1}_M'\\ &=\mathbf{I}_M-\left(% \begin{array}{ccc} 1/M & \cdots & 1/M \\ \vdots & \ddots & \vdots \\ 1/M & \cdots & 1/M \\ \end{array}% \right)\mathbf{1}_M\mathbf{1}_M', \end{align*} where $M$ denotes the number of observations per individual unit in the panel.

Premultiplication with $\mathbf{Q}$ centers the $\mathbf{y}_i$ and $\mathbf{Z}_i$ around their averages over $m$, \begin{align*} \mathbf{Q}\mathbf{y}_i&=\mathbf{y}_i-\mathbf{1}_M\mathbf{1}_M'\mathbf{y}_i/M\\&=\mathbf{y}_i-\mathbf{1}_M\overline{y_{i}}. \end{align*} The also implies that every time invariant variable from the set of regressors $\mathbf{Z}_i$ turns into a column of zeros, and hence is eliminated from the data.

This is a serious disadvantage of the FE estimator. Consider the example of wage regressions for a panel of employees. Variables such as gender or schooling are of primary interest, but (typically) do not change over time (anymore).

As $\mathbf{Q}\mathbf{1}_M=\mathbf{0}$, we have that, using the error-component model $\mathbf{y}_i=\mathbf{Z}_i\mathbf{\delta}+\mathbf{1}_M\alpha_i+\mathbf{\eta}_{i}$, where $\eta_i$ denotes the $M$-vector of idiosyncratic time-varying errors, \begin{align*} \mathbf{Q}\mathbf{y}_i&=\mathbf{Q}\mathbf{F}_i\mathbf{\beta}+\mathbf{Q}\mathbf{\eta}_{i}\qquad i=1,\ldots,n\\ \tilde{\mathbf{y}}_i&\equiv\tilde{\mathbf{F}}_i\mathbf{\beta}+\tilde{\mathbf{\eta}}_{i}, \end{align*} where $\mathbf{F}_i$ is the $(M\times L_b)$-matrix of the observations on the time variant regressors. Stacking the observations over the $n$ units gives $$ \underset{(Mn\times 1)}{\tilde{\mathbf{y}}}:=\left(% \begin{array}{c} \tilde{\mathbf{y}}_1 \\ \vdots \\ \tilde{\mathbf{y}}_n \\ \end{array}% \right)\qquad\underset{(Mn\times L_b)}{\tilde{\mathbf{F}}}:=\left(% \begin{array}{c} \tilde{\mathbf{F}}_1 \\ \vdots \\ \tilde{\mathbf{F}}_n \\ \end{array}% \right) $$

The FE estimator is simply OLS applied to these $Mn$ observations: \begin{align*} \widehat{\mathbf{\beta}}_{\text{FE}}&=(\tilde{\mathbf{F}}'\tilde{\mathbf{F}})^{-1}\tilde{\mathbf{F}}'\tilde{\mathbf{y}} \end{align*}

To see the equality between FE and least squares dummy variables, stack the observations a bit further: \begin{equation} \underset{(Mn\times 1)}{\mathbf{y}}:=\left(% \begin{array}{c} \mathbf{y}_1 \\ \vdots \\ \mathbf{y}_n \\ \end{array}% \right)\;\underset{(Mn\times L_b)}{\mathbf{F}}:=\left(% \begin{array}{c} \mathbf{F}_1 \\ \vdots \\ \mathbf{F}_n \\ \end{array}% \right) \end{equation} and \begin{equation} \underset{(Mn\times 1)}{\mathbf{\eta}}:=\left(% \begin{array}{c} \mathbf{\eta}_1 \\ \vdots \\ \mathbf{\eta}_n \\ \end{array}% \right)\; \underset{(n\times 1)}{\mathbf{\alpha}}:=\left(% \begin{array}{c} \alpha_1 \\ \vdots \\ \alpha_n \\ \end{array}% \right). \end{equation}

Further, let $$ \underset{(Mn\times n)}{\mathbf{D}}:=\mathbf{I}_n\otimes\mathbf{1}_M=\left(% \begin{array}{ccc} \mathbf{1}_M & & \mathbf{O} \\ & \ddots & \\ \mathbf{O}& & \mathbf{1}_M \\ \end{array} \right) $$

Then, the linear panel data model from under an error component assumption in matrix notation is obtained as $$ \mathbf{y}=\mathbf{D}\mathbf{\alpha}+\mathbf{F}\mathbf{\beta}+\mathbf{\eta}, $$ a dummy-variable model.

That is, we can also obtain an estimator of $\mathbf{\beta}$ from an OLS regression on the regressors and $n$ individual specific effects.

Now, note that the Frisch-Waugh-Lovell Theorem says that the OLS estimator of $\mathbf{\beta}$ can be found by regressing $\mathbf{M}_{\mathbf{D}}\mathbf{y}$ on $\mathbf{M}_{\mathbf{D}}\mathbf{F}$, where $$\underset{(Mn\times Mn)}{\mathbf{M}_{\mathbf{D}}}:=\mathbf{I}-\mathbf{D}(\mathbf{D}'\mathbf{D})^{-1}\mathbf{D}'$$ Using symmetry and idempotency of $\mathbf{M}_{\mathbf{D}}$ gives \begin{equation} \widehat{\mathbf{\beta}}_{\text{LSDV}}=(\mathbf{F}'\mathbf{M}_{\mathbf{D}}\mathbf{F})^{-1}\mathbf{F}'\mathbf{M}_{\mathbf{D}}\mathbf{y} \end{equation}

Now, \begin{align*} \mathbf{M}_{\mathbf{D}}&=\mathbf{I}_{Mn}-(\mathbf{I}_n\otimes\mathbf{1}_M)[(\mathbf{I}_n\otimes\mathbf{1}_M)'(\mathbf{I}_n\otimes\mathbf{1}_M)]^{-1}(\mathbf{I}_n\otimes\mathbf{1}_M)'\\ &=\mathbf{I}_{n}\otimes\mathbf{I}_{M}-(\mathbf{I}_n\otimes\mathbf{1}_M)[(\mathbf{I}_n\otimes\mathbf{1}_M')(\mathbf{I}_n\otimes\mathbf{1}_M)]^{-1}(\mathbf{I}_n\otimes\mathbf{1}_M')\\ &=\mathbf{I}_{n}\otimes\mathbf{I}_{M}-(\mathbf{I}_n\otimes\mathbf{1}_M)[\mathbf{I}_n\otimes\mathbf{1}_M'\mathbf{1}_M]^{-1}(\mathbf{I}_n\otimes\mathbf{1}_M')\\ &=\mathbf{I}_{n}\otimes\mathbf{I}_{M}-(\mathbf{I}_n\otimes\mathbf{1}_M)[\mathbf{I}_n\otimes M]^{-1}(\mathbf{I}_n\otimes\mathbf{1}_M')\\ &=\mathbf{I}_{n}\otimes\mathbf{I}_{M}-(\mathbf{I}_n\otimes\mathbf{1}_M)\left[\mathbf{I}_n\otimes \frac{1}{M}\right](\mathbf{I}_n\otimes\mathbf{1}_M')\\ &=\mathbf{I}_{n}\otimes\mathbf{I}_{M}-(\mathbf{I}_n\otimes\mathbf{1}_M)\left[\mathbf{I}_n\otimes \frac{1}{M}\mathbf{1}_M'\right]\\ &=\mathbf{I}_{n}\otimes\mathbf{I}_{M}-\mathbf{I}_n\otimes\mathbf{1}_M\frac{1}{M}\mathbf{1}_M'\\ &=\mathbf{I}_{n}\otimes\left(\mathbf{I}_{M}-\frac{1}{M}\mathbf{1}_M\mathbf{1}_M'\right)\\ &=\mathbf{I}_n\otimes\mathbf{Q} \end{align*}

Thus, \begin{align*} \mathbf{M}_{\mathbf{D}}\mathbf{F}&=(\mathbf{I}_n\otimes\mathbf{Q})\mathbf{F}\\ &=\left(% \begin{array}{ccc} \mathbf{Q} & & \\ & \ddots & \\ & & \mathbf{Q} \\ \end{array} \right)\mathbf{F}\\ &=\tilde{\mathbf{F}}, \end{align*} so that $$\widehat{\mathbf{\beta}}_{\text{LSDV}}=\widehat{\mathbf{\beta}}_{{FE}}.$$

Incidentally, while the notation works with balanced panel data, the result also goes through in the unbalanced case, as one can either check with more complicated notation or this numerical illustration:

library(plm)

# panel dimensions
n <- 10
m <- sample(2:4, n, replace=T) # unbalanced panel

# some data
alpha <- runif(n)
beta <- -2
y <- X <- y.d <- X.d <- c()
D <- matrix(0, sum(m), n) # for the dummy variable matrix
row.counter <- 0
for (i in 1:n) {
  X.n <- runif(m[i],i,i+1)
  X.d <- c(X.d, X.n - mean(X.n))
  X <- c(X,X.n)
  y.n <- alpha[i] + X.n*beta + rnorm(m[i])
  y <- c(y, y.n)
  y.d <- c(y.d, y.n - mean(y.n))
  
  D[(row.counter+1):(row.counter+m[i]), i] <- rep(1, m[i])
  row.counter <- row.counter + m[i]
}

Output:

> # plm
> paneldata <- data.frame(rep(1:n, times=m), unlist(sapply(m, function(i) 1:i)), y, X) # first two columns are for plm to understand the panel .... [TRUNCATED] 

> FE <- plm(y~X, data = paneldata, model = "within")

> # results:
> coef(FE)  # the slope coefficient
        X 
-2.331847 

> fixef(FE) # the intercepts
      1       2       3       4       5       6       7       8       9      10 
0.99396 2.30328 1.90957 2.22670 1.09438 3.10411 2.03265 4.39759 4.42384 4.15294 

> # FWL
> lm(y.d~X.d-1) # just the slope in this formulation

Call:
lm(formula = y.d ~ X.d - 1)

Coefficients:
   X.d  
-2.332  


> # LSDV
> lm(y~D+X-1) # intercepts and slope

Call:
lm(formula = y ~ D + X - 1)

Coefficients:
    D1      D2      D3      D4      D5      D6      D7      D8      D9     D10       X  
 0.994   2.303   1.910   2.227   1.094   3.104   2.033   4.398   4.424   4.153  -2.332