My goal is to run a panel-regression in order to asses the impact of county-level average household income on test-scores across a wide array of undergraduate-level university courses.
Specifically, I have observations on test-scores in course $j$ for student $i$ in semester $t$ where $$j\in\{\text{calculus, linear-algebra, advanced-calculus, … , comparative linguistics, linguistic typology, … }\}$$
and I can partition the set of courses by subject, i.e. define a mapping $s$ such that e.g.
$\,s(\text{calculus})=\text{math}$.
Currently, I'd go about this exercise by estimating
$$grade_{j,i,t}=\gamma_{s(j)}+\gamma_i+\gamma_t+\beta \cdot income_{c(i),t}+\delta x_{i,t}+\epsilon_{j,i,t}$$
where
- $\gamma_{s(j)}$ is a subject fixed effect in order to take care of the fact that grading scales tend to vary by subject (but very little within subject)
- $\gamma_i$ is a student fixed effect meant to capture unobserved (overall) intellectual ability, propensity to study, etc.
- $\gamma_t$ is a time fixed effect.
- I also control for parental income as well as $x_{i,t}$ – a set of demographics including parental income, employment status of parents, household size, race, etc.
As I am under the impression that the term fixed-effects has very different meanings in different disciplines, my usage of the term is meant to align with its usage in econometrics.
The coefficient of interest is $\beta$ where $c(i)$ denotes the county where student $i$ resides (my question is conceptual, so it does not really matter that in terms of a narrative the income of the county where a student grew up would presumably be more relevant).
It is my (perhaps erroneous) understanding that, in a simpler model $y_{it}=\alpha_i+\beta x_{it}+\epsilon_{it}$, by including a unit-fixed-effect I am effectively using the time-variation in $x_{it}$ to estimate $\beta$ so that $\beta$ is the impact of $x_{it}$ on $y_{it}$ within an $i$-cell (or rather the average of the impact within each $i$-unit taken over $i$-units?).
If I were to estimate $y_{it}=\alpha_t+\beta x_{it}+\epsilon_{it}$, I'd on the other hand use the cross-sectional variation so that $\beta$ is the impact of $x_{it}$ on $y_{it}$ at a given time.
So in my setting, I am wondering how to interpret $\beta$ exactly. I would like $\beta$ to isolate the cross-sectional impact of county-income on test-scores.
Questions:
Does my inclusion of $\gamma_{s(j)}$ alter anything regarding the interpretation of $\beta$? I am not sure how to think about the variance reduction of the independent variable here, since it is not indexed at $j$.
I feel tempted to drop $\gamma_i$, because I want the variation that drives the estimation of $\beta$ to be chiefly cross-sectional variation in county income. Including $\gamma_i$ means that the variation in county-income to be exploited is in the time-dimension, right? But I also cannot be ignorant of the fact that there is something like (subject- and time-invariant) student-specific ability that drives grades. Is there a solution to this trade-off?
Finally, I include $\gamma_t$ in order to force the cross-sectional variation in county-income to be central to the estimate of $\beta$. But, now, since I have both $i$ and $t$ fixed effects, what is the exact variation in independent variable indexed at $(i,t)$ that I am using? What is the exact interpretation of $\beta$?
Thanks a lot!
Best Answer
There are a couple of things that need to be addressed before getting into the $\gamma_{s(j)}$ issue, just to clear up some features of FEs. You mention that this is conceptual, so I don't want to assume you thought through every detail, necessarily, but there is something more fundamental that should be considered before looking at $\gamma_{s(j)}$. A really important part of this model is $\gamma_i$ (student FE). Including this means all time-invariant factors are already "controlled for" that are related to the students. Whatever time-invariant differences (average level of grade over the panel) between students that do exist are eliminated (students are now on the same expected overall "level"). Variables like race can't be in a model with $\gamma_i$. Race might be expect to generate different overall "levels" of grade, but it doesn't change within-person over the panel. The time-invariant "level" that might be captured by race is already captured by $\gamma_i$, and race would drop from the model due to collinearity. Just like race, county is also "controlled for" by $\gamma_i$, unless individuals are moving between counties (another can of worms). This is important, because you say:
However, the cross-sectional effect of income "level" on grades among counties is already absorbed by $\gamma_i$. If we set aside class-subject for the moment, the interpretation of $\beta$ would be the expected change in a student's grade associated with a one-unit increase in county-level income (as a deviation from the mean county-level income for each student over the panel). $\gamma_i$ changes all interpretations to within-student effects. If between-student is absorbed and students stay in their counties, between-county is also absorbed in the student FE.
Each student has many subjects during the panel, though, so that is NOT already absorbed by $\gamma_i$. Since we know that subjects have systematic variation in average "levels" or grading scales among all students, we know that a change from one subject to another will generate an expected change in their grade that is not related to $income_{c(i),t}$. Like we did with students (and thus, race, county etc), we can make all subjects "equal" in their time-invariant attributes by absorbing the effect of each subject on the "level" of grades in $\gamma_{s(j)}$. Without this, we don't know if the change came from income or from moving between class-subjects. Thus, $\gamma_{s(j)}$ is important to include and doesn't interfere with the cross-sectional effect of income, because that was already absorbed by $\gamma_i$.
After including $\gamma_i$ and $\gamma_{s(j)}$ (and accounting changes that affect all students in a semester with $\gamma_t$), the interpretation of $\beta$ is the expected increase in grades caused by a one-unit increase in $income_{c(i),t}$ after accounting for time-invariant differences in students (including their counties) and class-subjects (and overall trends). This model won't tell you about cross-sectional effects of county income.
If you are interested in a mix of the within- and between(cross-sectional)-effect of county income, you can use a random effect for student, but you won't be able to say how much effect comes from variation within- versus between-students (and thus between counties). This is a fundamental issue in research with observational data. Another option is the "hybrid model" that isolates within and between effects. This R package gives a nice explanation of that setup.