I was having a discussion with someone regarding OR’s estimated trough a logistic regression and then he claims that OR’s for continuous variables can only be estimated trough a logistic regression. AFAIK, OR’s are defined only for categorical variables so I was kind of confused. He then proceeds to claim as well that, in the case of a linear regression, the parameter associated with continuous covariates can also be interpreted as a mean differences.
Let us review the case for a general model with a categorical variable. Let us define $Y$ as our dependent variable and a covariate $W$ which has values $A$ and $B$. If we use a reference codification for W, for example $I_{W=B}$ (i.e: if $W=A$ then $I_{W=B}=0$ and if $W=B$ then $I_{W=B}=1$). Then a general model would be
$$g(E(Y|I_{W=B}=x))=\beta_0+\beta_1x$$
for this model it can be shown that $\beta_0=g(E(Y|I_{W=B}=0))=g(E(Y|W=A))$ and $$\beta_1=g(E(Y|I_{W=B}=1))-g(E(Y|I_{W=B}=0))=$$
$$g(E(Y|W=B))-g(E(Y|W=A))$$
if $g$ is the identity we get a linear (normal) model and $\beta_1$ could be interpreted as a mean difference (MD), if $g=logit$ then $\beta_1$ would be the logarithm of an odds ratio.
In the case the covariate is a continuous variable the model would be
$$g(E(Y|X=x))=\beta_0+\beta_1x$$
let us evaluate the model for a given $x$ and $x+1$ and solve it for $\beta_1$:
$$g(E(Y|X=x+1))=\beta_0+\beta_1(x+1)$$
then
$$ g(E(Y|X=x+1)) – g(E(Y|X=x))=\beta_0+\beta_1(x+1) – (\beta_0+\beta_1x)=\beta_1$$
Again for different link functions we would interpret $\beta_1$ as a MD or an OR.
However I have never seen such an interpretation of these parameters in such a way. In most text books (in the common case of identity link) you interpret $\beta_1$ as a rate of change (a derivative), not as a MD. I was arguing that the interpretation of MD or OR for continuous covariates was weird and only valid for increments in one unit in the convariate, breaking the “continuous” nature of the covariate. More over MD and OR make sense and are defined for categorical variables not for continuous ones.
What are your takes on this? Is appropriate to interpret parameters of continuous covariates as if they were effect measures for categorical variables?
Best Answer
Let's focus on the linear regression case, as this question is about the interpretation of regression coefficients, not about the role of the link function.
Mathematically, someone is correct. The regression coefficient $\beta_i$ for a continuous predictor $X_i$ is the difference between two (conditional) means, ie. it's a mean difference.
To be precise, let's assume that $X_i$ doesn't appear in any other regression terms: no interactions, polynomials, splines, etc. For clarity I write the rest of the regression equation as $\boldsymbol{\beta}_{-i}\mathbf{X}_{-i}$ to indicate that $\mathbf{X}_{-i}$ doesn't involve $X_i$.
$$ \begin{aligned} & \operatorname{E}\left\{Y | X_i=x_i+1,\mathbf{X}_{-i}=\mathbf{x}_{-i}\right\} -\operatorname{E}\left\{ Y | X_i=x_i,\mathbf{X}_{-i}=\mathbf{x}_{-i}\right\} \\ &= \Big(\beta_i(x+1)-\beta_ix\Big) +\Big(\boldsymbol{\beta}_{-i}\mathbf{x}_{-i}-\boldsymbol{\beta}_{-i}\mathbf{x}_{-i}\Big) \\ &= \beta_i \end{aligned} $$
So we have two statements about $\beta_i$:
These statements are equivalent as long as we don't interpret the "unit change in $X_i$" causally.
Here is a fictitious example. Say $X_i$ is number of cups of coffee per day and $Y$ is a (totally objective, universal, continuous measure of) happiness. We have some observational data on happiness and we estimate $\hat{\beta}_i$ = 0.5. This means that, on average, people who drink two cups of coffee per day are 0.5 units happier than people who drink one cup of coffee. It doesn't mean that if a person starts drinking one more cup of coffee a day, they well become 0.5 units happier than before.
I think that the "change in Y per unit change in X" language tends to invite a causal interpretation. That's justified only if we are doing causal analysis.
So even though it's wordier the "difference in means" interpretation might be more helpful.
References
Andrew Gelman and colleagues discuss the two interpretations (causal and predictive) of regression coefficients in their textbooks on regression analysis.
[1] Gelman and J. Hill. Data Analysis Using Regression and Multilevel/Hierarchical Models. Cambridge University Press, 2007. See Section 3.2.
[2] Gelman, J. Hill, and A. Vehtari. Regression and Other Stories. Cambridge University Press, 2020. See Section 10.2. It's freely available online.