I have a small pre/post study w/ a continuous outcome measure. A paired t-test is planned. Is it possible to adjust for potential confounders when using a paired t-test? If not, could I do a linear regression analysis instead, and adjust for potential confounders that way?
Paired t-Test – Adjusting for Potential Confounders
confoundingt-test
Related Solutions
Regression is equivalent to ANCOVA, you are right about that. So, if there is a point to using one there is a point to using the other. The typical format of the output varies, that's all.
For a dependent variable that is ordered, a good starting place is ordinal logistic regression.
Whether to use regression with the initial time point as a covariate or to use something like a multilevel model is another question; it has been discussed here in the past.
I assume you're trying to estimate the causal effect of $x_1$ on $y$, rather than just trying to predict $y$. In general, to find a correct conditioning set (if one exists), you need to know the causal relationships among the variables. Here are some examples to illustrate why:
- You must be sure that the variables you're controlling for are actually confounders. If any one of them is not a confounder but instead a common effect, then you must not control for it. For example, say that $x_1$ and $y$ both influence $x_2$; then controlling for $x_2$ will induce an association between $x_1$ and $y$. This association will bias your estimate of the true effect of $x_1$ on $y$.
- If $x_2$ mediates the relationship between $x_1$ and $y$ – that is, if $x_1$ influences $x_2$, and then $x_2$ influences $y$ – then conditioning on $x_2$ will remove this indirect effect of $x_1$ on $y$ from your estimate. If you are interested in only the direct effect of $x_1$ on $y$, then this is the right approach, but if you want to estimate the total effect, then you should not control for $x_2$.
- In some cases there is no set of variables we can condition on to get an unbiased estimate of the effect. Here is an example: the "M-structure".
In this case, the true effect of $X_1$ on $Y$ is zero. However, there are two unobserved confounders $U_1$ and $U_2$, and one observed confounder $X_2$. If we had observed $U_1$ and $U_2$ we could condition on all three confounders and get an unbiased estimate. However, since we only observed $X_2$ we are stuck. If we don't control for $X_2$ it will confound our estimate of the effet of $X_1$ on $Y$. But if we do control for it, we induce an association between $U_1$ and $U_2$, and therefore between $X_1$ and $Y$.
There are many cases in which you should not control for a particular covariate. If you don't know the causal structure, you may accidentally bias your estimate by controlling for the wrong set of covariates. In this case you can apply a causal structure learning algorithm as a first step, before you try to estimate the causal effect of $x_1$ on $y$.
Best Answer
I'm not sure of all aspects of design, but I can respond to what I do know.
It sounds like you have a single sample which can be stratified by a binary variable. Even in the case where you do not want to adjust for covariates, it would be a good idea to perform an ANCOVA (essentially, a linear regression in which the post scores are regressed onto pre scores and the binary indicator). Let $y$ be the post score, let $x$ be the pre score, and let $z$ be the binary indicator (1 for presence, 0 for absence).
The model would then be $y = \beta_0 + \beta_1x + \beta_2z$. This approach is suggested by Bland & Altman in their paper found here. That paper specifically talks about randomized experiments, but it should also apply here.
Note, adjusting for covariates is now natural, just add them to the regression equation (assuming you don't want a causal interpretation, in which case things become a little harder). The test you seek is the test associated with $\beta_2$. This coefficient will tell you if estimated means are different conditional on having the same pre-test score (or other covariates, should you choose to adjust for them)