For the repeated measures design D.C.Montgomery in his "Design and Analysis of Experiments" book provides the mathematical / statistical (linear) model (with changes): $y_{ij} = \mu + \beta_i + \tau_j + \epsilon_{ij}$, where $\beta$ is a parameter associated with the $i$th subject, $\tau$ is the effect of the $j$th treatment, and $\epsilon$ is the independent error.
In Weixing Song's document it is stated that to check the model's assumptions we should operate with residuals calculated as (A): $$\hat\epsilon_{ij} = Y_{ij} – \bar Y_{i.} – \bar Y_{.j} + \bar Y.$$
The formula (B) on Wikipedia is $$\hat\epsilon_i = Y_i – \bar Y.$$
My question is:
what formula is used to correctly calculate residuals in the model describing a single-factor repeated measures design to check its assumptions (multivariate normality, homoscedasticity, sphericity, linearity, additivity)?
Thank you.
P.S.
E.Vonesh and V.M.Chinchilli in their "Linear and Nonlinear Models for the Analysis of Repeated Measurements" monograph state that the model for single-sample RM design may be written as $y_{i} = \mu + \epsilon_{i}$.
Best Answer
If I understand the notation correctly here, $\bar Y$ is the overall mean. So $Y_{i} - \bar Y$ will give you the difference between an individual's score and the overall mean but does not relate at all to the treatment effect. So no, you can't use formula B to estimate residuals from the model you have specified.
On the other hand, formula B would estimate residuals from a simpler model - one with no treatment effect in it - if you are interested in the random component of such a model. As a lot of inference is done with this simpler model as the null hypothesis, there is an argument for paying considerable attention to the distribution of the residuals of this simpler model eg checking for non-breach of assumptions of normality, constant variance, etc (if your model has such assumptions - they are usual but you have not made them explicit here).