Solved – Linear Mixed Effects Model Variances

correlationcovariancemixed modelvariance

Consider the following model:

\begin{equation}
Y_i = X_i\beta + Z_ib_i + \varepsilon_i,
\end{equation}

where $b_i \sim N(0, D)$, and $\varepsilon_i \sim N(0, R_i(\gamma))$.

The variance of $Y_i$ conditional on $x_i$ is given by

\begin{equation}
\text{var}(Y_i \mid x_i) = Z_iDZ_i^\top + R_i(\gamma).
\end{equation}

The first term represents the contribution to the induced model for the overall covariance pattern due to among-individual sources of variance and correlation, and the second term represents the contribution due to within-individual sources.

What is the difference between among-individual and within-individual sources? Does anyone have some examples?

Best Answer

The terms within-individual variance and among-individual variance are not commonly found in the mixed effects model literature. It more commonly arises in the ANOVA literature, and rather than "among", the usual term is "between". Total variance is partitioned into that which is attributable to differences within individuals, for example the natural variation that occurs in the measurement of blood pressure of a person during a day, and that which is attributable to differences between (or among) individuals. Some people have generally different blood pressure than others, but each person's blood pressure also varies throughout the day.

A repeated measures ANOVA can be formulated as a mixed effects model. In the case of repeated measures within individuals, a random intercepts model will estimate a variance at the individual level, which will be the variance of $b$ in your notation above, while the residual variance is at the measurement level, and is the variance of $\varepsilon$ in your model. The former is between (among) and the latter is within.

Related Solutions

Mixed Model – Understanding Marginal Likelihood of Mixed Effects Models

I can reproduce the marginal log-likelihood returned by lme4:::logLik.merMod by realising that the marginal distribution of Y is multivariate normal (MVN) (as the marginal distribution of b and the conditional of Y are MVN).

So this code will reproduce

library(mvtnorm)
dmvnorm(cake$angle, predict(model, re.form=NA), as.matrix(v), log=TRUE)
#[1] -834.1132

where cake$angle are the observations, predict(model, re.form=NA) are the population predictions (calculated using the fixed effect coefficients), and v is the variance of the marginal model (as shown in question).

A couple of comments on the failing efforts in my question.

When calculating the conditional log-likelihood I used the univariate normal density function, whereas I could/should of used the multivariate. In this case it doesn't make a difference as within unit variance is $\sigma^2 I_n$

mvtnorm::dmvnorm(cake$angle, predict(model), diag(sigma(model)^2,270, 270), log=TRUE)
#[1] -801.6044

Trying to calculate the marginal distribution

First try used the predictions at population (XB) level but the incorrect variance (it ignored the between unit error)
```
sum(dnorm(cake$angle, predict(model, re.form=NA), sd=sigma(model), log=TRUE))
```
Second try was just nonsense I think
Third try uses the univariate rather than multivariate normal distribution.

Best Answer

Related Solutions

Mixed Model – Understanding Marginal Likelihood of Mixed Effects Models

Related Question