This is a very interesting question. I have been thinking this and related things for a long time.
For me the key to understanding this is to realise that: Random intercepts for a grouping factor are not always sufficient to capture the random variation in the data that is in excess of the residual variation. Because of this, we sometimes see models with random intecepts for interactions between a fixed factor and a grouping variable (and even sometimes random intercepts just for a fixed factor). Generally we advise that a factor can be fixed or random (intercepts) but not both - however there are important exceptions of which your example here is one.
Another hindrence to understanding this is for people like me who come from a multilevel modelling / mixed model background in observational social and medical sciences, we are often caught up in thinking about repeated measures and nesting vs crossed random effects without an understanding that things are a bit different in experimental analysis. More on this a bit later.
Judging by the comments we have both discovered the same thing. In the context of a repeated measures ANOVA, if you want to obtain the same results with lmer then you fit:
y ~ A + B + (1|id) + (1|id:A) + (1|id:B)
where I have discarded factor C without loss of generality.
and the reason why some people specify 1|id/A/B is that they are using nmle:lme and not lme4:lmer. I am not sure why this is needed in lme() but I am fairly sure that to replicate a repeated measures anova - where there is variation for each combination of id and the factors - then you fit the model above in lmer(). Note that (1|id/A/B) seems similar, however it is wrong because it would also fit (1|id:A:B) which is indistinguishable from the residual variance (as noted in your comment also).
It is important to note (and therefore worth repeating) that we only fit this type of model where we have reason to believe that there is variation for each combination of id and the factors. Typically with mixed models we would not do this. We need to understand experimental design. One type of experiment where this is common is the so-called split-plot design where blocking has also been used. This type of experimental design employs randomisation at different "levels" - or rather different combinations of factors, and this is why analysis of such experiemnts often includes random intercept terms that at first glance seem odd. However, the random structure is a property of the experimental design and without knowledge of this, it is virtually impossible to select the correct structure.
So, with regards to the your actual question where the experiment has a repeated factorial design, we can use our friend, simulation, to investigate further.
We will simulate data for the models:
y ~ A + B + (1|id)
and
y ~ A + B + (1|id) + (1|id:A) + (1|id:B)
and look at what happens when we use both models to analyse both datasets.
set.seed(15)
n <- 100 # number of subjects
K <- 4 # number of measurements per subject
# set up covariates
df <- data.frame(id = rep(seq_len(n), each = K),
A = rep(c(0,0,1,1), times = n),
B = rep(c(0,1), times = n * 2)
)
#
df$y <- df$A + 2 * df$B + 3 * df$id + rnorm(n * K)
m0 <- lmer(y ~ A + B + (1|id) , data = df)
m1 <- lmer(y ~ A + B + (1|id) + (1|id:A) + (1|id:B), data = df)
summary(m0)
m0 is the "right" model for these data because we simply created y with fixed effects for id (which we capture with random intercepts) and unit variance. This is a bit of an abuse but it is convenient and does what we want:
Groups Name Variance Std.Dev.
id (Intercept) 842.1869 29.0205
Residual 0.9946 0.9973
Number of obs: 400, groups: id, 100
Fixed effects:
Estimate Std. Error t value
(Intercept) 50.47508 2.90333 17.39
A 1.01277 0.09973 10.15
B 2.06675 0.09973 20.72
as we can see, we recover unit variance in the residual and good estimates for the fixed effects. However:
> summary(m1)
Random effects:
Groups Name Variance Std.Dev.
id:B (Intercept) 0.000e+00 0.0000
id:A (Intercept) 8.724e-03 0.0934
id (Intercept) 8.422e+02 29.0204
Residual 9.888e-01 0.9944
Number of obs: 400, groups: id:B, 200; id:A, 200; id, 100
Fixed effects:
Estimate Std. Error t value
(Intercept) 50.47508 2.90334 17.39
A 1.01277 0.10031 10.10
B 2.06675 0.09944 20.78
This is a singular fit - zero estimates for the variance of the id:B term and close to zero for id:A - which we happen to know is correct here because we didn't simulate any variance for those "levels". Also we find:
> anova(m0, m1)
m0: y ~ A + B + (1 | id)
m1: y ~ A + B + (1 | id) + (1 | id:A) + (1 | id:B)
npar AIC BIC logLik deviance Chisq Df Pr(>Chisq)
m0 5 1952.8 1972.7 -971.39 1942.8
m1 7 1956.8 1984.7 -971.39 1942.8 0.0052 2 0.9974
meaning that we strongly prefer the (correct) model m0
So now we simulate data with variation at these "levels" too. Since m1 is the model we want to simlulate for, we an use it's design matrix for the random effects:
# design matrix for the random effects
Z <- as.matrix(getME(m1, "Z"))
# design matrix for the fixed effects
X <- model.matrix(~ A + B, data = df)
betas <- c(10, 2, 3) # fixed effects coefficients
D1 <- 1 # SD of random intercepts for id
D2 <- 2 # SD of random intercepts for id:A
D3 <- 3 # SD of random intercepts for id:B
# we simulate random effects
b <- c(rnorm(n*2, sd = D3), rnorm(n*2, sd = D2), rnorm(n, sd = D1))
# the order here is goverened by the order that lme4 creates the Z matrix
# linear predictor
lp <- X %*% betas + Z %*% b
# add residual variance of 1
df$y <- lp + rnorm(n * K)
m2 <- lmer(y ~ A + B + (1|id) + (1|id:A) + (1|id:B), data = df)
m3 <- lmer(y ~ A + B + (1|id), data = df)
summary(m2)
'm2` is the corect model here and we obtain:
Random effects:
Groups Name Variance Std.Dev.
id:B (Intercept) 6.9061 2.6279
id:A (Intercept) 4.4766 2.1158
id (Intercept) 2.9117 1.7064
Residual 0.8704 0.9329
Number of obs: 400, groups: id:B, 200; id:A, 200; id, 100
Fixed effects:
Estimate Std. Error t value
(Intercept) 10.3870 0.3866 26.867
A 1.8123 0.3134 5.782
B 3.0242 0.3832 7.892
the SD for the id intercept is a little high, but otherwise we have good estimates for the random and fixed effects. On the other hand:
> summary(m3)
Random effects:
Groups Name Variance Std.Dev.
id (Intercept) 6.712 2.591
Residual 8.433 2.904
Number of obs: 400, groups: id, 100
Fixed effects:
Estimate Std. Error t value
(Intercept) 10.3870 0.3611 28.767
A 1.8123 0.2904 6.241
B 3.0242 0.2904 10.414
although the fixed effects point estimates are OK, their standard errors are larger. The random structure is, of course, completely wrong. And finally:
> anova(m2, m3)
Models:
m3: y ~ A + B + (1 | id)
m2: y ~ A + B + (1 | id) + (1 | id:A) + (1 | id:B)
npar AIC BIC logLik deviance Chisq Df Pr(>Chisq)
m3 5 2138.1 2158.1 -1064.07 2128.1
m2 7 1985.7 2013.7 -985.87 1971.7 156.4 2 < 2.2e-16 ***
showing that we strongly prefer m2
Best Answer
Update: The Federov algorithm implemented in AlgDesign optFederov lets you create nearly orthogonal designs for mixed factors, as shown in the documentation
The example in the post below is a non-orthogonal fractional factorial design https://stackoverflow.com/questions/5044876/how-to-create-a-fractional-factorial-design-in-r