It doesn't make sense to both include tank as a random effect and nest tank within the pop/temp fixed effect. You only need one of these, depending on how tank is coded.
If tank is coded 1-8, you only need the tank random effect. Nesting it within the pop/temp fixed effect results in the same 8 units, so is not necessary.
If tank is coded 1-2 (that is, which rep it was), you only need to nest tank within the pop/temp fixed effect, because that gives you your 8 unique tanks. Including the tank random effect is only desired if the tanks were first divided into two groups and then randomized to treatment; if the eight tanks were completely randomized to treatment, this is not necessary.
You could do this with likelihood based solutions such those in nlme and lme4 but if everything is balanced, it might be simpler to use the traditional ANOVA approach using aov.
Creating some sample data:
set.seed(5)
d <- within(expand.grid(pop=factor(c("A","B")),
temp=factor(c("warm", "cold")),
rep=1:2,
fish=1:100), {
tank <- factor(paste(pop, temp, rep, sep="."))
tanke <- round(rnorm(nlevels(tank))[unclass(tank)],1)
e <- round(rnorm(length(pop)),1)
m <- 10 + 2*as.numeric(pop)*as.numeric(temp)
growth <- m + tanke + e
})
Using aov like this:
a0 <- aov(growth ~ pop*temp + Error(tank), data=d)
summary(a0)
or lme like this:
library(nlme)
m1 <- lme(growth ~ pop*temp, random=~1|tank, data=d)
anova(m1)
In linear random effects models, the additional source of variability due to the random effect results in an additive increase in the total variance. In the beta-binomial model the additional variability is accounted for by a multiplicative overdispersion factor $\phi$. The random effect here is modeled implicitly.
\begin{align}
& y_{ji}|\pi_{j} \sim \textit{Ber}(\pi_{j}) , \quad \pi_j \sim Beta(\mu,\rho) \nonumber \\
& E(\pi_{j}) = \mu \quad \textbf{Var}(\pi_{j}) = \rho\mu(1-\mu)
\end{align}
$\mu$ and $\rho$ are the mean and intra-class correlation and can be reparametrized in terms of $\alpha$, $\beta$ of the Beta distribution.
Together the mean and variance of the marginal proportions $y_{j.}$ look like
\begin{align}
E(y_{j.}) &= m\pi_{j} & \\
\text{Var}(y_{j .}) &= m\mu(1-\mu) + m(m-1)\rho\mu(1-\mu) & \\
& = m\mu(1-\mu)\phi & \nonumber
\end{align}
The effect of assuming that $\pi_j$ is beta distributed is an overdispersion $\phi = (1+(m-1)\rho)$.
If your goal is inference on the mean proportions, method of moments estimates for $\mu$ and $\rho$ are quite effective, see [1]. These are particularly good estimates, if the number of observations per group are balanced.
References
[1] Kleinman, Joel C. "Proportions with extraneous variance: single and independent samples." Journal of the American Statistical Association 68.341 (1973): 46-54.
Best Answer
glmmADMBorglmmTMB(on Github) to fit a Beta model with a random effect of plot