I am new to these kind of statistics and I don't understand the error I get since the same code worked before on an other set of data (with different levels in the factors).
Here is my design. I have 3 factors: Substrate (4 levels); Origin (5 levels); and Genotype (18 levels). Genotype is nested in Origin.
I want to know if there is an effect of Origin on the measured variable (shannon) OR an effect of Genotype. I also need to know if there is an interaction between Substrate and Origin (or Genotype).
Here is my data.
I used gls because I need to include weights to take into acount the overal effect of Substrate on the variance of the data. Here is my model and the error it gives:
mod <- gls(shannon ~ Substrate:Origin + Substrate + Origin,
weights = varIdent(form = ~ 1|Substrate),
correlation = corCompSymm(form = ~ Genotype|Origin),
data = my_data,
na.action = "na.omit")
Error in glsEstimate(object, control = control) :
computed "gls" fit is singular, rank 18
When I remove the interaction term the model runs properly, however, as I said, I realy need this term.
When searching for this error, I found that it may be because of overfitting. Is there another way I could calculate the effect of the interaction term?
NOTE: There are only two missing values and the dataset meets the parametric assumptions. Also, there are 3 reps for every combination of Genotype:Substrate (except for the two missing values).
EDIT
I can ignore the singularity to produce a summary and make an anova.
mod <- gls(shannon ~ Substrate:Origin + Substrate + Origin,
weights = varIdent(form = ~ 1|Substrate),
correlation = corCompSymm(form = ~ Genotype|Origin),
data = my_data,
method = "REML",
na.action = "na.omit",
control = list(singular.ok = TRUE))
Here is the summary
Generalized least squares fit by REML
Model: shannon ~ Substrate:Origin + Substrate + Origin
Data: my_data
AIC BIC logLik
325.4394 402.3188 -137.7197
Correlation Structure: Compound symmetry
Formula: ~Genotype | Origin
Parameter estimate(s):
Rho
3.397167e-18
Variance function:
Structure: Different standard deviations per stratum
Formula: ~1 | Substrate
Parameter estimates:
NA.Waste.Rock Control A.Waste.Rock Tailings
1 1 1 1
Coefficients:
Value Std.Error t-value p-value
(Intercept) 6.601569 0.1684757 39.18409 0.0000
SubstrateControl -0.136807 0.2382607 -0.57419 0.5666
SubstrateNA.Waste.Rock -0.411788 0.2455935 -1.67671 0.0956
SubstrateTailings 0.319328 0.2382607 1.34025 0.1821
OriginEastern -0.230572 0.2455935 -0.93884 0.3492
OriginLa.Corne.Mine -0.108815 0.2228725 -0.48824 0.6260
OriginWestern -0.081117 0.2131068 -0.38064 0.7040
OriginWestwood -0.073239 0.2382607 -0.30739 0.7589
SubstrateControl:OriginEastern -0.263073 0.3421759 -0.76882 0.4431
SubstrateNA.Waste.Rock:OriginEastern 0.304783 0.3473217 0.87752 0.3815
SubstrateControl:OriginLa.Corne.Mine -0.072767 0.3151893 -0.23087 0.8177
SubstrateNA.Waste.Rock:OriginLa.Corne.Mine 0.020410 0.3207683 0.06363 0.9493
SubstrateTailings:OriginLa.Corne.Mine 0.077623 0.3151893 0.24628 0.8058
SubstrateControl:OriginWestern -0.320295 0.3033900 -1.05572 0.2927
SubstrateNA.Waste.Rock:OriginWestern -0.030207 0.3072085 -0.09833 0.9218
SubstrateControl:OriginWestwood -0.187284 0.3369515 -0.55582 0.5791
SubstrateNA.Waste.Rock:OriginWestwood -0.047583 0.3421759 -0.13906 0.8896
Correlation:
(Intr) SbstrC SbNA.W.R SbstrT OrgnEs
SubstrateControl -0.707
SubstrateNA.Waste.Rock -0.686 0.485
SubstrateTailings 0.000 0.000 0.000
OriginEastern -0.686 0.485 0.471 0.000
OriginLa.Corne.Mine -0.756 0.535 0.519 0.000 0.519
OriginWestern -0.791 0.559 0.542 0.000 0.542
OriginWestwood -0.707 0.500 0.485 -0.500 0.485
SubstrateControl:OriginEastern 0.492 -0.696 -0.338 0.000 -0.718
SubstrateNA.Waste.Rock:OriginEastern 0.485 -0.343 -0.707 0.000 -0.707
SubstrateControl:OriginLa.Corne.Mine 0.535 -0.756 -0.367 0.000 -0.367
SubstrateNA.Waste.Rock:OriginLa.Corne.Mine 0.525 -0.371 -0.766 0.000 -0.360
SubstrateTailings:OriginLa.Corne.Mine 0.000 0.000 0.000 -0.756 0.000
SubstrateControl:OriginWestern 0.555 -0.785 -0.381 0.000 -0.381
SubstrateNA.Waste.Rock:OriginWestern 0.548 -0.388 -0.799 0.000 -0.376
SubstrateControl:OriginWestwood 0.500 -0.707 -0.343 0.354 -0.343
SubstrateNA.Waste.Rock:OriginWestwood 0.492 -0.348 -0.718 0.348 -0.338
OL.C.M OrgnWstr OrgnWstw SbC:OE
SubstrateControl
SubstrateNA.Waste.Rock
SubstrateTailings
OriginEastern
OriginLa.Corne.Mine
OriginWestern 0.598
OriginWestwood 0.535 0.559
SubstrateControl:OriginEastern -0.372 -0.389 -0.348
SubstrateNA.Waste.Rock:OriginEastern -0.367 -0.383 -0.343 0.508
SubstrateControl:OriginLa.Corne.Mine -0.707 -0.423 -0.378 0.526
SubstrateNA.Waste.Rock:OriginLa.Corne.Mine -0.695 -0.415 -0.371 0.259
SubstrateTailings:OriginLa.Corne.Mine -0.303 0.000 0.378 0.000
SubstrateControl:OriginWestern -0.420 -0.702 -0.393 0.547
SubstrateNA.Waste.Rock:OriginWestern -0.415 -0.694 -0.388 0.270
SubstrateControl:OriginWestwood -0.378 -0.395 -0.707 0.492
SubstrateNA.Waste.Rock:OriginWestwood -0.372 -0.389 -0.696 0.242
SNA.W.R:OE SC:OL. SNA.W.R:OL ST:OL.
SubstrateControl
SubstrateNA.Waste.Rock
SubstrateTailings
OriginEastern
OriginLa.Corne.Mine
OriginWestern
OriginWestwood
SubstrateControl:OriginEastern
SubstrateNA.Waste.Rock:OriginEastern
SubstrateControl:OriginLa.Corne.Mine 0.259
SubstrateNA.Waste.Rock:OriginLa.Corne.Mine 0.541 0.491
SubstrateTailings:OriginLa.Corne.Mine 0.000 0.214 0.211
SubstrateControl:OriginWestern 0.269 0.594 0.292 0.000
SubstrateNA.Waste.Rock:OriginWestern 0.565 0.293 0.612 0.000
SubstrateControl:OriginWestwood 0.243 0.535 0.263 -0.267
SubstrateNA.Waste.Rock:OriginWestwood 0.508 0.263 0.550 -0.263
SbstrtCntrl:OrgnWstr
SubstrateControl
SubstrateNA.Waste.Rock
SubstrateTailings
OriginEastern
OriginLa.Corne.Mine
OriginWestern
OriginWestwood
SubstrateControl:OriginEastern
SubstrateNA.Waste.Rock:OriginEastern
SubstrateControl:OriginLa.Corne.Mine
SubstrateNA.Waste.Rock:OriginLa.Corne.Mine
SubstrateTailings:OriginLa.Corne.Mine
SubstrateControl:OriginWestern
SubstrateNA.Waste.Rock:OriginWestern 0.487
SubstrateControl:OriginWestwood 0.555
SubstrateNA.Waste.Rock:OriginWestwood 0.273
SbstrtNA.Wst.Rck:OrgnWstr
SubstrateControl
SubstrateNA.Waste.Rock
SubstrateTailings
OriginEastern
OriginLa.Corne.Mine
OriginWestern
OriginWestwood
SubstrateControl:OriginEastern
SubstrateNA.Waste.Rock:OriginEastern
SubstrateControl:OriginLa.Corne.Mine
SubstrateNA.Waste.Rock:OriginLa.Corne.Mine
SubstrateTailings:OriginLa.Corne.Mine
SubstrateControl:OriginWestern
SubstrateNA.Waste.Rock:OriginWestern
SubstrateControl:OriginWestwood 0.274
SubstrateNA.Waste.Rock:OriginWestwood 0.574
SbstrtCntrl:OrgnWstw
SubstrateControl
SubstrateNA.Waste.Rock
SubstrateTailings
OriginEastern
OriginLa.Corne.Mine
OriginWestern
OriginWestwood
SubstrateControl:OriginEastern
SubstrateNA.Waste.Rock:OriginEastern
SubstrateControl:OriginLa.Corne.Mine
SubstrateNA.Waste.Rock:OriginLa.Corne.Mine
SubstrateTailings:OriginLa.Corne.Mine
SubstrateControl:OriginWestern
SubstrateNA.Waste.Rock:OriginWestern
SubstrateControl:OriginWestwood
SubstrateNA.Waste.Rock:OriginWestwood 0.492
Standardized residuals:
Min Q1 Med Q3 Max
-2.40142528 -0.62227095 0.07345434 0.58031803 2.11407668
Residual standard error: 0.5054272
Degrees of freedom: 180 total; 160 residual
Here is the anova
Denom. DF: 160
numDF F-value p-value
(Intercept) 1 28368.157 <.0001
Substrate 3 14.268 <.0001
Origin 4 0.816 0.5168
Substrate:Origin 12 NA NA
Warning message:
In print.anova.lme(x) : NAs introduced by coercion
SECOND EDIT
Here is the model with lme() instead of gls() and the error. I don't know how to bypass that error.
mod <- lme(shannon ~ Substrate:Origin,
random = ~ 1 | Origin/Genotype,
weights = varIdent(form = ~ 1|Substrate),
data = my_data,
na.action = "na.omit")
Error in MEEM(object, conLin, control$niterEM) :
Singularity in backsolve at level 0, block 1
THIRD EDIT
Here is the result for the model with gls() whitout the correlation.
Note : I get the same error and need to bypass it with control = list(singular.ok = TRUE).
Generalized least squares fit by REML
Model: shannon ~ Substrate:Origin + Substrate + Origin
Data: my_data
AIC BIC logLik
323.4394 397.2436 -137.7197
Variance function:
Structure: Different standard deviations per stratum
Formula: ~1 | Substrate
Parameter estimates:
NA.Waste.Rock A.Waste.Rock Control Tailings
1 1 1 1
Coefficients:
Value Std.Error t-value p-value
(Intercept) 6.601569 0.1684757 39.18409 0.0000
SubstrateControl -0.136807 0.2382607 -0.57419 0.5666
SubstrateNA.Waste.Rock -0.411788 0.2455935 -1.67671 0.0956
SubstrateTailings 0.319328 0.2382607 1.34025 0.1821
OriginEastern -0.230572 0.2455935 -0.93884 0.3492
OriginLa.Corne.Mine -0.108815 0.2228725 -0.48824 0.6260
OriginWestern -0.081117 0.2131068 -0.38064 0.7040
OriginWestwood -0.073239 0.2382607 -0.30739 0.7589
SubstrateControl:OriginEastern -0.263073 0.3421759 -0.76882 0.4431
SubstrateNA.Waste.Rock:OriginEastern 0.304783 0.3473217 0.87752 0.3815
SubstrateControl:OriginLa.Corne.Mine -0.072767 0.3151893 -0.23087 0.8177
SubstrateNA.Waste.Rock:OriginLa.Corne.Mine 0.020410 0.3207683 0.06363 0.9493
SubstrateTailings:OriginLa.Corne.Mine 0.077623 0.3151893 0.24628 0.8058
SubstrateControl:OriginWestern -0.320295 0.3033900 -1.05572 0.2927
SubstrateNA.Waste.Rock:OriginWestern -0.030207 0.3072085 -0.09833 0.9218
SubstrateControl:OriginWestwood -0.187284 0.3369515 -0.55582 0.5791
SubstrateNA.Waste.Rock:OriginWestwood -0.047583 0.3421759 -0.13906 0.8896
Correlation:
(Intr) SbstrC SbNA.W.R SbstrT OrgnEs
SubstrateControl -0.707
SubstrateNA.Waste.Rock -0.686 0.485
SubstrateTailings 0.000 0.000 0.000
OriginEastern -0.686 0.485 0.471 0.000
OriginLa.Corne.Mine -0.756 0.535 0.519 0.000 0.519
OriginWestern -0.791 0.559 0.542 0.000 0.542
OriginWestwood -0.707 0.500 0.485 -0.500 0.485
SubstrateControl:OriginEastern 0.492 -0.696 -0.338 0.000 -0.718
SubstrateNA.Waste.Rock:OriginEastern 0.485 -0.343 -0.707 0.000 -0.707
SubstrateControl:OriginLa.Corne.Mine 0.535 -0.756 -0.367 0.000 -0.367
SubstrateNA.Waste.Rock:OriginLa.Corne.Mine 0.525 -0.371 -0.766 0.000 -0.360
SubstrateTailings:OriginLa.Corne.Mine 0.000 0.000 0.000 -0.756 0.000
SubstrateControl:OriginWestern 0.555 -0.785 -0.381 0.000 -0.381
SubstrateNA.Waste.Rock:OriginWestern 0.548 -0.388 -0.799 0.000 -0.376
SubstrateControl:OriginWestwood 0.500 -0.707 -0.343 0.354 -0.343
SubstrateNA.Waste.Rock:OriginWestwood 0.492 -0.348 -0.718 0.348 -0.338
OL.C.M OrgnWstr OrgnWstw SbC:OE
SubstrateControl
SubstrateNA.Waste.Rock
SubstrateTailings
OriginEastern
OriginLa.Corne.Mine
OriginWestern 0.598
OriginWestwood 0.535 0.559
SubstrateControl:OriginEastern -0.372 -0.389 -0.348
SubstrateNA.Waste.Rock:OriginEastern -0.367 -0.383 -0.343 0.508
SubstrateControl:OriginLa.Corne.Mine -0.707 -0.423 -0.378 0.526
SubstrateNA.Waste.Rock:OriginLa.Corne.Mine -0.695 -0.415 -0.371 0.259
SubstrateTailings:OriginLa.Corne.Mine -0.303 0.000 0.378 0.000
SubstrateControl:OriginWestern -0.420 -0.702 -0.393 0.547
SubstrateNA.Waste.Rock:OriginWestern -0.415 -0.694 -0.388 0.270
SubstrateControl:OriginWestwood -0.378 -0.395 -0.707 0.492
SubstrateNA.Waste.Rock:OriginWestwood -0.372 -0.389 -0.696 0.242
SNA.W.R:OE SC:OL. SNA.W.R:OL ST:OL.
SubstrateControl
SubstrateNA.Waste.Rock
SubstrateTailings
OriginEastern
OriginLa.Corne.Mine
OriginWestern
OriginWestwood
SubstrateControl:OriginEastern
SubstrateNA.Waste.Rock:OriginEastern
SubstrateControl:OriginLa.Corne.Mine 0.259
SubstrateNA.Waste.Rock:OriginLa.Corne.Mine 0.541 0.491
SubstrateTailings:OriginLa.Corne.Mine 0.000 0.214 0.211
SubstrateControl:OriginWestern 0.269 0.594 0.292 0.000
SubstrateNA.Waste.Rock:OriginWestern 0.565 0.293 0.612 0.000
SubstrateControl:OriginWestwood 0.243 0.535 0.263 -0.267
SubstrateNA.Waste.Rock:OriginWestwood 0.508 0.263 0.550 -0.263
SbstrtCntrl:OrgnWstr
SubstrateControl
SubstrateNA.Waste.Rock
SubstrateTailings
OriginEastern
OriginLa.Corne.Mine
OriginWestern
OriginWestwood
SubstrateControl:OriginEastern
SubstrateNA.Waste.Rock:OriginEastern
SubstrateControl:OriginLa.Corne.Mine
SubstrateNA.Waste.Rock:OriginLa.Corne.Mine
SubstrateTailings:OriginLa.Corne.Mine
SubstrateControl:OriginWestern
SubstrateNA.Waste.Rock:OriginWestern 0.487
SubstrateControl:OriginWestwood 0.555
SubstrateNA.Waste.Rock:OriginWestwood 0.273
SbstrtNA.Wst.Rck:OrgnWstr
SubstrateControl
SubstrateNA.Waste.Rock
SubstrateTailings
OriginEastern
OriginLa.Corne.Mine
OriginWestern
OriginWestwood
SubstrateControl:OriginEastern
SubstrateNA.Waste.Rock:OriginEastern
SubstrateControl:OriginLa.Corne.Mine
SubstrateNA.Waste.Rock:OriginLa.Corne.Mine
SubstrateTailings:OriginLa.Corne.Mine
SubstrateControl:OriginWestern
SubstrateNA.Waste.Rock:OriginWestern
SubstrateControl:OriginWestwood 0.274
SubstrateNA.Waste.Rock:OriginWestwood 0.574
SbstrtCntrl:OrgnWstw
SubstrateControl
SubstrateNA.Waste.Rock
SubstrateTailings
OriginEastern
OriginLa.Corne.Mine
OriginWestern
OriginWestwood
SubstrateControl:OriginEastern
SubstrateNA.Waste.Rock:OriginEastern
SubstrateControl:OriginLa.Corne.Mine
SubstrateNA.Waste.Rock:OriginLa.Corne.Mine
SubstrateTailings:OriginLa.Corne.Mine
SubstrateControl:OriginWestern
SubstrateNA.Waste.Rock:OriginWestern
SubstrateControl:OriginWestwood
SubstrateNA.Waste.Rock:OriginWestwood 0.492
Standardized residuals:
Min Q1 Med Q3 Max
-2.40142528 -0.62227095 0.07345434 0.58031803 2.11407668
Residual standard error: 0.5054272
Degrees of freedom: 180 total; 160 residual
```
Best Answer
The main issue in this case is that the experiment was factorial in nature but due to some unobserved combinations of one level of one of the variables with another, the design matrix for the fixed effects was rank deficient. ie it was an incomplete