Multilevel Model Error – Hessian Matrix Not Positive Definite for Certain Covariance Structures

convergencecovariance-matrixmixed modelmultilevel-analysisrandom-effects-model

I am analyzing data using multilevel modeling / growth curve analysis. For my best fit model, using likelihood ratio tests, I have landed on a random quadratic time interaction model (see syntax below). When I use an unstructured covariance matrix, the model converges normally. When I try to test out alternative covariance matrices (I tried AR1, TP, CS, TPH, CSH), I get a "Hessian matrix not positive definite" error in SPSS. Specifically, it says:

The final Hessian matrix is not positive definite although all convergence criteria are satisfied. The MIXED procedure continues despite this warning. Validity of subsequent results cannot be ascertained.

Here is the syntax that produces the model that converges normally:

MIXED Score with Time  Quad_Time  Condition
/FIXED INTERCEPT  Time  Quad_Time  Condition  Time*Condition Quad_Time*Condition  |  SSTYPE(3)
/RANDOM INTERCEPT Time  Quad_Time  | Subject(PID) COVTYPE (UN)
/PRINT = SOLUTION TESTCOV 
/METHOD = ML.

I would like to test out alternative covariance structures. I read this online: "A common cause of this warning is a model specification that involves redundant covariance parameters. You may need a simpler covariance structure specification in order to avoid this problem. … In other cases, increasing the number of step-halvings allowed or the number of Fisher scoring steps may help to achieve convergence on all criteria, including the Hessian matrix." (source: https://www.ibm.com/support/pages/final-hessian-matrix-not-positive-definite-or-failure-converge-warning)

I increased max step-halvings to 100 and max Fisher scoring steps to 10 and it did not eliminate the error (also, it gave me a message that the convergence was achieved prior to the max number of scoring steps).

I didn't fully understand whether the covariance parameters could be redundant but I wondered if it had to do with random effects for both time and quad time together and/or forcing the covariance parameter estimates to be mathematically related, so I experimented with taking random quad time out, as well as with using heterogeneous versions of covariance structures. Here are the results:

  • UN; Random quad time included = Yes; Convergence = No error
  • AR1, CS, TP; Random quad time included = Yes, Convergence = Error
  • AR1, CS, TP; Random quad time included = No, Convergence = Error
  • CSH, TPH ; Random quad time included = Yes, Convergence = Error
  • ARH1 ; Random quad time included = No, Convergence = No error
  • ARH1, CSH, TPH ; Random quad time included = No, Convergence = No error

Does anyone have any thoughts on what might be happening or how I can probe the data more to figure out which covariance structure to use?

Best Answer

I've done additional research and determined that I do not need to focus on selecting a covariance structure, because an unstructured covariance matrix is highly appropriate for a growth curve model / a linear mixed model with random effects. This is because the effects in the G matrix refer to different constructs (variances of and covariances between random intercept, random slope, random quadratic time). So, unlike in the case of covariances between residuals for repeated measures, we would not expect the G matrix effects to have a particular pattern of relationships.

Related Question