Solved – Ridging a singular covariance matrix (SEM)

structural-equation-modeling

Structural equation models sometimes cannot be fit due to a singular sample covariance matrix. Now some authors suggest to apply a "gentle ridging" to the diagonal which helps (this is for example done in Mplus).

  1. How does ridging exactly work? and
  2. If your preferred SEM software does not support ridging, how can you do it "by hand"?

Best Answer

I can kill two birds with one stone by explaining how you get an ols software to fit ridge regression for a fixed ridging parameter. You simply augment the data set you have with one observation for each beta you intend to ridge but set the covariate pattern to be all zero except for the ridging variable which gets set to the square root of the ridging parameter.

This includes the intercept. So you need to define your own intercept variable as it will have zeros in the data augmented version.

Also you set the data augmented "response" variable to whatever you want the betas to be shrunk towards. Typically these are all zero.

If you do the maths on the augmented data set, you'll get the ridged beta estimate.

You will also find that the augmented x matrix is now full column rank. I can add the maths if you want but I don't think you need it.

Note that this only helps with understanding for fixed value of the ridging parameter. Choosing a good value is another issue.