For my thesis there's a big chance that I will need some sort of mixed-effects specification. I have some (non-syntax) experience with SPSS but feel that it won't suffice for my analysis. I have very basic knowledge in Stata and decided to experiment more with that package.
I decided to try and replicate results from SPSS in Stata for a basic model. I have data on 4059 students in 65 schools, investigating the influence of entry level score (standlrt) of students on their final exam score (normexam).
In a previously followed course which had a brief introduction to multilevel modeling, my teacher provided me with a syntax in SPSS.
MIXED
normexam WITH standlrt
/FIXED = standlrt
/PRINT = SOLUTION TESTCOV
/RANDOM INTERCEPT standlrt | SUBJECT(school) COVTYPE(VC) .
Now I tried replicating these results in Stata but the results are not consistent. Magnitude and sometimes even sign of the betas differ.
First I use xtset school to indicate that my data is clustered. Then I use
xtmixed normexam standlrt || school: standlrt .
What may be the cause of these inconsistent results?
Thanks in advance!
ps. this is not homework, and I hope I specified my first question properly.
pps. a possibility may be that the 'problem' has multiple optima but I don't think this is the case in such a basic model, also because it's an uni-variate regression. Also, the iterative procedures performed while estimating may have different results, but I only think this would have big effects like sign changes.
EDIT
This is my Stata output
xtmixed normexam standlrt || school: standlrt
Mixed-effects REML regression Number of obs = 4059
Group variable: school Number of groups = 65
Obs per group: min = 2
avg = 62.4
max = 198
Wald chi2(1) = 768.21
Log restricted-likelihood = -4667.8385 Prob > chi2 = 0.0000
------------------------------------------------------------------------------
normexam | Coef. Std. Err. z P>|z| [95% Conf. Interval]
-------------+----------------------------------------------------------------
standlrt | .5570213 .0200971 27.72 0.000 .5176317 .5964108
_cons | -.0080944 .0400842 -0.20 0.840 -.086658 .0704691
------------------------------------------------------------------------------
------------------------------------------------------------------------------
Random-effects Parameters | Estimate Std. Err. [95% Conf. Interval]
-----------------------------+------------------------------------------------
school: Independent |
sd(standlrt) | .1214197 .0191066 .0891958 .1652852
sd(_cons) | .3032317 .0309434 .2482638 .3703699
-----------------------------+------------------------------------------------
sd(Residual) | .7440605 .0083943 .7277885 .7606962
------------------------------------------------------------------------------
LR test vs. linear regression: chi2(2) = 438.60 Prob > chi2 = 0.0000
And this is my SPSS output
-2 Restricted Log Likelihood 9335,677
Type III Tests of Fixed Effects(a)
|---------|------------|--------------|-------|----|
|Source |Numerator df|Denominator df|F |Sig.|
|---------|------------|--------------|-------|----|
|Intercept|1 |60,466 |,041 |,841|
|---------|------------|--------------|-------|----|
|standlrt |1 |56,936 |768,207|,000|
|---------|------------|--------------|-------|----|
a. Dependent Variable: normexam = final exam scores.
Estimates of Fixed Effects(a)
|---------|--------|----------|------|------|----|-----------------------------------|
|Parameter|Estimate|Std. Error|df |t |Sig.|95% Confidence Interval |
| | | | | | |-----------------------|-----------|
| | | | | | |Lower Bound |Upper Bound|
|---------|--------|----------|------|------|----|-----------------------|-----------|
|Intercept|-,008094|,040084 |60,466|-,202 |,841|-,088262 |,072073 |
|---------|--------|----------|------|------|----|-----------------------|-----------|
|standlrt |,557021 |,020097 |56,936|27,717|,000|,516777 |,597266 |
|---------|--------|----------|------|------|----|-----------------------|-----------|
a. Dependent Variable: normexam = final exam scores.
Covariance Parameters
Estimates of Covariance Parameters(a)
|-------------------------------------|--------|----------|------|----|-----------------------------------|
|Parameter |Estimate|Std. Error|Wald Z|Sig.|95% Confidence Interval |
| |--------| | | | |-----------------------|-----------|
| | | | | |Lower Bound |Upper Bound|
|----------------------------|--------|--------|----------|------|----|-----------------------|-----------|
|Residual |,553626 |,012492 |44,319|,000|,529676 |,578659 |
|----------------------------|--------|--------|----------|------|----|-----------------------|-----------|
|Intercept [subject = school]|Variance|,091949 |,018766 |4,900 |,000|,061635 |,137174 |
|----------------------------|--------|--------|----------|------|----|-----------------------|-----------|
|standlrt [subject = school] |Variance|,014743 |,004640 |3,177 |,001|,007956 |,027319 |
|----------------------------|--------|--------|----------|------|----|-----------------------|-----------|
a. Dependent Variable: normexam = final exam scores.
As you can see, the log likelihoods are the same. Additionally, the fixed effects tables are the same. However the random effects are different. I'm not very skilled in interpretation yet but the results seem to differ.
These are the settings for the variance-covariance matrix
Model
covariance(vartype) variance-covariance structure of the random
effects
vartype Description
-------------------------------------------------------------------------
independent one variance parameter per random effect, all
covariances zero; the default unless a factor
variable is specified
exchangeable equal variances for random effects, and one
common pairwise covariance
identity equal variances for random effects, all
covariances zero; the default for factor
variables
unstructured all variances and covariances distinctly
estimated
And I read online that COVTYPE(VC) requests the default (variance component) structure for random effects, which assumes all random effects are independent.
Best Answer
Stata reports the estimated standard deviations of the random effects, whereas SPSS reports variances (this means you are not comparing apples with apples). If you square the results from Stata (or if you take the squared root of the results from SPSS), you will see that they are exactly the same.
For example, squaring the results from Stata: