Solved – Comparing means with unequal variance and very different sample sizes

meansample-sizesmall-samplestatistical significancet-test

I am trying to compare the means of the same variable between men and women. This is the statistics:

     N        Mean        Variance    Coef. Var.     Gender    
   2000      26.12         10.89         0.13         Male        
     50      56.10         25.01         0.09        Female

Neither variable is normally distributed but taking the log makes it pretty darn close. What is the appropriate way to test the means between males and females? Should I use the log or not? Any additional advice using Stata would be helpful.

My initial reaction is that females fare better than men, but I want to be statistically rigorous.

Best Answer

The traditional test for comparing two sample means is the t-test. There are no assumptions about the sizes of the samples, so it is OK if they are different.

However, you touch upon the normality assumption. Even if the population is not normally distributed, the Central Limit Theorem allows us to infer normality as the sample sizes increase. This means your test will be approximate, but the sample size for female is a little low.

Finally, the result of the t-test will be different for the original and log-ed data. Do you have a specific reason based on your data to use the logarithm? Perhaps there is another assumption you would like to test about the behavior of the log of your data? Do not take the log simply to create a normal curve if there is no deeper meaning, but for fun compare the difference between the two results anyway!

Related Solutions

Solved – Significant difference between difference scores of 2 groups on two measures

To me, your hypothesis sounds like an interaction effect of the two factors "version" (of questionnaire) and "sex" (of participant). As the data are dependent / repeated measures, this should be taken into account by means of a multilevel aka hierachical aka mixed model. Instead of a linear model, you should use something like a logit link (cf. logistic regression) for your binary DV. Suppose your data are in long format, i. e., two rows for every participant, the first for the old version, the second for the new version. The DV is experience of crime 0/1. Data for 3 participants could like this:

     id sex version DV
[1,]  1   1       0  1
[2,]  1   1       1  0
[3,]  2   0       0  1
[4,]  2   0       1  1
[5,]  3   0       0  1
[6,]  3   0       1  0

According to your hypothesis, you predict a cross-level interaction between the level 1-predictor "version" and the level 2-predictor "sex" (in ANOVA / repeated measures terms, this is an interaction between a within- and a between-factor). If you code your variables like this sex: female=1, male=0; version: old=1, new=0, then you probably predict a positive beta of the interaction and you could test this one-sided. Code in R using the package lme4 could look like this (could be done with any multilevel software):

m1 <- lmer(DV ~ version*sex + (1|id), data=mydata, family=binomial(link="logit"))
summary(m1)

HTH

Kolmogorov-Smirnov Test vs T-Test – Comparative Analysis for Distributions

As an example of why you'd want to use the two sample Kolmogorov-Smirnov test:

Imagine that the population means were similar but the variances were very different. The Kolmogorov-Smirnov test could pick this difference up but the t-test cannot.

Or imagine that the distributions have similar means and sd's but the males have a bimodal distribution (red) while the females (blue) don't:

Do males and females perform differently? Yes -- the males tend to either score somewhere around 7.5-8 or 12.5-13, while the females tend more often to score more toward the middle (near 10 or so) but are much less clustered about that value than the two values the males tend to score near to.

So the Kolmogorov-Smirnov can find much more general kinds of difference in distribution than the t-test can.

Best Answer

Related Solutions

Solved – Significant difference between difference scores of 2 groups on two measures

Kolmogorov-Smirnov Test vs T-Test – Comparative Analysis for Distributions

Related Question