I have got 2 large samples and want to compare the difference. There are 2 methods:
- Assume the population variances are unknown and unequal, and use the Welch $t$ test.
- Assume the population variances are known and unequal (estimated as sample variances, which is ok since my sample size is million level and the sample mean can be considered as normally distributed), and use the $Z$ test.
I found these 2 tests are pretty much the same formula and same result. So are these 2 tests the same when applied to the real world situation? And if not, what is the difference and when should I use the $Z$ test/welch $t$ test?
Best Answer
Correct, Welch is for unequal variances. $Z$-test is for large-sample approximation when the standard normal distribution approximates the $t$-distribution. For sample sizes above about $n=130$, you will find that $Z$-scores are close to $t$-variates. The $t$-distribution is not needed when there are hundreds of objects (or records), since the standard normal distribution approximates the $t$ distribution very well for large $n$.