I'm trying to use the chi square test goodness of fit to see if heights in a survey are normally distributed. I'm using google sheets to do the calculations. When I look at the charts the distribution looks very "normal" shaped. However I get dramatically different p-values if I do the test with percent frequency vs doing it with regular frequency.
This image below is the test done by comparing the percent frequencies, and the result is an almost 100% p-value meaning it is a good fit.
This image below is the test done by comparing the regular frequencies, and the result is an about 40% p-value, which I think is inconclusive.
My question is what test is correct? My gut likes the percent frequency because the distribution looks very normal. But an example I saw on khan academy used the regular frequency to do the chi-squared test.
Can someone please shed some light on what test is correct and why.
Also a secondary question, this chi-squared test doesn't seem to value in sample size, why not?
Best Answer
Your "percent frequency" calculation is not the correct use of the test. The test statistic is $$\chi^2 = \sum_{i=1}^n \frac{(O_i - E_i)^2}{E_i},$$ where $O_i$ is the number of outcomes observed in category $i$, and $E_i$ is the number of expected outcomes in category $i$ under the assumption of the null hypothesis. If you use the proportion of such outcomes, the statistic becomes $$\chi^2 = N \sum_{i=1}^n \frac{(O_i/N - p_i)^2}{p_i},$$ where $N$ is the total number of observations, and $p_i$ is the probability of observing an outcome in category $i$ under the null hypothesis; that is to say, $p_i = E_i/N$. That is why you are wondering why the sample size doesn't appear in your calculation. It does when you do the calculation with frequencies, because the frequency reflects the amount of data you observed.
In fact, if you add up all the observed counts, you get $N = 123$, and note that $$(123)(0.05825005883) = 7.164757236.$$
One other point: a chi-squared test may be useful for testing deviations from a hypothesized distribution, but it cannot be used to confirm that the observations follow such a distribution. This is because the test statistic is computed under the assumption that the data are in fact drawn from the null distribution; therefore, it cannot confirm what is assumed to be true in the first place. Such a test can only either be inconclusive (insufficient evidence to support the claim that the data do not follow the null distribution), or the data do not follow the null distribution with some possibility of Type I error at most $\alpha$.