I have a simple problem
The sample population is 384 people and $12$% of it have a disease. The hypothesis is that $5$% of the real population have a disease. So, $H_0: \mu = 0.05$ and $H_a: \mu \neq 0.05$
First I calculate the Z-statistic.
$$
z = \frac{\hat{p}-p}{\sqrt{p*(1-p)/384}} = \frac{0.12-0.05}{\sqrt{0.05*(1-0.05)/384}} = 6.3
$$
Using the p-value to test the hypothesis (I used R 2*(1-pnorm(6.3))):
$$
2*Φ(-|6.3|)= 2.976457e^{-10}
$$
which is less than 0.05 level of significance, thus, reject the Null.
Using the confidence interval with 0.05 significance level:
$$
\hat{p} \pm z*\sqrt{\hat{p}(1-\hat{p})/384} = 0.12 \pm 6.3*\sqrt{0.12(1-0.12)/384}
\\ \mbox{I get the following} \\
0.0155 < 0.05 <0.224
$$
So according to the confidence interval I cannot reject the Null. Is it I doing something wrong or is it sometimes the case that there is a disagreement between the two approaches? Granted there are cases in which the the CI and the p-value disagree which one should I go with? Thanks.
Best Answer
With large $N$, your test statistic will be distributed as a normal. Thus, we can refer to your test statistic as "$z$", and we can also use "$z$" to refer to the asymptotic sampling distribution. However, these are not the same $z$'s. When you form your $1-\alpha\%$ confidence interval, you need to multiply the standard error by $z_{1-\alpha/2}$ to get the right increment. You then add (subtract) the product from your observed percentage to get the confidence limits. For example, if you wanted a $95\%$ confidence interval, you would multiply your standard error by $z_{.975} = 1.96$. If you use this value, your confidence interval is:
\begin{align} \hat{p} \pm z_{1-\alpha/2}\times \sqrt{\frac{\hat{p}(1-\hat{p})}{N}} &= 0.12 \pm 1.96\times\sqrt{\frac{0.12(1-0.12)}{384}} \\[10pt] &= 0.0872 < 0.12 < 0.15 \end{align} which agrees with your hypothesis test.