This is a question regarding the binomial test in R.
What happens if I get a p-value < 0.05, which would generally cause me to reject the null hypothesis that the population value is equal to a certain proportion (in this case 0.75), but the hypothesised proportion to be tested (i.e. the null hypothesis) still lies within the 95% confidence interval?
Do I still reject the hypothesis of equal proportions?
Here is an example R code:
x <- 31
n <- 50
binom.test(x, n, p=0.75)
# p-value = 0.04812
# 95 percent confidence interval:
# 0.4717492 0.7534989
Best Answer
The problem with tests of binomial proportion is that the tests used are generally approximate (since the exact "Clopper-Pearson" test is ridiculously conservative). Therefore, it's not clear that the procedure used to get the CI is the same as that used to test the hypothesis. Theoretically, either approach should lead to the same conclusion if you are using just one CI and one test.
You have a border case. Either statistic is telling you that your observation is not all that common under the null hypothesis. Remember: there is nothing special about 5% significance...its a cultural artifact from Ronald Fisher in the 1930's. Its a guideline.
For what it's worth, I'd conclude that its unlikely that the true success probability is as high as 0.75.
Per @John
In a strict hypothesis testing situation, you're stuck with your 0.05, so you would reject the null hypothesis under that criteria. However, I wouldn't run to the press quite yet ;-)...hypothesis tests can really destroy any nuance in inference.