I have 11 subjects and also have the percentage value for the accuracy of a method I developed to predict future actions for each individual. I want to use a right tail one-sample t-test to find out the minimum accuracy of the method for the population from the limited information I have from 11 subjects. I carry out right tail one-sample t-tests for null hypothesis mean value of 60%,65%,70%,75%,80% (alternate hypothesis is the mean is greater than the mean in null hypothesis) and evaluate for which mean value p become less than 0.01. One of my colleagues told me it is not the right way of doing things as false positives can accumulate and suggested to use Bonferroni correction. In Bonferroni correction the new significance level is 0.01/no. of tests. So my new alpha=0.01/5=0.002. However if I do t-tests for 60%,70%,80% then my new alpha=0.01/3=0.0033. This does not feel right. It looks like I can just select the way I divide my range to satisfy the p value. Is this stupid way of doing things. How will you do it? Will appreciate any help.
Solved – Multiple t-test and Bonferroni correction
bonferronimultiple-comparisonst-test
Best Answer
Your question is quite confusing so let me state it with some mathematical notation.
You have 11 samples from some random variable $X$. You want to test the hypothesis that $\bar{x}>\mu$.
You should formulate your null hypothesis before you start to analyse the data but you haven't, you haven't chosen a specific value of $\mu$. Instead you want to test if $\bar{x}<60\%$ and if the result is not significant you will test if $\bar{x}<65\%$, and so on until you find what value of $\mu$ gives a statistically significant result.
You're not actually testing any hypothesis, you are finding the cut-off value of the percentage so that the test of any hypothesized mean above that would have resulted in a rejecting that hypothesis.
Bonferoni or any other type of multiple-comparison correction is not used for this type of thing. I think it's beyond the scope of this question to explain when multiple comparison corrections are used.