I have three paired groups, let's call them A, B and C.
I want to check if A's mean is statistically better than B and if A's mean is better than C. I don't care how B and C interact.
I believe I should run paired one sided t tests. But it seems like if I want to have 95% confidence, overall, I can't use 95% for each test.
Am I correct in wanting to use one sided t tests?
If so, What confidence value should I use to compare against?
Secondly suppose I have 5 separate, completely independent frameworks; within each framework, I have paired values for A, B, and C. Can I run the above one sided t tests 5 times (once for each framework)? Or must I do something more complicated.
Best Answer
Since you do only two tests, divide the $\alpha$ level per test in two. That's a Bonferroni correction. You don't need to correct for a third possible test that doesn't interest you. No need to divide by three.
Do two sided tests. One sided tests should only be done if it is logically impossible for the outcome to be on the other side (i.e. you are testing a quantity that can only ever be positive against the null hypothesis that it is 0).
Don't use your intuitions to justify one sided tests. Everybody will think you made the theory up after seeing the direction of the result and you wanted to have more significant results.