I know only the basics about statistics, and I work in a biological lab where most people know less than I do. I was hoping someone who is a bit more informed about statistics to help me out. I have looked on the net for some help on my problem, and I think I know the answer, but I was looking for the collective wisdom of those knowledgeable in statistics.
I submitted a manuscript to a journal, and we essentially are published based on the reviews, but one reviewer asked a particular question about one our data sets (and obviously wants an answer). I think I have an answer, but am unsure if it's correct.
Let me quickly explain the data: We are measure protein X expression with the following groups:
-Placebo
-Drug 1
-Drug 2
-Drugs 1+ 2
The protein expression is measured via Western blot and is expression as fold change and is normalized to the placebo-treated group (value is 1). We performed these experiments on average N=4-6 separate times.
In the manuscript, I compared the effects of each drug to the placebo using a one-sample t-test because the placebo group contains normalized data and all the values are 1. Using Prism, I set a hypothetical value to 1, and compared drugs 1, 2, and drugs 1+2 to the placebo column using the one-sample t-test, and reported these results in the manuscript.
The question the reviewer asked was why we did t-tests instead of a post-hoc ANOVA. It never occurred to me to do an ANOVA because the placebo groups are all at a value of 1, and thus have no error bars, so I thought the one-sample t-test would be most appropriate using the hypothetical mean of 1. I was under the assumption that to do an ANOVA, the variances had to be similar, and my gut told me that an ANOVA would be rather inaccurate because the placebo groups have no standard error/deviation, thus the effect might be exaggerated and inaccurate. Plus our previous papers have used this before and no one had a beef about that.
Can someone lend their insight into whether or not this makes sense? Was I OK in performed a pairwise one-sample t-test for each group to placebo, or should I have used a post-hoc ANOVA to find where the differences are?
Best Answer
You seem to be at a different stage in the game than "how should I set this up", so I'll provide two answers: one for you, and one for the generic problem.
For you: I'd recommend submitting the other articles you mention as evidence of common practice. It sounds like you're already on the cusp and just need a bit of a push. In my experience, a few citations (and a reasonable logic, as follows for the generic answer) go a long way.
In general: I think your approach of multiple pairwise comparisons is a fine way to compare for what you're looking for. Know that an ANOVA approach would simply yield a "yes/no" answer, which may not be as helpful in your situation as knowing which drugs are different from 1. I'd recommend that you account for multiple comparisons by using your favorite approach (say, Bonferroni), but even if you did do an ANOVA (and even if it was significant) you would still need to follow it up with something to test where the true difference are. Traditional follow up methods tend to rely on the homogeneity of variance assumption, so you would probably be looking at doing multiple single sample t-tests anyway. When you say "compare multiple groups to a control" I immediately think of Dunnett's test (seen here), though I think that for your case it would be inappropriate (due to your desire, not to compare with a control group, but a control number).
All to say, I think you're well supported with your multiple one sample t-tests as, even if you did an ANOVA, you'd likely end up doing them anyway.