What statistical test can I use to compare two ratios from two independent samples. The ratios are after to before results. I need to compare the after/before ratios for two independent models and show whether they are have significant difference or not. Please help!
Solved – Statistical test to compare two ratios from two independent models
statistical significance
Related Solutions
Is your null hypothesis that the methods are different, or is your null hypothesis that a given dataset is drawn from your mixture? They are two different things, one demands that you test each separately against, e.g., 1/4, 1/4, 1/4, 1/4. The other ignores the fact that you know what the truth is and just asks do the two sequencing platforms give different results.
For the first, you don't need to do any normalization. Just use chi-squared goodness of fit test directly. You compare the counts to the known probabilities, e.g., 1/4,1/4,1/4,1/4. Then to compare the methods, you can ask which statistic is smaller (here you would want to normalize because the p-value is going to be count dependent).
One way to do that cheap and dirty is to just subsample an equal number of counts from each platform and then do the above comparison, repeat 100 times, and now you have two distributions of p-values representing the goodness of fit test stats. If one platform is better it will have larger p-values. From the above numbers you posted, it is clear that both will reject the null of a "good fit", but maybe one method is much closer to a good fit.
To compare the platforms against one another, I would recommend a glm. You can see how to use them in this paper (http://www.biomedcentral.com/1471-2105/11/94 and http://genomebiology.com/2010/11/10/r106 (don't worry that those are about transcriptomics, it's just counting statistics for both in the end)). Basically, you'd be testing for whether a "platform" effect exists. The normalization would come into the glm offsets. Your model would be:
Count ~ platform * bacteria
Each of your replicates would be one row in the table. Since there are only two levels on the platform it's just a t-test.
With a lot of zeros in both series, it may be difficult to reject that the null that the means are the same. But you could test for differences in the deciles (or other quantiles) from the two distributions. If the tails are different then the samples are different. For a test see Li, Tiwari and Wells (1996).
Best Answer
In response to an old question, and given that a good response has been provided already elsewhere by jbowman and StasK to a very similar (but better defined) problem. I refer anyone who stumbles on this to the following question (and answers): Test for significant difference in ratios of normally distributed random variables
The permutations test should be easy to implement in most statistical tools and many programming languages. Additionally, it doesn't assume that you have count data but means that you can use a ratio of rates or other appropriate metrics.