I have data from two populations of different sizes. Both have Gamma distributions with different shapes and scales (as estimated in R):

```
fitdistr(x/10000, "gamma") #186 members
shape 0.586219900 (0.050840587);
rate 0.012159695 (0.001564117);
mean(x) 44.26789
fitdistr(y/10000, "gamma") #50 members
shape 0.491757644 (0.080661189);
rate 0.006671180 (0.001709453);
mean(y)
68.15945
```

I want to answer the question: What is the probability that the two populations are significantly different. By different, I am referring to the mean and not to the shape and rate of the distributions. Does the average of y indicate that there is a significant change from the average of x?

## Best Answer

Let's simulate some data based on your fits:

Now, if you are only interested in whether the means of the underlying distributions differ, the canonical approach is a standard t test on the data themselves, without fitting anything:

Note that this is (asymptotically) valid. t tests make pretty weak assumptions on the shape of the distributions of the underlying data. This is because by standard theorems, means of samples (which we are interested in here) are asymptotically normally distributed. You have sample sizes of 186 and 50. In such situations, I'll usually use a t test without a second thought.

That said, your estimated rate parameters indicate that you have some pretty degenerate gammas (if, as @Glen_b suggests, they are gamma at all):

(Compare what Wikipedia authors think is a "standard gamma".)

So the question pops up whether we have enough samples for asymptotics to kick in and make the t test valid, after all. One possibility would be to nonparametrically bootstrap the difference in means:

Now, I would say that $p=.0012$ from the nonparametric bootstrap is close enough to $p=.007$ from the t test, considering that the smaller group has 50 observations and no more.

Another alternative would be a parametric bootstrap, if you are really sure (as per theory) that your data actually should be gamma. Bootstrap your data, fit gammas to the resampled data, then either resample from the fitted gammas (my approach below), or directly calculate the expectations of the fitted gammas as shape/rate:

Now we only have $p=.02$, quite a bit more than above.

Bottom line: you should really look at your data closely. If you fit a gamma, it will be pretty degenerate - so degenerate, in fact, that I'd be wary of making

anyinferences about whether the means of these fitted (!) gammas will be significantly different. I'd most trust the nonparametric bootstrap - but even that can be unstable for heavily skewed distributions (Good mentions a threshold of $n=100$ in his 2006 book). One other alternative would be a permutation test, where you assess the null distribution of means under random permutation of group labels.And the

bottombottom line: of course p values are not "probabilities that populations (or means) are significantly different". See here.