I was just recently using the Student t-Test to check whether values from two samples could have an identical mean or not. I was wondering whether there is a complementary technique in bayesian statistics that does a similar thing.
Especially I am wondering about the $P$ values that I obtain as a result (http://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.ttest_ind.html). With a sample size of 2 in my two samples, I figure that the resulting p value would probably fluctuate quite a bit if I added some more measurements or take some away.
Best Answer
Two come to mind.
Morey and Rouder present "Bayesian t tests for accepting and rejecting the null hypothesis". The reference paper is here, there is a handy web interface, and an R package. There are also equivalent programs for ANOVA and correlation.
John Kruschke claims that "Bayesian estimation supersedes the t test" ("BEST"). The reference paper is here, a web app by Rasmus Bååth here, and of course an R package. For more, see the web page, including a Python implementation. Rasmus Bååth has implemented a range of further tests in this tradition, but also provides a very readable explanation of BEST.
The primary difference between the two, BEST and Rouder/Morey's test, is that Morey and Rouder present their approach explicitly in the tradition of hypothesis testing, whereas BEST is parameter estimation. Under the hood, the two are not too dissimilar - default priors, MCMC sampling - but the outpood is quite different; the first gives you the Bayes Factors for or against your hypothesis, the other focuses the graphical presentation on the credible interval for the estimated parameter.