I really want to learn about Bayesian techniques, so I have been trying to teach myself a bit. However, I am having a hard time seeing when using Bayesian techniques ever confer an advantage over Frequentist methods. For example: I've seen in the literature a bit about how some use informative priors whereas others use non-informative prior. But if you're using a non-informative prior (which seems really common?) and you find that the posterior distribution is, say, a beta distribution…couldn't you have just fit a beta distribution in the beginning and called it good? I don't see how constructing a prior distribution that tells you nothing…can, well, really tell you anything?
It does turn out that some methods I have been using in R use a mixture of Bayesian and Frequentist methods (the authors acknowledge this is somewhat inconsistent) and I cannot even discern what parts are Bayesian. Aside from distribution fitting, I can't even figure out HOW you would use Bayesian methods. Is there "Bayesian regression"? What would that look like? All I can imagine is guessing at the underlying distribution over and over again while the Frequentist thinks about the data some, eyeballs it, sees a Poisson distribution and runs a GLM. (This isn't a criticism…I really just don't understand!)
So..maybe some elementary examples would help? And if you know of some practical references for real beginners like myself, that would be really helpful too!
Best Answer
Here are some links which may interest you comparing frequentist and Bayesian methods:
In a nutshell, the way I have understood it, given a specific set of data, the frequentist believes that there is a true, underlying distribution from which said data was generated. The inability to get the exact parameters is a function of finite sample size. The Bayesian, on the other hand, think that we start with some assumption about the parameters (even if unknowingly) and use the data to refine our opinion about those parameters. Both are trying to develop a model which can explain the observations and make predictions; the difference is in the assumptions (both actual and philosophical). As a pithy, non-rigorous, statement, one can say the frequentist believes that the parameters are fixed and the data is random; the Bayesian believes the data is fixed and the parameters are random. Which is better or preferable? To answer that you have to dig in and realize just what assumptions each entails (e.g. are parameters asymptotically normal?).