Bayesian – Comparing Bayesian and Frequentist Estimation Methods

bayesianestimationfrequentistinferencemean

I don't really understand the connection between bayesian to "normal" frequentist estimation.

Suppose we want to estimate the expected value of a population given a sample.

In frequentist statisics i'd calculate the sample mean $\overline{x}=\frac{1}{n}\sum_{i=1}^nx_i$ which is, as we know unbiased, consistent and efficent – and therefore the "best" estimator for the true mean we can use.
This estimator has a standard deviation of $\text{SE}_\bar{x}\ = \frac{s}{\sqrt{n}}$.

In contrast in bayesian statistics i'd have some a-priori distribution, plug in data and calculate with the bayesian formula an a-posteriori distribution.

  • So how does this a-posteriori distribution compare to the sample mean point estimate?
  • Is the expected value of the a-posteriori distribution equal to the sample mean?
  • Is the standard deviation of the a-posteriori distribution equal to the standard deviation of the sample mean?
  • Which one of the two methods is the better one – or does the bayesian approach just give you more information (it gives you a distribution whereas the frequentist estimation only gives you point values)
  • Is it possible to get point estimates like in the frequentist estimation from an a-posterio distibution (e.g. the expected value for an point estimation for the true mean)?

Edit:

Ok, I got the idea but what really bothers me is for example in https://en.wikipedia.org/wiki/German_tank_problem#Example
the german tank problem.

We want to estimate the total number of tanks where an intelligence officer has spotted k = 4 tanks with serial numbers, 2, 6, 7, and 14.

The frequentist estimate to the tank count is $16.5$ whereas the bayesian is $19.5 \pm
10$ (although the frequentist answer is in the sd. of the bayesesian).

How can two different mathematical (scientific) approaches for the same question lead to two different answers? Intuitively there must be one "more" correct answer, or not?

Best Answer

The simple answer for all your questions is: in Bayesian model you include a priori information in your model besides the data,

$$ \text{posterior} \propto \text{likelihood} \times \text{prior} $$

so if you include in your model additional information, than your estimates (mean, standard deviation etc.) can possibly differ from maximum likelihood estimates. You can use noninformative priors, and in this case estimates should be the same as in maximum likelihood case. One example where Bayesian methods can outperform other methods are cases where we are dealing with small samples (see nice example here), even in cases such as predicting future mid-air collisions in times when non such collision happened yet - in such cases out-of-data information helps us to learn from insufficient data.

You ask also about posterior distribution vs. sample mean and about point estimates. In posterior distribution you have a whole distribution for your parameter of interest instead of a point value, you can take mean (or median, or possibly other statistics) of this distribution to get a point value. If you are interested only in point estimate, you can use maximum a posteriori methods and not bother with posterior distribution.

Finally, neither of the methods are "better", or "worse". They are just different. By reviewing multiple questions tagged as you can learn much about their pros and cons, for example: When are Bayesian methods preferable to Frequentist?, or Why are Bayesian methods widely considered particularly "convenient"?.

Related Question