When running models from a bayesian perspective — a regression for example — we get posterior distribution for every parameter/statistic we want, right? I’m wondering whether I should see this this posterior distribution as a regular distribution (as in rnormwith a mean and Standard Deviation), or as a sample mean distribution (with a mean and Standard Error).
Bayesian Analysis – Correct Way to Interpret Posterior Distributions
bayesianposteriorpymcstan
Best Answer
The distinction you are trying to make in your question is quite artificial, and it suggests there is perhaps a deeper interpretive problem here. Probability distributions describe the probabilistic behaviour of random variables. We sometimes refer to "families" of distributions (e.g., the normal distribution) and these families of distributions are somewhat "regular" in the sense that they are commonly used in probability and statistical problems. However, there is no distinction between a "regular distribution" that applies to a single value versus an "irregular distribution" which applies to a sample mean.
Here it is worth noting that the sample mean of a set of random variables is itself a random variable, so it will have a probability distribution of some kind; typically something close to a normal distribution in most cases involving a large number of sample values. Indeed, if you consider the normal distribution to be a "regular" distribution then it is worth noting that the sample mean of normal random variables will also follow a normal distribution (albeit with a different variance).
In view of this, my recommendation would be to set aside Bayesian statistics for a moment, and first interrogate what you mean by a "regular" distribution. I think you will find that you are trying to draw an artifical distinction that is not sensible in the first place.