I am looking for a prior for a scale parameter for which I have prior knowledge such that: "$\sigma$ typically does not exceed 100." ("typically" meaning that occasionnally this can happen).
In such a context, I notice in the paper "Prior distributions for variance parameters in
hierarchical models" of Andrew Gelman
the following recommandation:
[…] When more prior information is desired, for instance to restrict σ away from very
large values, we recommend working within the half-t family of prior distributions,
which are more flexible and have better behavior near 0, compared to the inverse-gamma family. A reasonable starting point is the half-Cauchy family, with scale set to
a value that is high but not off the scale."
As I understand it, a Cauchy (thus half-Cauchy) distribution has an infinite variance and I am not confortable with the idea of building an informative prior with an infinite variance density. Have you some insight on why my interpretation is bad/unsuited ?
Moreover, have you some alternative proposals for my prior ?
Best Answer
An alternative to using a Half-Cauchy distribution with a well-defined variance is a Half-Student-t with $\nu>2$ degrees of freedom, e.g. $\nu=3$.
$$\pi(\nu)= \frac{12 \sqrt{3}}{\pi \left(x^2+3\right)^2},\,\,\, \nu>0. $$
This prior has semi-heavy tails and it should produce fairly similar results as the Half-Cauchy prior. You can visualise it in R with the following code
curve(2*dt(x,df=3),0,10). It can also be interpreted as "I have prior information, but not much" since you are using something that resembles a "vague prior" but you are adding a bit of information since you think that the tails shouldn't be that heavy. The mass cumulated on $(100,\infty)$ is $1.102261e-06$.