Consider spatially distributed data $\boldsymbol x = \{x(t_1),x(t_2),\dots,x(t_n)\}$ which is described by a multivariate Gaussian distribution with mean $\mu$, standard deviation $\sigma$ and stationary correlation function which depends on the absolute distance $\tau = |t_1-t_2|$ between points:
$$\rho(\tau) = \exp\left(-\frac{2\tau}{\theta}\right)$$
Which produces a correlation matrix $\boldsymbol \rho(\theta)$.
$$\boldsymbol x \sim \mathcal{N}(\mu,\sigma^2 \boldsymbol \rho)$$
Given samples of $\boldsymbol x$ the aim is to predict $\mu$, $\sigma$ and $\theta$. Under the Bayesian approach I believe that a joint prior distribution must be specified. Since I want to use Markov Chain Monte Carlo to sample from the posterior I believe the prior should be proper. My approach is to specify marginal distributions $P(\mu)$, $P(\sigma)$, $P(\theta)$ and take the joint prior to be given by:
$$P(\mu,\sigma,\theta) = P(\mu)P(\sigma)P(\theta)$$
Is it reasonable to take the priors to be independent?
Can I take the priors to be uniform if am confident the parameter values fall in certain intervals i.e.
$$\mu \sim U[-100,100]$$
$$\sigma \sim U[0,1000]$$
$$\theta \sim U[0,1000]$$
Or should I use a Cauchy for $\mu$ and the folded-Cauchy for $\sigma$, $\theta$?
Best Answer
I believe you are using uniform priors in an attempt to be non-informative, but I believe your priors are more informative than you would like them to be. Do you really believe $\sigma$ or ($\theta$) is equally likely to be between (0,1) as (999,1000)? And that there is absolutely no probability whatsoever that these parameters are above 1000?
Also, there is no requirement to have proper priors for MCMC.
My suggestion is $\sigma \sim Ca^+(0,c_\sigma)$ (i.e. a folded-Cauchy or half-Cauchy), $\theta \sim Ca^+(0,c_\theta)$, an improper uniform for $\mu$ and all independent. You will need to choose $c_\sigma$ and $c_\theta$ for your problem, but since you are already "confident" the parameter values fall into the regions given, you can just choose these values so the probability outside those regions is small. I prefer heavy-tailed distributions like the half-Cauchy because the data can more easily overwhelm the prior in case you were wrong in your prior assumptions. If you really want the prior to be proper, you could use a Cauchy for $\mu$ and specify a location and scale parameter.