I have recently started exploring some bayesian statistics and I cannot seem to understand something about improper priors. In particular, the set up consists of a Poisson likelihood $ p(X|\theta) = \frac{\theta^x e^{-\theta}}{x!}$ and some improper prior on $\theta$, $p(\theta) = \frac{1}{\sqrt{\theta}}$. Now to find the prior predictive distribution we use the following:
$$p(X) = \int_{0}^{\infty}p(X|\theta)p(\theta)d\theta = \int_{0}^{\infty}\frac{\theta^{x-\frac{1}{2}} e^{-\theta}}{x!}d\theta = \frac{\Gamma{(x+\frac{1}{2})}}{x!}\int_{0}^{\infty}\frac{1}{\Gamma{(x+\frac{1}{2})}}\theta^{ (x+\frac{1}{2}) -1} e^{-\theta}d\theta = \frac{\Gamma{(x+\frac{1}{2})}}{x!}$$.
If $p(X)$ is a probability distribution it must equal to one when summed over its support, which does not happen here.
Therefore, I wanted to ask whether that is happening because of the structure of the prior or if there is perhaps something else that I'm missing. I have seen some examples of discrete likelihoods with finite support where we can't use improper priors, which makes sense, however, I'm confused why we can use an improper prior in this case even though we get a posterior density in a recognisable form when at the same time the prior predictive does not sum up to 1.
Best Answer
You are right that in this case, for the prior predictive we get that $$ \sum_{x=0}^\infty p(x)=\infty ,$$ so if you need the prior predictive distribution, you will need to do the work to model a reasonable proper prior distribution. You did not tell us why you did choose the prior $\frac1{\theta^{1/2}}$, probably you need a better choice of prior.
Sometimes (but seemingly not this case), problems with improper priors can be solved by first integrating $p(x \mid \theta)p(\theta)$ over a finite interval (say, $[0, M]$) and afterwards take the limit. For examples, see Numbers of draws on a modified Bernouilli process