Solved – Convert probabilities of “binomial events” into beta distribution for binomial parameter (theta)

Question

I have three events that essentially arise from a binomial distribution with k=2, i.e., X~Binomial(theta,n=2). Given some model and data I obtained a conditional (pos distribution for a particular X_i, that is Pr(X_i|Data), e.g., Pr(X={0,1,2}|Data)={.05,.45,.50}. I would now like to convert these "binomial probabilities" into a continous probability distribution of theta for this particular instance of X_i, that is I am interested in p(theta_i|Data). I know the beta-binomial model and how to get p(theta|Data) in that case from the so called "hard" data (say X={2,1,2,0,0} for 5 observations), but in my problem I have "soft" data on one observation only.

Answer 1

I have previously searched for quite a while on this topic in a slightly different context. I don’t think there is a “standard” answer. The best theory source I found was:

F. Groen, "Foundations of probabilistic inference with uncertain evidence", 2005: http://www.sciencedirect.com/science/article/pii/S0888613X04000830

Your question (I think) is about the posterior predictive distribution for X . To my mind the main parameter we need to estimate is theta - once we know this we can play tunes on the probability of data, and estimate predictive distributions of future data. So re-phrasing your question:

What is P(theta | D), given that D is uncertain?

It sounds like you are familiar with the standard Bayesian approach to solving this with integer data, so I won’t review this. Two possible (practical) approaches are given below.

Approach 1 - An ad hoc approach

An ad hoc solution to soft data is proceed as normal but to use the fractional values in the exponents of the binomial distribution, and use the gamma function to generalise the factorials in the combinatorial term to continuous values. i.e. allow r (and possibly n) to be fractional in the below equation.

$$P(r; n, \theta)= C_r^n\theta^r(1-\theta)^{n-r}$$

And use the gamma function for the combinatorial term (which reduces to factorials for integer values):

$$C_r^n=\frac{\Gamma(n+1)}{\Gamma(r+1)\Gamma(n+1-r)}$$

For the data you quote you would use r=1.45 and n=2. Please be cautious as this is definitely a "practical" approach - obviously the "binomial distribution with fractional values" isn't a probability distribution. However, if you follow it through then the posterior is still a proper probability distribution due to the normalisation integral. So it gives "sensible" results - i.e. answers that are bounded on either side by "proper" integer observations, and scale in a "reasonable" way in between.

Approach 2 - Weight each possibility

In the likelihood expression weight each possible outcome by the probability that it occurred. So for the likelihood term is of the form:

$$P(r; n, \theta)= \sum_{r=0}^{n} P(r){C_r^n\theta^r(1-\theta)^{n-r}}$$

The difficulty with this approach is that the number of possibilities expands (combinatorially) as you make more soft observations. However, since you only have a few data observations this shouldn’t be a problem for you. See section 5.1 and section 7.1 in the reference supplied above - in particular equations 20 and 76.