Assuming that I performed n iid tests, and the total number of test is n which is a fixed value, and the observaton of 1 which corresponding to successful results is X observations yeild with probability p, I know X a random variable draw from binomial distribution B(n,p). Now, assuming the prior P(p) from an uniform distribution, I want to calculate a posterior probability P(p|n,X) with respect to different X values.
From some textbooks, I know the conjugate prior for the binomial distritbution is a beta distribution. But the deta distribution is for continueous random variables. So how can I calculate the posterior P(p|n,X)?
Shall I use beta-binomial distribution instead of beta distribution to calculate the posterior probabilty P(p|n,X)?
I tried to calculate the posterior P(p|n,X) directly, using Bayesian rule, but I'm not sure wehther or not I can calculate P(p|n,X) in this way. Suppose n is fixed, P(p) is uniform distribution. My method is that I simply calculate the posterior P(p|n,X) from P(X|n,p)~B(n,p) with different p values. For instance, I calculated a 100X100 2D table of P(X|n,p) with different X (X as the first dimension of the table, range form 1 to n, with interval 1) and p (p as the second dimension of the table range form 0.01 to 1.0, with interval 0.01), then the posterior probability P(p|n,X) for each discretized p with respect to praticular X value from the 2D table by using Bayesian rule. Is this a correct way to calculate the P(p|n,X)?
Best Answer
A uniform distribution is a special kind kind of beta distribution, which for the binomial distribution we know that it is a conjugate distribution. This means that if we combine the binomial likelihood with our beta prior, then our posterior will be a beta distribution.
Note that assuming a beta prior corresponds to considering the parameter $p$ as a continuous random variable on the (0,1) interval. This needs not be the case in general. For instance, we may assume that the prior is just a two-point distribution, for instance $P(\theta = 0.3) = 0.7$ and $P(\theta = 0.7) = 0.3$, and our posterior will also be a two-point distribution.
Returning to the uniform prior case, we know that our posterior will be of the form
$$p\left(\theta|\mathbf{x} \right) \propto f\left(\mathbf{x} |\theta \right) g\left(\theta\right) = \binom{n}{x} \theta^x \left(1-\theta\right)^{n-x} 1, \quad 0<\theta<1 $$
And you can determine the normalizing constant by figuring out what you need for this integrate to one (as a function of $\theta$). Once you plug in the values of $n$ and $x$ you can plot this as a function of $\theta$ and determine the value you like the most, e.g. mean, median, mode etc. Where did you get stuck?