Suppose that $X_1, X_2, \ldots, X_n$ are IID Bernoulli random variables with success probability equal to an unknown parameter $\theta \in [0,1]$.
Let $A$ and $B$ be nonnegative constants. If we impose the prior $\pi(\theta) \propto (\theta^a)((1-\theta)^b)$, then what is the Bayesian posterior mode?
Best Answer
To find the Posterior mode, we need to maximize the posterior distribution with respect to $\theta$. So first we find the joint density:
$\displaystyle f(X|\theta) = \prod_{i=1}^n \theta^{x_i}(1-\theta)^{1-x_i}=\theta^{\sum x_i}(1-\theta)^{n-\sum x_i}$
Then we can get the posterior distribution:
$\displaystyle \pi(\theta |x)\propto f(x|\theta)\pi(\theta)\propto \theta^{\sum x_i}(1-\theta)^{n-\sum x_i} \theta^a(1-\theta)^b=\theta^{\sum x_i+a}(1-\theta)^{n+b-\sum x_i}=\exp\left((n-n\bar{X}+b)\log(1-\theta)+(n\bar{X}+a)\log\theta\right)$
To find the posterior mode, we need to maximize the posterior with respect to $\theta$:
Let $g(\theta)=(n-n\bar{X}+b)\log(1-\theta)+(n\bar{X}+a)\log\theta$
$\displaystyle g^{\prime}(\theta)=-\frac{n-n\bar{X}+b}{1-\theta}+\frac{n\bar{X}+a}{\theta}=0$
Solving for $\theta$, we get $\displaystyle\hat{\theta}=\frac{n\bar{X}+a}{n+a+b}$, which you can check is the maximum by making sure $g^{\prime\prime}(\theta)>0$.