[Math] Maximum a Posteriori (MAP) Estimator of Exponential Random Variable with Uniform Prior

estimationestimation-theoryprobabilityprobability distributionsprobability theory

What would be the Maximum a Posteriori (MAP) estimator for $ \lambda $ for IID $ \left\{ {x}_{i} \right\}_{i = 1}^{N} $ where $ {x}_{i} \sim \exp \left( \lambda \right), \; \lambda \sim U \left[ {u}_{0}, {u}_{1} \right] $?

One could assume that $u_0 > 0 $.

The Exponential Distribution is given by:

$$ f(x; \lambda) = \begin{cases}
\lambda {e}^{-\lambda x} & x \ge 0, \\
0 & x < 0.
\end{cases} $$

Best Answer

You have a likelihood function $$ L(\lambda) = (\lambda e^{-\lambda x_1})\cdots(\lambda e^{-\lambda x_n}) = \lambda^n e^{-\lambda(x_1+\cdots+x_n)}, \tag 1 $$ and $$ \ell(\lambda)=\log L(\lambda) = n\log\lambda -\lambda(x_1+\cdots+x_n) \tag 2 $$ The density of the uniform distribution on the interval $\lambda\in[u_0,u_1]$ does not depend on $\lambda$. Therefore the posterior density is proportional to $(1)$ on the interval $[u_0,u_1]$. The maximum value of the posterior density therefore occurs at the point in the interval $[u_0,u_1]$ where $(1)$ attains its maximum value. Since $(2)$ is an increasing function of $(1)$, that is the same as the point in the interval $[u_0,u_1]$ where $(2)$ attains its maximum. We have $$ \ell'(\lambda) = \frac n \lambda - (x_1+\cdots+x_n)\quad\begin{cases} >0 & \text{if }0<\lambda < \bar x = (x_1+\cdots+x_n)/n, \\[6pt] =0 & \text{if } \lambda=\bar x, \\[6pt] <0 & \text{if } \lambda>\bar x. \end{cases} $$ Therefore the maximum value is attained at $$ \lambda = \begin{cases} u_0 & \text{if } u_0 > \bar x, \\[6pt] \bar x & \text{if } u_0\le\bar x\le u_1, \\[6pt] u_1 & \text{if }u_1<\bar x. \end{cases} $$ If you like, you can write this as $$ \lambda = (u_0 \vee \bar x) \wedge u_1 = u_0 \vee (\bar x \wedge u_1) $$ where $a\vee b$ is whichever of $a,b$ is bigger and $a\wedge b$ is whichever is smaller.

Related Question