Likelihood Function vs PDF – Why Is a Likelihood Function Not a PDF?

density functionlikelihood

What is the reason that a likelihood function is not a pdf (probability density function)?

Best Answer

We'll start with two definitions:

  • A probability density function (pdf) is a non-negative function that integrates to $1$.

  • The likelihood is defined as the joint density of the observed data as a function of the parameter. But, as pointed out by the reference to Lehmann made by @whuber in a comment below, the likelihood function is a function of the parameter only, with the data held as a fixed constant. So the fact that it is a density as a function of the data is irrelevant.

Therefore, the likelihood function is not a pdf because its integral with respect to the parameter does not necessarily equal 1 (and may not be integrable at all, actually, as pointed out by another comment from @whuber).

To see this, we'll use a simple example. Suppose you have a single observation, $x$, from a ${\rm Bernoulli}(\theta)$ distribution. Then the likelihood function is

$$ L(\theta) = \theta^{x} (1 - \theta)^{1-x} $$

It is a fact that $\int_{0}^{1} L(\theta) d \theta = 1/2$. Specifically, if $x = 1$, then $L(\theta) = \theta$, so $$\int_{0}^{1} L(\theta) d \theta = \int_{0}^{1} \theta \ d \theta = 1/2$$

and a similar calculation applies when $x = 0$. Therefore, $L(\theta)$ cannot be a density function.

Perhaps even more important than this technical example showing why the likelihood isn't a probability density is to point out that the likelihood is not the probability of the parameter value being correct or anything like that - it is the probability (density) of the data given the parameter value, which is a completely different thing. Therefore one should not expect the likelihood function to behave like a probability density.

Related Question