In the $6$th chapter of the textbook *A first course in probability* by Ross, there is the following question:

Let $N$ be a geometric random variable with parameter $p.$ Suppose that the conditional distribution of $X$ given that $N = n$ is the gamma distribution with parameters $n$ and $\lambda.$ Find the conditional probability mass function of $N$ given that $X = x.$

At first, I'll setup notation and a solution. Then, I'll start towards another solution, which I cannot complete at the moment.

We are given $f_{X|N=n}(x) = \frac{\lambda e^{- \lambda x} (\lambda x)^{n-1}}{\Gamma(n)} = \frac{\lambda e^{- \lambda x} (\lambda x)^{n-1}}{(n-1)!},$ since $n$ is a positive integer; and we want to find $P(N=n | X=x).$ So, we write

$$P(N=n | X=x) = \frac{f_{X|N=n}(x) P(N=n)}{f_{X}(x)},$$

which is equal to

$$c \frac{(\lambda(1-p)x)^{n-1}}{(n-1)!},$$

where $c$ is a term that does not depend upon $n.$ Now, we know that

$$1 = c \sum_{n=1}^{\infty} \frac{(\lambda(1-p)x)^{n-1}}{(n-1)!} = c e^{\lambda(1-p)x},$$ so $c$ must be equal to $e^{- \lambda(1-p)x}.$ Thus,

$$P(N=n | X=x) = e^{- \lambda(1-p)x} \frac{(\lambda(1-p)x)^{n-1}}{(n-1)!}$$ is equivalent to a Poisson probability mass function, where the parameter is $\lambda(1-p)x$ and $n=1$ corresponds to $1 – 1 = 0,$ and so on, in terms of the numerical values.

Now, the alternative solution starts the same, but aims to avoid the constant term $c.$ That is,

$$P(N=n | X=x) = \frac{f_{X|N=n}(x) P(N=n)}{f_{X}(x)} =

\frac{\frac{\lambda e^{- \lambda x} (\lambda x)^{n-1}}{(n-1)!} (1-p)^{n-1}p}{\sum_{n=1}^{\infty} \frac{\lambda e^{- \lambda x} (\lambda x)^{n-1}}{(n-1)!}},$$

which simplifies to

$$\bigg(\frac{p}{\sum_{n=1}^{\infty} \frac{(\lambda x)^{n-1}}{(n-1)!}} \bigg) \bigg(\frac{(\lambda(1-p)x)^{n-1}}{(n-1)!}\bigg).$$

So, the first term in parenthesis is equivalent to $c.$ Moreover, it is equal to

$$\frac{p}{\sum_{m=0}^{\infty} \frac{(\lambda x)^{m}}{(m)!}} = \frac{p}{e^{\lambda x}} = p e^{- \lambda x}.$$ But I do not see how to proceed from here, or, where I should trace back to in the derivation and proceed from there.

## Best Answer

The infinite series in the denominator is missing a factor of $\left(1-p\right)^{n-1}p$. Basically you need to add up the value from the numerator over all possible $n$. After including this factor, you can cancel out the $p$ in the numerator, and the series will sum to $e^{\lambda\left(1-p\right)x}$, which is what you need.