Solved – How to find the expectation of the maximum of independent exponential variables

expected valueexponential distributionorder-statisticsprobability

Suppose $x_1, x_2, ……, x_n$ are i.i.d. random variable of exponential distribution $Exp(1)$, i.e., $f(x)=e^{-x}, x\gt0$.

Given the order statistics $x_{(1)} \le x_{(2)} \le……\le x_{(n)}$,
it is easy to find out that
$$F_{x(1)}(x)= 1 – \Big[1-F_{x}(x)\Big]^n = 1- \Big[1-(1-e^{-x})\Big]^n=1-e^{-nx}$$
$$F_{x(n)}(x) = \Big[F_{x}(x)\Big]^n = (1-e^{-x})^n$$
Taking the derivative,
$$f_{x(1)}(x) = ne^{-nx}$$
$$f_{x(n)}(x) = n(1-e^{-x})^{n-1}e^{-x}$$

Taking the integral, we have the expectation:
$$E(x_{(1)}) = \int_0^\infty xne^{-nx}dx = \int_0^\infty xn d\Big(-\frac{e^{-nx}}{n}\Big) = \Big[-xe^{-nx}\Big]_0^\infty + \int_0^\infty e^{-nx}dx=\frac{1}{n}$$

But how could I obtain the $E(x_{(n)})$, the expectation of the largest order statistic?

Best Answer

The answer referenced in the comments is great, because it is based on straightforward probabilistic thinking. But it is possible to obtain the answer through elementary means, beginning from definitions.

Because $x_{(n)}$ is the largest of $n$ independent variables, the event $x_{(n)}\le x$ is the event that all the $x_i \le x.$ Stipulating the $x_i$ have Exponential$(1)$ distributions says that for $x\gt 0,$ these have common probability $1 - e^{-x}$ (and otherwise have zero probability).

Since probabilities of independent events multiply,

$$\Pr(x_{(n)} \le x) = \left(1 - e^{-x}\right)^n.$$

One well-known formula for the expectation of a positive random variable with distribution function $F$ is the integral of $1-F$ from $0$ to $\infty.$ (Take the usual integral for the expectation and integrate by parts.) We are looking, then, to compute

$$E_n = E\left[x_{(n)}\right] = \int_0^\infty 1 - \left(1 - e^{-x}\right)^n\,\mathrm{d}x$$

for $n=1, 2, 3, \ldots.$

That initial "$1$" in the integrand is thorny, because its integral diverges, so we cannot separate it out. However, the differences between these quantities are considerably simpler to compute because the $1$'s cancel:

$$E_{n} - E_{n-1} = \int_0^\infty 1 - \left(1 - e^{-x}\right)^n - \left[1 - \left(1 - e^{-x}\right)^{n-1}\right]\,\mathrm{d}x = \int_0^\infty \left(1 - e^{-x}\right)^{n-1}e^{-x}\,\mathrm{d}x.$$

This is a textbook case for integration by substitution: the natural form to try is $u = 1-e^{-x},$ reducing the integral to

$$E_{n} - E_{n-1} = -\int_1^0 u^{n-1}\,\mathrm{d}u = \frac{1}{n}.$$

Beginning with $E_0=\int (1-1)\mathrm{d}x = 0,$ we obtain recursively

$$\begin{aligned}E_n &= E_0 + (E_1 - E_0) + (E_2 - E_1) + \cdots + (E_n - E_{n-1}) \\&= 0 + \frac{1}{1} + \frac{1}{2} + \cdots + \frac{1}{n} = H(n), \end{aligned}$$

the $n^\text{th}$ harmonic number.