1. Introduction
I want to know how the distribution of the maximum of two independent normal distribution is like.
The background is that I am making AI of some games, and I am trying to do something similar to Minimax algorithm. However, with assumption that the noise of evaluation function is normally distributed, I realized that the algorithm needs to estimate how the maximum of several normal distribution is like.
(There is NO requirement to understand the background like Minimax, for answering this question.)
2. Writing Distribution Mathematically
Let $f_1(x) = \frac{1}{\sqrt{2\pi}} e^{-\frac{x^2}{2}}$, the probability density function (PDF) of standard normal distribution, and let $\Phi_1(x)$ be the cumulative distribution function (CDF) of it.
Let $f_2(x)$ and $\Phi_2(x)$ be the PDF and CDF of the maximum of two independent standard normal distribution. It is not difficult to see that $\Phi_2(x) = \Phi_1(x)^2$. Therefore,
$$f_2(x) = \frac{d}{dx} \Phi_2(x) = 2 \frac{d\Phi_1(x)}{dx} \Phi_1(x) = 2 f_1(x) \Phi_1(x)$$
We drew a graph at desmos.com of the PDF and CDF, and it was like the following figure (PDF is red and CDF is blue).
We can see that the PDF is not symmetric, and is a little bit skewed to the right (positive skew).
3. The average and variance
Using desmos.com, I could obtain the following result:
- The average is $\mu = 0.5641895835\dots$
- The standard deviation is $\sigma = 0.8256452711\dots$
I could understand that $\sigma = \sqrt{1 – \mu^2}$, because one can show that the expected square of variable is exactly $1$.
$$E[\max(x_1, x_2)^2] = \int_{-\infty}^{\infty} t^2 f_2(t) dt = \int_{-\infty}^{\infty} t^2 f_1(t) \Phi_1(t) dt$$
$$= \frac{1}{\pi} \int_{-\infty}^{\infty} t^2 e^{-\frac{t^2}{2}} \left(\int_{-\infty}^t e^{-\frac{u^2}{2}} du\right) dt = \frac{1}{\pi} \int_{-\infty}^{\infty} \int_{-\infty}^t t^2 e^{-\frac{t^2 + u^2}{2}} du dt$$
$$= \frac{1}{\pi} \int_0^{\infty} \int_{-\frac{3}{4} \pi}^{\frac{1}{4} \pi} r^3 \cos^2 \theta \cdot e^{-\frac{r^2}{2}} d\theta dr = \frac{1}{\pi} \cdot \left(\int_0^{\infty} r^3 e^{-\frac{r^2}{2}} dr \right)\left(\int_{-\frac{3}{4} \pi}^{\frac{1}{4} \pi} \cos^2 \theta d\theta \right)$$
$$= \frac{1}{\pi} \cdot \left[-(r^2 + 2) e^{-\frac{r^2}{2}}\right]_0^{\infty} \cdot \left[\frac{1}{2} (\sin x \cos x + x)\right]_{-\frac{3}{4} \pi}^{\frac{1}{4} \pi} = 1$$
4. Question
However, I don't have clue how to calculate the exact value of $\mu = 0.5641895835 \dots$
I thought that it is a well-known constant because it is a consequence of a simple problem. However, although I searched on the Internet, there did not seem to be any result about it.
So, I would like to know:
- whether there is a formula of expressing $\mu$, and
- the way to calculate $\mu$ precisely.
Additionally, I am also thinking about generalizing to arbitrary set of independent normal distributions about the mean/variance of maximum value.
Best Answer
Let $X_1, X_2 \sim \mathcal{N}(0,1)$ be independent. The trick here is to write $$\max\{X_1,X_2\}=\frac{1}{2}(X_1+X_2+|X_1-X_2|)$$ From the linearity of expectation, we get that $$\mathbb{E}(\max\{X_1,X_2\})=\frac{1}{2}\mathbb{E}(|X_1-X_2|)$$ Remember now that $X_1-X_2 \sim \mathcal{N}(0,2)$. The expectation of the absolute value of a Gaussian with mean $\mu=0$ and variance $\sigma$ comes out to $\sigma \sqrt{\frac{2}{\pi}}$ so in this case ($\sigma=\sqrt{2}$) we get that $$\mathbb{E}(\max\{X_1,X_2\})=\frac{1}{2}\cdot\frac{2}{\sqrt{\pi}}=\frac{1}{\sqrt{\pi}}\approx 0.56418\dots$$ which is what you calculated graphically.
Generalizing to an arbitrary set of independent standard normal distributions seems to be hard, according to this MathOverflow question.