From Michigan State University's Herzog contest:
Three points are taken at random on a unit sphere. What is the probability that the area of the spherical triangle exceeds the area of a great circle?
I assume we always take the unique proper spherical triangle, with sides and angles not greater than $\pi$. I assume "the area of a great circle" is the plane area $\pi$, since a proper spherical triangle never has area larger than the spherical area $2 \pi$ on each side of a great circle. I assume the random distribution is uniform, such that the probability each point is within a given measurable subset of the sphere is proportional to the spherical area of that subset.
First, given triangle vertices $A,B,C$ on the unit sphere, we can rotate and/or mirror the coordinate system so that $C = (0,0,1)$, $A$ is on the $x \geq 0, y=0$ half-plane, and $B$ is on the $y \geq 0$ hemisphere. Label the side lengths opposite $A$, $B$, and $C$ as $a$, $b$, and $c$ respectively. Label the (dihedral) angles at $A$, $B$, and $C$ as $\alpha$, $\beta$, and $\gamma$ respectively. Ignore a degenerate triangle case with any of these variables exactly equal to $0$ or $\pi$; this happens with probability zero. Then all six variables are in the interval $\{a,b,c,\alpha,\beta,\gamma\} \subset (0,\pi)$. We can parameterize the probability distribution as:
$$ \begin{align*}
P(a < a_0) &= \frac{1-\cos a_0}{2} \\
P(b < b_0) &= \frac{1-\cos b_0}{2} \\
P(\gamma < \gamma_0) &= \frac{\gamma_0}{\pi}
\end{align*}$$
Variables $a$, $b$, and $\gamma$ are independent, so the overall probability density function is
$$ dP = \frac{1}{4\pi} \sin a \sin b\, da\, db\, d\gamma $$
The spherical area $\sigma$ of the triangle is
$$ \sigma = \alpha + \beta + \gamma – \pi $$
The outcome $\sigma > \pi$ is only possible if $\gamma > \frac{\pi}{2}$, since the spherical triangle covers a subset of the region between the planes containing $\{O,A,C\}$ and $\{O,A,B\}$ with area, $2 \gamma$. Similarly $\sigma > \pi$ implies $\alpha > \frac{\pi}{2}$ and $\beta > \frac{\pi}{2}$.
We also have Napier's analogy
$$ \tan \frac{\alpha+\beta}{2} = \frac{\cos \frac{a-b}{2}}{\cos \frac{a+b}{2}} \cot \frac{\gamma}{2} $$
Since $\{a,b\} \subset (0, \pi)$, $|a-b|<\pi$ and $\cos \frac{a-b}{2} > 0$. If $\sigma > \pi$ then $\{\alpha, \beta, \gamma\} \subset (\frac{\pi}{2}, \pi)$ which implies $\tan \frac{\alpha+\beta}{2} < 0$ and $\cot \frac{\gamma}{2} > 0$. Therefore $\cos \frac{a+b}{2} < 0$, and $\frac{a+b}{2} > \frac{\pi}{2}$.
So given $\{\alpha, \beta, \gamma, \frac{a+b}{2}\} \subset (\frac{\pi}{2}, \pi)$, all these inequalities are equivalent:
$$ \begin{align*}
\sigma &> \pi \\
\alpha + \beta + \gamma &> 2\pi \\
\pi – \frac{\gamma}{2} &< \frac{\alpha+\beta}{2} \\
\tan \left(\pi – \frac{\gamma}{2}\right) &< \tan \frac{\alpha+\beta}{2} \\
-\tan \frac{\gamma}{2} &< \frac{\cos \frac{a-b}{2}}{\cos \frac{a+b}{2}} \cot \frac{\gamma}{2} \\
\tan^2 \frac{\gamma}{2} &> – \cos \frac{a-b}{2}\, \sec \frac{a+b}{2} \\
\gamma &> 2\tan^{-1} \sqrt{- \cos \frac{a-b}{2}\, \sec \frac{a+b}{2}} = \gamma_L(a,b)
\end{align*}$$
where the function $\gamma_L(a,b)$ is defined where $0 \leq a \leq \pi$, $0 \leq b \leq \pi$, and $a+b>\pi$.
The probability in question is
\begin{align}P(\sigma > \pi) &= \int_0^\pi \int_{\pi-a}^\pi \int_{\gamma_L(a,b)}^\pi \frac{1}{4\pi} \sin a \sin b\, d\gamma\, db\, da\\&= \frac{1}{4\pi} \int_0^\pi \int_{\pi-a}^\pi [\pi – \gamma_L(a,b)] \sin a \sin b\, db\, da\end{align}
I've found we can also write
$$ \gamma_L(a,b) = \pi – \cos^{-1}\left(\cot \frac{a}{2} \cot \frac{b}{2}\right) $$
Either way, integration by parts can get rid of the inverse trig function in the integrand, leaving a square root, but I haven't been able to make much progress beyond that in solving the definite integral.
Wolfram Alpha gives the numeric answer as apparently $\frac{1}{6}$ (1) (2). But it seems it can't solve it symbolically either.
Given this was in a limited-time competition, maybe there's some other simpler way to go about this. Some sort of symmetry grouping related points or triangles? How can we prove the exact probability?
Best Answer
The substitution $u=\cot(a/2)\cot(b/2)$ yields $$I(a)=\int_{\pi-a}^\pi\arccos\left(\cot\frac a2\cot\frac b2\right)\sin b\,db=\int_0^1\frac{4u\tan^2(a/2)\arccos u}{(1+u^2\tan^2(a/2))^2}\,du$$ so that $$P(\sigma>\pi)=\frac1{4\pi}\int_0^\pi I(a)\sin a\,da=\frac4\pi\int_0^{\pi/2}J(a)\sin^2a\tan a\,da$$ where $\displaystyle J(a)=\int_0^1\frac{u\arccos u}{(1+u^2\tan^2a)^2}\,du$. Integration by parts yields \begin{align}J(a)&=-\frac{\arccos u}{2(1+u^2\tan^2a)\tan^2a}\bigg\vert_0^1-\frac1{2\tan^2a}\int_0^1\frac{du}{(1+u^2\tan^2a)\sqrt{1-u^2}}\\&=\frac\pi{4\tan^2a}-\frac1{2\tan^2a}\cdot\frac\pi{2\sqrt{\tan^2a+1}}\\&=\frac{\pi(1-\cos a)}{4\tan^2a}\end{align} where the last integral is evaluated by taking $t=u/\sqrt{1-u^2}$. Therefore, \begin{align}P(\sigma>\pi)=\int_0^{\pi/2}(1-\cos a)\sin a\cos a\,da=\frac16.\end{align} Not sure if there is a quicker geometrical answer.