Probability of an obtuse triangle in a circle.

geometryprobability

Suppose we randomly pick 2 points A, B within a circle centered
at point O. What is the probability that the triangle formed by ABO is
an obtuse one? (Note that A and B are not exclusively on the
circumference).

And what is the conclusion extended to A, B within a ball instead?

Thanks!

PS this is from a Quant interview.

The following is what I have derived during the exam: (Edited, thank you for your corrections!)
consider the joint probability of x, y coordinate for any point in a unit circle, then $f_{XY}(x, y) = \frac{1}{\pi}$, uniformly distributed inside the circular region. The distance between the point and the center, has thus a distribution $f_Z(z) = 2z$ for $z$ in [0, 1] ($z^2 = x^2 + y^2$).
Randomly pick an A, rotate the circle so that A is right on top of the center O. Suppose now A has a distance $z = a$ $(a > 0)$ away from the origin, then B could only be chosen in the region of

  1. $y > a$
  2. $y < 0$
  3. within an inner circle whose diameter is $OA$

Therefore, given that the distance is $z = a$, the probability of ABO being an obtuse triangle is given corresponds to area $\arccos(a) – a\sqrt{1-a^2} + \frac{a^2\pi}{4} + \frac{\pi}{2}$ (Upper, inner, and lower). By this conditional probability, we could derive the total probability, which is
$$
\int_0^1 \frac{1}{\pi}\left(\arccos(a) – a\sqrt{1-a^2} + \frac{a^2\pi}{4} + \frac{\pi}{2}\right) \cdot 2a \; da = \frac{3}{4}
$$

But is there an easier way to solve this? This looks like a math competition style of question and I expect some tricks to be at play. Thanks!

Best Answer

The probability that point $A$ is at a distance from the center in $[a,a+da]$ is $2\pi a\,da/\pi=2a\,da$. The probability that, fixed $A$ as above, $\angle ABO>90°$, is the same as the probability that $B$ lies inside a circle of diameter $OA$, that is $\pi(a/2)^2/\pi=a^2/4$. Hence the overall probability that $\angle ABO>90°$ is: $$ p(\angle ABO>90°)=\int_0^1 {a^2\over4}\cdot 2a\,da={1\over8}. $$

The probability that triangle $ABO$ is obtuse is then: $$ p(\angle ABO>90°)+p(\angle BAO>90°)+p(\angle AOB>90°) ={1\over8}+{1\over8}+{1\over2}={3\over4}. $$

EDIT.

For a 3D sphere one can repeat the same argument, obtaining: $$ p(\angle ABO>90°)=\int_0^1 {a^3\over8}\cdot 3a^2\,da={1\over16}. $$