[Math] How to find the distance between a point and line joining two points on a sphere

analytic geometryspherical coordinatesspherical-geometry

How do I calculate the distance between the line joining the two points on a spherical surface and another point on same surface? I have illustrated my problem in the image below.

Sphere

In the above illustration, the points A, B and X lies on a spherical surface, I need to find the distance between points (A,B) and X. I am not a mathematics guy. If possible please illustrate me the solution as non-mathematics guys could understand. Thanks.

Best Answer

This assumes that everything is on the surface of the sphere. Furthermore I assume the sphere has radius $1$.

Change the coordinates so that $A$ and $B$ are both on the equator of the sphere. For definiteness, move $A$ to $(1,0,0)$ and move $B$ to $(\cos \theta, \sin \theta, 0)$ where $\theta$ is the angle between the vectors from the center of the sphere to $A$ and to $B$. This is a linear transformation.

Then what you care about is the latitude of $X$. If $X$ is in the sector of the sphere immediately north or south of the line $AB$, then the answer is just $2\pi \phi$ where $\phi$ is the latitude of $X$. If $X$ is not in that sector then the answer is just the distance to either $A$ or $B$, whichever is closer.