Assume that I transformed geodetic coordinates (lat/lon) to Cartesian (x/y/z) with respect to a spherical earth model with radius R.
Now, I compute the distance between two points P1 and P2 as D.
How can I compute the great circle distance (arc length) between P1 and P2 from D?
Best Answer
If you compute the dot product, you can have the cosine of the angle between $P1$ and $P2$.
From that, multiply the radius of the Earth $R$ by the angle to get the arc length.
Assuming that both points are at the surface of the Earth and that we can neglect the effect of different altitudes and that the Earth is a sphere, we have that the angle \alpha is: $$ \alpha = \arccos \left(\frac{\vec{P1} \cdot \vec{P2}}{R^2} \right) $$
And the arc length is: $$ \alpha R $$
Edit:
Using the straight line distance $D$ you may also compute the arc length by imagining the triangle rectangle with hypotenuse equal to the radius of the Earth $R$, and catheti: one equal to the half of the distance $D$ and the other linking the middle point between $P1$ and $P2$ and the centre of the Earth.
Then, the semi-angle $\beta = \alpha/2$ is: $$ \beta = \arcsin \left( \frac{D/2}{R} \right) $$
And the arc length is: $$ 2 \beta R = \alpha R $$