This is terrible code for general-purpose use because it can give erroneous results or even fail altogether for short distances. Use the Haversine Formula instead.
(The formula on which your code is based converts two points on the sphere (not an ellipse) into their 3D Cartesian coordinates (xa,ya,za) and (xb,yb,zb) on the unit sphere and forms their dot product, which will equal the cosine of the angle between them. The ACos function returns that angle, which when scaled by the earth's radius will estimate the distance. The problem is that the cosine of a small angle, say of size 'e' in radians, differs from 1 by an amount close to e^2/2. This disappears into the floating point error cloud when e is smaller than the square root of twice the floating point precision. If you're computing in single precision this means values of e less than 0.001 -- about one kilometer -- will be confused with zero! In double precision the cutoff is around e = 10^-8, but by the time e = 10^-4 or so (about 10 meters) you potentially can lose so much precision that you need to worry, unless the implementation of the ACos function is unusually accurate (e.g., has some high-precision internal computations built in)).
I finally got there myself with help from @whuber. While @mkadunc shows how to ‘undo’ a rotation, where the rotated coordinate system is first rotated around the y'-axis and then the z'-axis to match a regular coordinate system, I was interested in performing a rotation from a regular grid; thus, the regular coordinate system shall first be rotated around the z-axis and then the y-axis.
Hence, when you calculate the product of:
x ( cos(φ), sin(φ), 0) ( cos(ϑ), 0, sin(ϑ)) (x')
y = (-sin(φ), cos(φ), 0).( 0 , 1, 0 ).(y')
z ( 0 , 0 , 1) ( -sin(ϑ), 0, cos(ϑ)) (z')
where the first matrix (A) represent the rotation around the z-axis and the second matrix (B) represent the rotation around the y-axis, A*B becomes:
x ( cos(ϑ)cos(φ) , -cos(ϑ)sin(φ) , -sin(ϑ)) (x')
y = ( sin(φ) , cos(φ) , 0 ).(y')
z ( sin(ϑ) cos(φ), -sin(ϑ) sin(φ), cos(ϑ)) (z')
which is indeed the inverse of BA, or (BA)^T since it's orthogonal.
In case anyone is interested I've shared a MATLAB script on the file exchange transforming regular lat/lon to rotated lat/lon and vice versa: Rotated grid transform
Best Answer
What you're looking for is the initial bearing (or forward azimuth), which if followed in a straight line along a great-circle arc will take you from the start point to the end point.
Here is some simple JavaScript from this link:
The above link has a wealth of useful information beyond this for related calculations.
As your question states, this is the simplest method - since the Earth is not a true sphere this calculation will not be 100% accurate, but it is a close approximation.