Using the methodology described in FiveThirtyEight.com's "How We Found the Fastest Flights" in the footnote:
A simplified formula for target time is as follows:
.117*distance + .517*(longitude of origin – longitude of destination) + 43.2
The formula produces an estimated travel time in minutes. In the formula, distance is the great circle distance converted to miles between the origin and destination airports. The coefficient next to distance (.117) implies that flights travel at 513 miles per hour at cruising speeds.
Longitude origin and longitude destination represent the longitude of the airport of origin and destination, respectively. Each degree of longitude traversed westbound adds about 30 seconds to the flight time.
The constant term, 43.2 minutes, reflects the time airlines budget into their schedules for taxiing in and out of the airport, plus potentially inefficient routing.
The more precise version of target time reflects that degrees of longitude are further apart near the equator, and that the jet stream is slightly stronger at middle latitudes. It is calculated as follows:
.117*distance – .0255*westmiles+ .00144*westmiles*(90-latitude of midpoint)^2 – .0000146*westmiles*(90-latitude of midpoint)^2 + 43.2
In this formula, latitude of midpoint is the latitude at the midpoint of the journey (using a great circle route) and westmiles is the number of miles traveled westbound. Westmiles is calculated with this formula:
(longitude of origin – longitude of destination)*-.0073*(90 – latitude of midpoint)^2+1.424*(90 – latitude of midpoint)
Here's a link to my Google sheet where you can see the formulas in action, but I have also tried to explain it below.
The example in the methodology is San Francisco airport (SFO) to Los Angeles airport (LAX). The inputs were not provided so I have come up with the following using Google maps and the Great-Circle Distance calculator:
SFO latitude 37.6213171
SFO longitude -122.3811494
LAX latitude 33.9459649
LAX longitude -118.4061815
Latitude midpoint 35
distance (miles) 338
Longitude origin 122
Longitude destination 118
To calculate the variable westmiles:
(longitude of origin – longitude of destination)*-.0073*(90-latitude of midpoint)^2+1.424*(90 – latitude of midpoint)
Which I interpreted as:
(122 - 188)*-.0073*(90 - 35)^2 _ 1.424* (90 - 35)
and it equals -10 (I don't know if that is correct).
Putting the detailed formula together, I'm supposed to get 1 hour 19 min, but instead I got 40 min.
.117*distance - .0255*westmiles+ .00144*westmiles*(90-latitude of midpoint)^2 - .0000146*westmiles*(90-latitude of midpoint)^2 + 43.2
This is how I interpreted it:
.117 * 338 - .0255 * (-10) + .00144 * (-10) * (90 - 35)^2 - .0000146 * (-10) * (90 - 35)^2 + 43.2
I believe ^2 means to the power of two, but I could be wrong.
Can I get feedback where I went wrong? Or does someone have an alternative to this formula?
Best Answer
Make sure you are using negative longitudes in all your formulas, because your cities are in the Western Hemisphere. So in your example, Westmiles equals -223.5 which makes sense in this context.
Another useful formula for the calulation of east-west lengths at a specific latitude is the following: $$\left(\lambda_o - \lambda_d\right) r \text{ cos }\phi$$ Where $\lambda_o$ and $\lambda_d$ are your longitudes of origin and destination (in radians), $r$ is the Earth's radius and $\phi$ is the latitude.