The first application I was shown of the calculus of variations was proving that the shortest distance between two points is a straight line. Define a functional measuring the length of a curve between two points:
$$
I(y) = \int_{x_1}^{x_2} \sqrt{1 + (y')^2}\, dx,
$$
apply the Euler-Langrange equation, and Bob's your uncle.
So far so good, but then I started thinking: That functional was derived by splitting the curve into (infinitesimal) – wait for it – straight lines, and summing them up their lengths, and each length was defined as being the Euclidean distance between its endpoints*.
As such, it seems to me that the proof, while correct, is rather meaningless. It's an obvious consequence of the facts that (a) the Euclidean norm satisfies the triangle inequality and (b) the length of a curve was defined as a sum of Euclidean norms.
Getting slightly philosophical, I would conjecture that proving that the shortest distance between two points is a straight line is looking at things the wrong way round. Perhaps a better way would be to say that Euclidean geometry was designed to conform to our sensory experience of the physical world: the length of string joining two points is minimized by stretching the string, and at that point, it happens to look/feel straight.
I'm just wondering whether people would agree with this, and hoping that I may get some additional or deeper insights. Perhaps an interesting question to ask to try to go deeper would be: why does a stretched string look and feel straight?
*: To illustrate my point further, imagine we had chosen to define the length of a line as the Manhattan distance between its endpoints. We could integrate again, and this time it would turn out that the length of any curve between two points is the Manhattan distance between those points.
Best Answer
I think a more fundamental way to approach the problem is by discussing geodesic curves on the surface you call home. Remember that the geodesic equation, while equivalent to the Euler-Lagrange equation, can be derived simply by considering differentials, not extremes of integrals. The geodesic equation emerges exactly by finding the acceleration, and hence force by Newton's laws, in generalized coordinates.
See the Schaum's guide Lagrangian Dynamics by Dare A. Wells Ch. 3, or Vector and Tensor Analysis by Borisenko and Tarapov problem 10 on P. 181
So, by setting the force equal to zero, one finds that the path is the solution to the geodesic equation. So, if we define a straight line to be the one that a particle takes when no forces are on it, or better yet that an object with no forces on it takes the quickest, and hence shortest route between two points, then walla, the shortest distance between two points is the geodesic; in Euclidean space, a straight line as we know it.
In fact, on P. 51 Borisenko and Tarapov show that if the force is everywhere tangent to the curve of travel, then the particle will travel in a straight line as well. Again, even if there is a force on it, as long as the force does not have a component perpendicular to the path, a particle will travel in a straight line between two points.
Also, as far as intuition goes, this is also the path of least work.
So, if you agree with the definition of a derivative in a given metric, then you can find the geodesic curves between points. If you define derivatives differently, and hence coordinate transformations differently, then it's a whole other story.