You asked for something that wasn't a proof or formal argument, so I hope this helps.
In any geometry, including non-Euclidean geometry (e.g. hyperbolic, or spherical geometry), "straight lines" are really called geodesics, which are defined to be the shortest line between two points. This means you stand somewhere holding one end of some rope, your friend stands somewhere else holding the other end, and together you pull the rope taut and this gives you your shortest path.
For example, say you're standing on the surface of a ball (i.e. a 2-sphere, such as the surface of the Earth), and your friend is some way away also on the surface, both of you holding the rope tight. This is spherical geometry. The taut rope or our "straight line" or geodesic is really the shortest path between us that lies on the surface, i.e. where the rope goes. This geodesic will look curved to someone in Euclidean space because there, the geodesic/"straight line" would pass through the ball.
Therefore it turns out that our definition of "straight" depends on the geometry we're using and how we pull the rope taut (the metric we use). It just so happens that in Euclidean geometry, this gives us lines that we call straight.
Interesting note: in other spaces, there are some super cool and peculiar metrics that make the taut rope (shortest path) go into weird shapes in Euclidean geometry! One example: https://en.wikipedia.org/wiki/Taxicab_geometry
I think a more fundamental way to approach the problem is by discussing geodesic curves on the surface you call home. Remember that the geodesic equation, while equivalent to the Euler-Lagrange equation, can be derived simply by considering differentials, not extremes of integrals. The geodesic equation emerges exactly by finding the acceleration, and hence force by Newton's laws, in generalized coordinates.
See the Schaum's guide Lagrangian Dynamics by Dare A. Wells Ch. 3, or Vector and Tensor Analysis by Borisenko and Tarapov problem 10 on P. 181
So, by setting the force equal to zero, one finds that the path is the solution to the geodesic equation. So, if we define a straight line to be the one that a particle takes when no forces are on it, or better yet that an object with no forces on it takes the quickest, and hence shortest route between two points, then walla, the shortest distance between two points is the geodesic; in Euclidean space, a straight line as we know it.
In fact, on P. 51 Borisenko and Tarapov show that if the force is everywhere tangent to the curve of travel, then the particle will travel in a straight line as well. Again, even if there is a force on it, as long as the force does not have a component perpendicular to the path, a particle will travel in a straight line between two points.
Also, as far as intuition goes, this is also the path of least work.
So, if you agree with the definition of a derivative in a given metric, then you can find the geodesic curves between points. If you define derivatives differently, and hence coordinate transformations differently, then it's a whole other story.
Best Answer
Every now and then it's nice to nuke a mosquito.
Let's assume that the path connecting two points $(a,y(a))$ and $(b,y(b))$ can be expressed as a function, and the curve $C(x)$ is given by $C(x) = (x,y(x))$. Then we will proceed using the Calculus of Variations.
The derivative of $C$ wrt $x$ is $(1, y')$, and the functional we want to minimize is the length of the curve $L = \int \|C'\|dx = \int_a^b\sqrt{1 + y' ^2} dx$. If we take $f(x,y,y') = \sqrt{1 + y'^2}$, we get that $\frac{df}{dy} = 0, \frac{df}{dy'} = \frac{y'}{\sqrt{1 + y'^2}}$. Then the Euler-Lagrange equation, sometimes referred to as the fundamental equation of the Calculus of Variations, says exactly that $\dfrac{d}{dx} \left( \frac{y'}{\sqrt{1 + y'^2}}\right) = 0$, which is exactly that $y'$ is a constant.
Thus, if the path connecting the two points is expressible as a function, then the shortest such path is given by a straight line.
EDIT I was certain that someone was in the middle of writing an answer when I typed my tongue-in-cheek response (as so often happens), but as I now see that there is more to add, allow me to extend my answer
The problem here is that we must first define "distance." In the standard Euclidean Plane, the distance between two points is defined to be the length of the line segment between them. So we can drop the word 'shortest' and say that "The distance between any two distant points is the length of the line segment joining them."
Presumably, you want to know that going along any other path will be at least as long. One way of 'seeing this' is that you can approximate any curve with a polygonal path, and these satisfy the triangle inequality, which will make the path longer.