Linear splines are easy to discuss. Knots are where the slopes change, and only one level of continuity is enforced.
When discussing cubic splines (with the usual 3 levels of continuity) or natural cubic splines (linear tail restricted cubic splines) I often speak loosely as "a knot is where a curvature change happens" or where a "shape change happens". For functions, the formal definition of curvature has the second derivative dominating the calculation, and so formally speaking the curvature is changing everywhere in a cubic spline function.
What is the best language to use? Should we say "the knots are where we allow shape changes"? Where we allow rapid shape changes? Or is it best to be this explicit: the knots are points where there are changes in jolt (jerk; 3rd derivative) of the function? Other ideas?
Based on comments below, the best language I can think of at the moment is the following, for a general audience:
Knots are where different cubic polynomials are joined, and cubic splines force there to be three levels of continuity (the function, its slope, and its acceleration or second derivative (slope of the slope) do not change) at these points. At the knots the jolt (third derivative or rate of change of acceleration) is allowed to change suddenly, meaning the jolt is allowed to be discontinuous at the knots. Between knots, jolt is constant.
Alternative version:
Knots are where cubic polynomials are joined, and continuity restrictions make the joins invisible. The function, its slope, and its acceleration (slope of slope; second derivative) do not change at a knot. But the rate of change of the acceleration (jolt; third derivative) is allowed to change abruptly at a knot.
It would still be nice if there is a simpler term not unlike "shape change" that could be used to describe this in addition to the technically correct wording.
Update: I added a graph showing a cubic spline function and its first three derivatives in Section 2.4.4 of my Regression Modeling Strategies course notes.
Best Answer
This answer on math.stackexchange.com suggests one way to proceed. In particular:
Starting from the limitations of fitting an infinitely mathematically smooth (in the sense of infinite differentiability) high-degree polynomial might help heuristically. I'd suggest something like:
If there is time, an illustration like the following based on Runge's phenomenon might help.
Consider the following 9 points joined with straight lines:
We want to fit a smooth curve to these points, to avoid the sharp changes in the line at the points. We could try to fit a curve that is infinitely smooth mathematically through these points, in the sense that not only is the curve continuous, but the slope of the curve is continuous, as is the slope of the slope, and so on forever (infinite differentiability). Polynomials are infinitely smooth in that sense, but here's what you get if you fit a polynomial through these points:
As @bubba has put it about the high-degree polynomials needed for this type of fitting:
If we remove the requirement for infinite mathematical smoothness at the knots, however, we can do much better. Then we can use an infinitely smooth 3rd-degree polynomial between each pair of adjacent knots, and at the knots require just the minimum smoothness needed to make the joins invisible:
where the orange line is a cubic spline fit and the blue line shows the smooth Runge function from which the points were sampled. This approach provides "the least possible amount of wiggling in between" the knots and thus meets an intuitive sense of "smoothness."