I have heard of functions being Lipschitz Continuous several times in my classes yet I have never really seemed to understand exactly what this concept really is.
Here is the definition.
$\left | f(x_{1})-f(x_{2}) \right |\leq K\left | x_{1}-x_{2} \right |$
Here is the function I'm using. It is known that this is Lipschitz Continuous.
$f(x)=\sqrt{x^2+5}$
If you pick some points. Here I picked (1, 0.408) and (2, 0.66).
The result is:
$\left | 0.252 \right |\leq K\left | 1 \right |$
So as long as K is 0.252 or bigger then this function is Lipschitz Continuous?
What if I pick K to be 0.0001 is the function no longer Lipschitz Continuous?
To me this is hard to understand, why not always pick K to be very large such that the function is always Lipschitz Continuous?
Unless the left hand side of the inequality is infinity, can't you always find a K big enough to satisfy this inequality?
Best Answer
Lipschitz continuity doesn't say that if you take any old $x,y$ and plot $|f(x)-f(y)|\leq M|x-y|$. It says that $M$ is fixed and the inequality holds for all $x,y \in \mathbb R$ . This is much stronger of a condition. If all you needed to do was pick some $M$ for every choice of $x,y$, the condition would mean nothing.
If your function is differentiable, Lipschitz continuity just says that the function has bounded derivative. I think of this as a "wiggling" and "stretching" bound.
For the stretching, it says that a function $f:\mathbb R \to \mathbb R$ can't grow too fast. For example, if the Lipschitz constant were $1$ and $f(0)=0$, then it is trapped between the two lines $y=x-1,x+1$. This essentially means that it can't grow too quickly, or wiggle too much.
If the constant $M <1$, a function is said to be a contraction which has many nice properties as well.
--