I found this answer that stated the following theorem –
Theorem: Let 𝑓 be a real valued function defined in a neighbourhood of point 𝑎 and continuous at 𝑎 and lets assume that it is approximated by a linear function 𝑔 given by 𝑔(𝑥)=𝐴𝑥+𝐵 in the neighbourhood of 𝑎. Then we say that 𝑔 is best linear approximation of 𝑓 in the neighbourhood of 𝑎 if the following equation holds :
$$ \lim_{x\to a} \frac{f(x)-g(x)}{x-a}=0$$
Such a linear approximation exists if and only if 𝑓′(𝑎) exists and moreover in that case we have 𝑔(𝑥)=𝑓(𝑎)+𝑓′(𝑎)(𝑥−𝑎).
This answer also uses this theorem to prove that the derivative is truly the best linear approximation. More like this is the 'sense' in which it is the best approximation.
After researching online I found that the idea seems to be that the derivative is the only linear approximation for which the approximation error tends to $0$ faster than $𝑥-𝑎$ as $𝑥→𝑎$, and based on this we call it the best approximation.
My question is, how does this actually prove that the derivative will beat any other linear approximation? How does it formally (if possible intuitively also) prove that the derivative is better than all the other approximations.
Best Answer
Let $f$ your function, $f_1 : x \rightarrow a(x-x_0) + b$ the approximation given by the derivative, and $f_2 : x \rightarrow a'(x-x_0) + b'$ some other approximation. The approximations are done in a neighborhood of $x_0$.
Of course, if $b' \neq b$, $f_2$ is a bad approximation since $f(x_0) \neq f_2(x_0)$.
Let us suppose $a \neq a'$.
You have $$\lim_{x \rightarrow x_0} \frac{f(x) - f_1(x)}{x - x_0} = 0$$. And : $$\frac{f(x) - f_2(x)}{x-x_0} = \frac{f(x) - f_1(x)}{x-x_0} + \frac{f_1(x) - f_2(x)}{x-x_0} = \frac{f(x) - f_1(x)}{x-x_0} + a - a'$$
In such a way that :
$$\lim_{x \rightarrow x_0} \frac{f(x) - f_2(x)}{x - x_0} = a - a'$$
This enables to show that (take the quotient of the two preceding limits) :
$$ \lim_{x \rightarrow x_0} \frac{f(x) - f_1(x)}{f(x) - f_2(x)} = 0$$
This last equality encapsulates the intuition that the (local at $x_0$) approximation $f_1$ is far better than $f_2$. If you write it with the epsilons : for all $\epsilon > 0$ there exists $\delta > 0$ such that for $x \in ]x_0 - \delta, x_0 + \delta[$, $|f(x) - f_1(x)| \leq \epsilon | f(x) - f_2(x) |$
EDIT : The last equality is also true if $b' \neq b$ (to be more precise, it is "more true", since in this case, $\lim_{x \rightarrow x_0} \frac{f(x) - f_2(x)}{x- x_0} = +\infty$). It justifies the nomination "best approximation" for the approximation given by the derivative : it is far better than any other.