The following is a proof of the chain rule from John K. Hunter's lecture notes. It uses the little-oh concept, i.e. a function $f(h)$ is little-oh of $h$ if $\lim_{h\to 0} f(h)/h=0$. Come to think of it, the proof below answer's this question. Anyway, I have only one question about the proof at the very end.
The proof uses the following proposition:
Proposition 8.11. Suppose that $f: (a,b)\to \mathbb{R}$. Then $f$ is differentiable at $c\in (a,b)$ if and only if there exists a constant $A\in\mathbb{R}$ and a function $r: (a-c,b-c)\to\mathbb{R}$ such that $$f(c+h)=f(c)+Ah+r(h), \quad \lim_{h\to 0}\frac{r(h)}{h}=0.$$ In that case, $A=f'(c)$.
Theorem 8.21 (Chain rule). Let $f: A\to\mathbb{R}$ and $g: B\to\mathbb{R}$ where $A\subset\mathbb{R}$ and $f(A)\subset B$ and suppose that $c$ is an interior point of $A$ and $f(c)$ is an interior point of $B$. If $f$ is differentiable at $c$ and $g$ is differentiable at $f(c)$, then $g\circ f: A\to\mathbb{R}$ is differentiable at $c$ and $$(g\circ f)'(c)=g'(f(c))f'(c).$$
In the proof below, I think there is a typo in the tagged equation (it should be = instead of <).
Proof. Since $f$ is differentiable at $c$, there is a function $r(h)$ such that
$$
f(c+h)=f(c)+f^{\prime}(c) h+r(h), \quad \lim _{h \rightarrow 0} \frac{r(h)}{h}=0,
$$
and since $g$ is differentiable at $f(c)$, there is a function $s(k)$ such that
$$
g(f(c)+k)=g(f(c))+g^{\prime}(f(c)) k+s(k), \quad \lim _{k \rightarrow 0} \frac{s(k)}{k}=0 .
$$
It follows that
$$
\begin{aligned}
(g \circ f)(c+h) & =g\left(f(c)+f^{\prime}(c) h+r(h)\right) \\
& =g(f(c))+g^{\prime}(f(c)) \cdot\left(f^{\prime}(c) h+r(h)\right)+s\left(f^{\prime}(c) h+r(h)\right) \\
& =g(f(c))+g^{\prime}(f(c)) f^{\prime}(c) \cdot h+t(h)
\end{aligned}
$$
where
$$
t(h)=g^{\prime}(f(c)) \cdot r(h)+s(\phi(h)), \quad \phi(h)=f^{\prime}(c) h+r(h) .
$$
Since $r(h) / h \rightarrow 0$ as $h \rightarrow 0$, we have
$$
\lim _{h \rightarrow 0} \frac{t(h)}{h}=\lim _{h \rightarrow 0} \frac{s(\phi(h))}{h} .
$$
We claim that this limit exists and is zero, and then it follows from Proposition 8.11 that $g \circ f$ is differentiable at $c$ with
$$
(g \circ f)^{\prime}(c)=g^{\prime}(f(c)) f^{\prime}(c) .
$$
To prove the claim, we use the facts that
$$
\frac{\phi(h)}{h} \rightarrow f^{\prime}(c) \quad \text { as } h \rightarrow 0, \quad \frac{s(k)}{k} \rightarrow 0 \quad \text { as } k \rightarrow 0 .
$$
Roughly speaking, we have $\phi(h) \sim f^{\prime}(c) h$ when $h$ is small and therefore
$$
\frac{s(\phi(h))}{h} \sim \frac{s\left(f^{\prime}(c) h\right)}{h} \rightarrow 0 \quad \text { as } h \rightarrow 0 .
$$
In detail, let $\epsilon>0$ be given. We want to show that there exists $\delta>0$ such that
$$
\left|\frac{s(\phi(h))}{h}\right|<\epsilon \quad \text { if } 0<|h|<\delta .
$$
First, choose $\delta_{1}>0$ such that
$$
\left|\frac{r(h)}{h}\right|<\left|f^{\prime}(c)\right|+1 \quad \text { if } 0<|h|<\delta_{1}
$$
If $0<|h|<\delta_{1}$, then
$$
\begin{align}
|\phi(h)| & \leq\left|f^{\prime}(c)\right||h|+|r(h)| \\
& <\left|f^{\prime}(c)\right||h|+\left(\left|f^{\prime}(c)\right|+1\right)|h| \\
& <\left(2\left|f^{\prime}(c)\right|+1\right)|h| \tag{*}.
\end{align}
$$
Next, choose $\eta>0$ so that
$$
\left|\frac{s(k)}{k}\right|<\frac{\epsilon}{2\left|f^{\prime}(c)\right|+1} \quad \text { if } 0<|k|<\eta
$$
(We include a "1" in the denominator on the right-hand side to avoid a division by zero if $f^{\prime}(c)=0$.) Finally, define $\delta_{2}>0$ by
$$
\delta_{2}=\frac{\eta}{2\left|f^{\prime}(c)\right|+1}
$$
and let $\delta=\min \left(\delta_{1}, \delta_{2}\right)>0$.If $0<|h|<\delta$ and $\phi(h) \neq 0$, then $0<|\phi(h)|<\eta$, so
$$
|s(\phi(h))| \leq \frac{\epsilon|\phi(h)|}{2\left|f^{\prime}(c)\right|+1}<\epsilon|h| .
$$
If $\phi(h)=0$, then $s(\phi(h))=0$, so the inequality holds in that case also. This proves that
$$
\lim _{h \rightarrow 0} \frac{s(\phi(h))}{h}=0 .
$$
If $\phi(h)= 0$, then $s(\phi(h))=0$. Why? I have a far-fetched idea. The author states that we have $ \frac{s\left(f^{\prime}(c) h\right)}{h} \rightarrow 0 $ as $h \rightarrow 0$. If $o(h)$ denotes all functions that approach zero faster than $h$ as $h\to 0$, and if $s(h)\in o(h)$, then by limit substitution also $s(ah)\in o(h)$, where $a$ is a constant. However, what happens when $a=f'(c)=0$, as presumably can be the case. Then does $ \frac{s\left(f^{\prime}(c) h\right)}{h} \rightarrow 0 $ when $h\to 0$? Only if $s(0)=0$. I don't know, maybe there's a simpler explanation.
Best Answer
Yes that seems to be a careless typo in the text you flagged.
I believe that the conditions imposed on $r(h)$ in Proposition 8.11 ought to have stated explicitly that $r(h)$ is continuous as $h\to 0$. Then it is evident that $r(0)$ must be zero. The same applies to the expression $s(k)$.
There is a much cleaner conceptual approach to the definition of the derivative that was developed by Caratheodory about 100 years ago (based upon the simpler concept of continuity) . See e.g Question about Caratheodory's approach and the reference to Bernstein' notes cited therein. The Chain Rule can be proved in one line using this approach.
A nice expository article explores this in detail:
The Derivative á la Carathéodory Stephen Kuhn The American Mathematical Monthly, Vol. 98, No. 1 (Jan., 1991), pp. 40-44 (5 pages)
https://www.jstor.org/stable/2324035
Quoting from its introduction:
On the Differentiability of Functions of Several Variables Michael W. Botsko, Richard A. Gosser The American Mathematical Monthly, Vol. 92, No. 9 (Nov., 1985), pp. 663-665 (3 pages)
https://www.jstor.org/stable/2323717