Instead of putting so many long comments I thought it would be better to make a small answer. We start with a point $a$ and let us keep notation $(f\circ g)(x) = f(g(x))$. Then we have $$(f\circ g)'(a) = \lim_{x \to a}\frac{f(g(x)) - f(g(a))}{x - a}$$ We are given that $$\lim_{y \to g(a)}\frac{f(y) - f(g(a))}{y - g(a)} = f'(g(a)) = A, \lim_{x \to a}\frac{g(x) - g(a)}{x - a} = g'(a) = B$$ We need to show that $(f\circ g)'(a) = AB$. Clearly we need to distinguish two cases:
1) There is a neighborhood of $a$ in which $g(x) \neq g(a)$ when $x \neq a$. In this case we have $$(f\circ g)'(a) = \lim_{x \to a}\frac{f(g(x)) - f(g(a))}{x - a} = \lim_{x \to a}\frac{f(g(x)) - f(g(a))}{g(x) - g(a)}\cdot\frac{g(x) - g(a)}{x - a} = AB$$
2) Every neighborhood of $a$ contains infinitely many points $x\neq a$ such that $g(x) = g(a)$. Hence it is possible to find a sequence $x_{n} \to a$ such that $x_{n} \neq a$ and $g(x_{n}) = g(a)$. Now limit $B$ exists and hence $$B = \lim_{x \to a}\frac{g(x) - g(a)}{x - a} = \lim_{n \to \infty}\frac{g(x_{n}) - g(a)}{x_{n} - a} = 0$$ Now we need to show that $(f\circ g)'(a) = AB = 0$. Consider the ratio $$F(x, a) = \frac{f(g(x)) - f(g(a))}{x - a}$$ if $g(x) = g(a)$ then $F(x, a) = 0$. If $g(x) \neq g(a)$ then we can write $$F(x, a) = \frac{f(g(x)) - f(g(a))}{g(x) - g(a)}\cdot\frac{g(x) - g(a)}{x - a}$$ and the first factor is near $A$ and second factor is near $B = 0$. So effectively if we have a sufficiently small neighborhood of $a$ then we either have $F(x, a) = 0$ or $F(x, a)$ is very small. Using $\epsilon, \delta$ argument we can show that for any $\epsilon > 0$ there is a $\delta > 0$ such that $|F(x, a)| < \epsilon$ whenever $0 < |x - a| < \delta$. This shows that $\lim_{x \to a}F(x, a) = 0$ i.e. $(f\circ g)'(a) = 0$ as was to shown.
To understand this, you need to think of the intuition behind the $\epsilon$-$\delta$ definition. We want $\lim_{x\to a}f(x)=L$ if we can make $f(x)$ as close to $L$ as we like by making $x$ sufficiently close to $a$. Worded differently, we might say that:
$\lim_{x\to a}f(x)=L$ if given any neighborhood $U$ of $L$, there is a neighborhood $V$ of $a$ such that elements of $V$ are mapped by $f$ to elements of $U$ (except possibly $a$ itself).
In this context, a "neighborhood" of a point $p$ should be understood to mean "points sufficiently close to $p$". Let's make that precise by defining what we mean by "close". For $\epsilon>0$ (assumed, but not required, to be very small) define
$$B(x,\epsilon):=\{y\,:\,|x-y|<\epsilon\},$$
the ball of radius $\epsilon$ about $x$. For our purposes, we say $U$ is a neighborhood of $x$ if $U=B(x,\epsilon)$ for some $\epsilon>0$. (The usual definition only requires that $U$ contains such a ball.) Assuming $\epsilon>0$ is very small, this agrees with our intuition of what closeness should mean. Now if we go back to our neighborhood "definition" of a limit, you should be able to think about it for a bit and convince yourself that it is equivalent to the usual definition.
How does this relate to the problem with infinity? Given that infinity is not a real number (and things like distance from infinity do not make sense), we must revise what it means to be "close" to infinity. So for $M>0$ (assumed this time to be very large) define
$$B(+\infty,M):=\{y\,:\,y>M\},\quad B(-\infty,M):=\{y\,:\,y<-M\},$$
the neighborhoods of $\pm\infty$. Hopefully you can see why these make sense as definitions; a number should be close to infinity if it is very large (with the correct sign), so a neighborhood of infinity should contain all sufficiently large numbers.
Now we extend our neighborhood definition of limits to include the case where $a$ or $L$ can be $\pm\infty$. It is a similar exercise to before to verify now that the definition is still equivalent to the old one, only now we have in some sense unified somewhat.
Best Answer
Claim: Suppose $f$ and $g$ are functions such that $\lim_{x \rightarrow a} f(x) = L_1$ and $\lim_{y \rightarrow L_1} g(y) = L_2$ then $\lim_{x \rightarrow a} g ( f(x)) = L_2$.
Proof: Let $\epsilon > 0$ and choose $\delta >0$ such that if $0<|x-a|<\delta$ then $|f(x)-L_1|< \delta_2$ where $\delta_2 >0$ is small enough to force $|g(y)-L_2| < \epsilon$ for all $y \in \mathbb{R}$ such that $0 < |y-L_1| < \delta_2$.
We can choose $\delta_2 > 0$ as above because we were given that $\lim_{y \rightarrow L_1} g(y) = L_2$. Further, we can also choose $\delta >0$ to force $|f(x)-L_1| < \delta_2$ because we were also given that $\lim_{x \rightarrow a} f(x) = L_1$.
Suppose that $x \in \mathbb{R}$ such that $0 < |x-a| < \delta$ and observe that $|g(f(x))-L_2 | < \epsilon$. Therefore, by the definition of the limit, $\lim_{x \rightarrow a} g ( f(x)) = L_2$. $\Box$
Thanks to Vim for correcting comment. Let me attempt a modified proof, it may be useful to locate the error in the logic above,
Modified Claim: Suppose $f$ and $g$ are continuous functions such that $\lim_{x \rightarrow a} f(x) =L_1$ and $\lim_{y \rightarrow L_1} g(y) = L_2 = g(L_1)$ then $\lim_{x \rightarrow a} g ( f(x)) = L_2$.
Modified Proof: Since $\lim_{x \rightarrow a} f(x) = f(a)=L_1$ it follows that for each $\delta_2 >0$ there exists $\delta>0$ such that $0 < |x-a|< \delta$ implies $|f(x)-L_1| < \delta_2$.
Let $\epsilon>0$ and pick $\delta_2>0$ such that $0< |y-L_1|< \delta_2$ implies $|g(y)-L_2| < \epsilon$. This choice of $\delta_2$ is possible since we are given that $\lim_{y \rightarrow L_1} g(y) = L_2$.
Suppose that $x \in \mathbb{R}$ such that $0 < |x-a| < \delta$ implies $|f(x)-L_1| < \delta_2$. Thus, for $y=f(x)$,we have $|y-L_1|< \delta_2$. We don't quite have what is needed to conclude just yet since we need $0<|y-L_1| < \delta_2$ in order to conclude $|g(y)-L_2|=|g(f(x))-L_2 | < \epsilon$. Consider two cases:
Hence, in all cases possible,$0 < |x-a| < \delta$ implies $|g(f(x))-L_2|< \epsilon$. Therefore, by the definition of the limit, $\lim_{x \rightarrow a} g ( f(x)) = L_2$. $\Box$
Of course, we can also state this result as it is often applied: the continuity of the outer function allows us to pull the limit inside out: $$ \lim_{x \rightarrow a} g(f(x)) = g \left( lim_{x \rightarrow a} f(x) \right)$$ where once again I should emphasize, the continuity of $g$ at $lim_{x \rightarrow a} f(x)$ is assumed.