As I can see from your edit, you noticed that the only difficult part is to show that $\exp'(0) = 1$, that is, $\lim_{h\to 0}\left({\dfrac{\exp(h)-1}{h}}\right) = 1$.
So here's my proof, using only the definition of the exponential function and elementary properties of limits.
We use the following definition of the exponential function:
\begin{align*}
&\exp : \mathbb{R} \to \mathbb{R}\\
&\exp(x) = \lim_{k \to +\infty} \left(1 + \frac{x}{k}\right)^k
\end{align*}
Let's define
\begin{align*}
&A : \mathbb{R}^* \to \mathbb{R}\\
&A(h) = \frac{\exp(h) - 1}{h} - 1
\end{align*}
We're going to show that $\lim_{h \to 0} A(h) = 0$. This will imply that $\lim_{h \to 0} \frac{\exp(h) - 1}{h} = 1$
and consequently, that $\exp'(0) = 1$.
Let's show that for all $h \in [-1,1]\setminus\{0\}$, $|A(h)| \leq |h|$
Let $h \in [-1,1]\setminus\{0\}$. We define the sequence $(u_k)_{k \in \mathbb{N}^*}$ by
\begin{align*}
u_k = \frac{\left(1+\frac{h}{k}\right)^k - 1}{h} - 1
\end{align*}
From the definition of the exponential function and from the rules of addition and multiplication of limits, we get:
\begin{align*}
\lim_{k \to + \infty} u_k = A(h)
\end{align*}
The continuity of the absolute value function then brings:
\begin{align*}
\lim_{k \to + \infty} |u_k| = |\lim_{k \to + \infty} u_k| = |A(h)|
\end{align*}
If we manage to show that after a certain rank, $|u_k| \leq |h|$, we'll be able to conclude that
$|A(h)| = \lim_{k \to +\infty}|u_k| \leq |h|$.
For $k \in \mathbb{N}^*$, we have
$$
u_k = \frac{\left(\sum\limits_{i=0}^{k} \binom{k}{i} \left(\frac{h}{k}\right)^i \right) - 1 - h}{h}
= \frac{1}{h}\sum_{i=2}^k \frac{\binom{k}{i}}{k^i} h^i = h \sum_{i=2}^k \frac{\binom{k}{i}}{k^i} h^{i-2}
$$
The triangle inequality brings:
$$
|u_k| \leq |h| \sum_{i=2}^k \frac{\binom{k}{i}}{k^i} |h|^{i-2}
$$
We have $h \in [-1,1]$. So $|h|^{i-2} \leq 1$ for every $i \in \mathbb{N}$ such as $i \geq 2$.
Moreover, for $k,i \in \mathbb{N}\setminus\{0,1\}$:
$$
\frac{\binom{k}{i}}{k^i}
= \frac{\prod\limits_{j=0}^{i-1}(k-j)}{i!\prod\limits_{j=0}^{i-1}k}
\leq \frac{1}{i!} \leq \frac{1}{2^{i-1}}
$$
Therefore, as soon as $k \geq 2$:
$$
|u_k|
\leq |h| \sum_{i=2}^k \frac{1}{2^{i-1}}
= |h| \sum_{i=1}^{k-1} \frac{1}{2^i}
= |h| \left(\frac{1-\frac{1}{2^k}}{1-\frac{1}{2}} - 1\right)
= |h| \left(1-\frac{1}{2^{k-1}}\right)
\leq |h|
$$
Hence:
$$
|A(h)| = \lim_{k \to \infty} |u_k| \leq |h|
$$
This is true for all $h \in [-1,1]\setminus\{0\}$.
Therefore:
$$
\lim_{h \to 0} A(h) = 0
$$
which shows that $\exp'(0) = 1$.
The expression we are interested in is:
$\displaystyle \lim_{x\to a} \frac{f(x)-f(a)}{x-a}$
In a comment you talk about "applying the limit laws first". Well, let's try and see how far we get.
Sure, by continuity, and the limit law that "the difference of two limits is the limit of the difference", we have $\displaystyle \lim_{x\to a} (f(x)-f(a)) =0$.
Even easier, we have $\displaystyle \lim_{x\to a} (x-a) =0$.
But now what? It seems as if you want to apply the limit law that "a quotient of limits is the limit of the quotient", or
If $R:=\displaystyle \lim_{x\to a} r(x)$ and $S:=\displaystyle \lim_{x\to a} s(x)$ exist, then the limit $\lim_{x\to a}\dfrac{r(x)}{s(x)}$ also exists and is equal to $R/S$.
Well, if stated like this, this limit law is wrong, and if you look closely in your lecture notes or textbooks, you will see (I hope) that instead, it is only true under a special assumption on $S$. Do you see what is this extra condition?
In other words, thinking you can evaluate $$\displaystyle \lim_{x\to a} \frac{f(x)-f(a)}{x-a}$$ by evaluating $$\displaystyle \frac{\lim_{x\to a} (f(x)-f(a))}{\lim_{x\to a} (x-a)}$$ is exactly the wrong step you are looking for which is "inherently missing some crucial information".
The extra condition on the denominator we would need for that manipulation is exactly not satisfied in your case, and so you cannot apply that "law". The limit of the quotient has to be computed another way, and no contradiction arises.
Best Answer
Note that we have $$\frac{b^h-1}{h}=\frac{e^{h\log(b)}-1}{h} \tag 1$$
Now, in THIS ANSWER, I showed using only the limit definition of the exponential function and Bernoulli's Inequality that the exponential function satisfies the inequality
$$e^x\ge 1+x \tag 2$$
From $(2)$ (along with the property $e^xe^{-x}=1$) it is easy to see that for $x<1$
$$e^x\le \frac{1}{1-x} \tag 3$$
Using $(2)$ and $(3)$, we can bound $(1)$ as
$$\log(b) \le \frac{e^{h\log(b)}-1}{h}\le \frac{\log(b)}{1-h\log(b)}$$
whereupon application of the squeeze theorem yields the coveted limit
$$\lim_{h\to 0}\frac{b^h-1}{h}=\log(b)$$
And we are done!