The only problem is that you’re looking at the wrong three points: you’re looking at $x+2h,x+h$, and $x$, and the version that you want to prove is using $x+h,x$, and $x-h$. Start with $$f\,''(x)=\lim_{h\to 0}\frac{f\,'(x)-f\,'(x-h)}h\;,$$ and you’ll be fine.
To see that this really is equivalent to looking at $$f\,''(x)=\lim_{h\to 0}\frac{f\,'(x+h)-f\,'(x)}h\;,$$ let $k=-h$; then
$$\begin{align*}
f\,''(x)&=\lim_{h\to 0}\frac{f\,'(x)-f\,'(x-h)}h\\
&=\lim_{-k\to0}\frac{f\,'(x)-f\,'(x-(-k))}{-k}\\
&=\lim_{k\to 0}\frac{f\,'(x-(-k))-f\,'(x)}k\\
&=\lim_{k\to 0}\frac{f\,'(x+k)-f\,'(x)}k\;,
\end{align*}$$
and renaming the dummy variable back to $h$ completes the demonstration.
One needs to establish the following two properties:
1) $\displaystyle \lim_{x \to 0}\frac{e^{x} - 1}{x} = 1$
2) $\displaystyle e^{x + y} = e^{x}\cdot e^{y}$
and it turns out that both of these can be derived (albeit with some minor difficulty) using the definition $$e^{x} = \lim_{n \to \infty}\left(1 + \frac{x}{n}\right)^{n}$$
We start with 1) first and that too with limitation $x \to 0+$. We have
$\displaystyle \begin{aligned}\lim_{x \to 0+}\frac{e^{x} - 1}{x} &= \lim_{x \to 0+}\dfrac{{\displaystyle \lim_{n \to \infty}\left(1 + \dfrac{x}{n}\right)^{n} - 1}}{x}\\
&= \lim_{x \to 0+}\lim_{n \to \infty}\frac{1}{x}\left\{\left(1 + \dfrac{x}{n}\right)^{n} - 1\right\}\\
&= \lim_{x \to 0+}\lim_{n \to \infty}\frac{1}{x}\left\{\left(1 + x + \dfrac{(1 - 1/n)}{2!}x^{2} + \dfrac{(1 - 1/n)(1 - 2/n)}{3!}x^{3} + \cdots\right) - 1\right\}\\
&= \lim_{x \to 0+}\lim_{n \to \infty}\left(1 + \dfrac{(1 - 1/n)}{2!}x + \dfrac{(1 - 1/n)(1 - 2/n)}{3!}x^{2} + \cdots\right)\\
&= \lim_{x \to 0+}\lim_{n \to \infty}(1 + \phi(x, n))\end{aligned}$
where $\phi(x, n)$ is a finite sum defined by $$\phi(x, n) = \frac{(1 - 1/n)}{2!}x + \cdots + \frac{(1 - 1/n)(1 - 2/n)\cdots(1 - (n - 1)/n)}{n!}x^{n - 1}$$ For fixed positive $x$ the function $\phi(x, n)$ is a increasing sequence bounded by the convergent series $$F(x) = \frac{x}{2!} + \frac{x^{2}}{3!} + \cdots$$ Hence the limit $\lim_{n \to \infty}\phi(x, n)$ exists and let say it is equal to $\phi(x)$. Then $0 \leq \phi(x) \leq F(x)$. Now let $x < 2$ and then we can see that $$F(x) \leq \frac{x}{2} + \frac{x^{2}}{2^{2}} + \frac{x^{3}}{2^{3}} + \cdots = \frac{x}{2 - x}$$ Hence $\lim_{x \to 0+}F(x) = 0$ and therefore $\lim_{x \to 0+}\phi(x) = 0$.
We now have
$\displaystyle \begin{aligned}\lim_{x \to 0+}\frac{e^{x} - 1}{x} &= \lim_{x \to 0+}\lim_{n \to \infty}1 + \phi(x, n)\\
&= \lim_{x \to 0+}1 + \phi(x)\\
&= 1 + 0 = 1\end{aligned}$
From this it follows that $\lim_{x \to 0+}e^{x} = 1$. To handle the case for $x \to 0-$ we need to use another trick. We show that for $x > 0$ we have $$\lim_{n \to \infty}\left(1 - \frac{x}{n}\right)^{-n} = e^{x}$$ Clearly we have
$\displaystyle \begin{aligned}\left(1 - \frac{x}{n}\right)^{-n} - \left(1 + \frac{x}{n}\right)^{n} &= \left(1 + \frac{x}{n}\right)^{n}\left\{\left(1 - \frac{x^{2}}{n^{2}}\right)^{-n} - 1\right\}\\
&< e^{x}\left\{\left(1 - \frac{x^{2}}{n}\right)^{-1} - 1\right\} = \frac{x^{2}e^{x}}{n - x^{2}}\end{aligned}$
and this last expression tends to $0$ as $n \to \infty$ and hence $$\lim_{n \to \infty}\left(1 - \frac{x}{n}\right)^{-n} = \lim_{n \to \infty}\left(1 + \frac{x}{n}\right)^{n} = e^{x}$$ Taking reciprocals we see that $$\lim_{n \to \infty}\left(1 - \frac{x}{n}\right)^{n} = \frac{1}{e^{x}}$$ or in other words $e^{-x} = 1/e^{x}$ for $x > 0$ and by duality it holds for $x < 0$ also. Thus we can see that if $x \to 0-$ then we can write $x = -y$ so that $y \to 0+$ and then
$\displaystyle \begin{aligned}\lim_{x \to 0-}\frac{e^{x} - 1}{x} &= \lim_{y \to 0+}\frac{e^{-y} - 1}{-y}\\
&= \lim_{y \to 0+}\frac{e^{y} - 1}{y}\frac{1}{e^{y}} = 1\cdot 1 = 1\end{aligned}$
Thus we have established two properties of $e^{x}$ namely $$\lim_{x \to 0}\frac{e^{x} - 1}{x} = 1,\,\, e^{-x} = \frac{1}{e^{x}}$$ The second property allows us to consider only positive arguments of the exponential function. Thus to establish the fundamental property $e^{x + y} = e^{x} \cdot e^{y}$ we need to consider $x, y > 0$ (for $x = y = 0$ it is obviously true). We can see that
$\displaystyle \begin{aligned} f(x, y, n) &= \left(1 + \frac{x}{n}\right)^{n}\left(1 + \frac{y}{n}\right)^{n} - \left(1 + \frac{x + y}{n}\right)^{n}\\
&= \left(1 + \frac{x + y}{n} + \frac{xy}{n^{2}}\right)^{n} - \left(1 + \frac{x + y}{n}\right)^{n}\\
&= \left(1 + \frac{x + y}{n}\right)^{n}\left\{\left(1 + \frac{xy}{n(n + x + y)}\right)^{n} - 1\right\}\\
&< e^{x + y}\left\{\left(1 + \frac{xy}{n^{2}}\right)^{n} - 1\right\}\\
&= e^{x + y}\left\{\frac{xy}{n} + \frac{(1 - 1/n)}{2!}\left(\frac{xy}{n}\right)^{2} + \cdots\right\}\\
&< e^{x + y}\left\{\frac{xy}{n} + \left(\frac{xy}{n}\right)^{2} + \cdots\right\}\\
&= e^{x + y}\frac{xy}{n - xy}\end{aligned}$
This shows that for fixed $x, y > 0$ the function $f(x, y, n) \to 0$ as $n \to \infty$. And therefore we have established $e^{x}e^{y} - e^{x + y} = 0$. Now we can easily show that
$\displaystyle \begin{aligned}\frac{d}{dx}e^{x} &= \lim_{h \to 0}\frac{e^{x + h} - e^{x}}{h}\\
&= \lim_{h \to 0}e^{x}\cdot\frac{e^{h} - 1}{h} = e^{x} \cdot 1 = e^{x}\end{aligned}$
Best Answer
Simple answer: You are right that by definition of derivative the expression you gave for the second derivative is the correct one. However it turns out that the other one is equal, though not obviously.
Proof by L'Hopital
One way to prove it is to use L'Hopital's rule once to get:
$\lim_{h \to 0} \dfrac{f(x+2h)-2f(x+h)+f(x)}{h^2} = \lim_{h \to 0} \dfrac{2f'(x+2h)-2f'(x+h)}{2h}$
because numerator and denominator are zero when $h = 0$ and are differentiable with respect to $h$. What we get is not quite the definition of the second derivative so we have to manipulate:
$\lim_{h \to 0} \dfrac{f'(x+2h)-f'(x+h)}{h} = \lim_{h \to 0} 2\dfrac{f'(x+2h)-f'(x)}{2h} - \lim_{h \to 0} \dfrac{f'(x+h)-f'(x)}{h}$
which is valid because both limits on the right exist. Note that:
$\lim_{h \to 0} 2\dfrac{f'(x+2h)-f'(x)}{2h} = 2 \lim_{h \to 0} \dfrac{f'(x+h)-f'(x)}{h}$
since $h \to 0$ is equivalent to $2h \to 0$ or just going by the definition of limit, and the $2$ factors out of the limit. Finally we get:
$\lim_{h \to 0} \dfrac{f(x+2h)-2f(x+h)+f(x)}{h^2} = \lim_{h \to 0} \dfrac{f'(x+h)-f'(x)}{h} = f''(x)$
by definition of second derivative.
Note
Notice that $f$ must be differentiable in an open interval around $x$ which was critical for L'Hopital's rule to work. But $f'$ need not be differentiable or even continuous in an open interval around $x$. All that is necessary, as used in the proof, is that $f'$ is differentiable at the single point $x$.