[Math] Finite difference second derivative: Error analysis

Question

I want to approximate $f''(x)$ using finite differences. From
\begin{align*}
f(x+h)=f(x)+f'(x)h+\frac{1}{2}f''(x)h^2+\frac{1}{6}f'''(x)h^3+O(h^4),\\
f(x-h)=f(x)-f'(x)h+\frac{1}{2}f''(x)h^2-\frac{1}{6}f'''(x)h^3+O(h^4),
\end{align*}
we easily obtain
\begin{align}
f''(x)=\frac{f(x+h)-2f(x)+f(x-h)}{h^2}+O(h^2).
\end{align}
Alternatively, we can apply centered finite differences twice with step size $h/2$ and error $O(h^2)$ to write:
\begin{align}
f''(x) &= \frac{f'(x+h/2)-f'(x-h/2)}{h}+O(h^2)\\
&=\frac{\frac{f(x+h)-f(x)}{h}-\frac{f(x)-f(x-h)}{h}+O(h^2)}{h}+O(h^2)\\
&=\frac{f(x+h)-2f(x)+f(x-h)}{h^2}+O(h)+O(h^2).
\end{align}
I do not understand the inconsistency between the two approaches. Should the $O(h)$ error term above vanish?

Answer 1

You get $$f''(x)=\frac{\frac{f(x+h)-f(x)}{h}-\frac{f(x)-f(x-h)}{h}+O(h^2)}{h}$$by reasoning that the two approximations to $f'$ both have leading error $O(h^2)$ and therefore their difference will have error $O(h^2)$.

In fact they have the same leading error term which will cancel upon subtraction, and leave you with the result that you were expecting.