[Math] Error of central difference quotient vs forward difference quotient

derivativesnumerical methods

Most people are familiar with this difference quotient:

$$ f'(x) = \lim_{n \rightarrow \infty} \frac{f(x+h) – f(x)}{h} $$

We require that $f(x)$ is twice differentiable.

The error of that forward difference quotient is, using Taylor:

$$f'(x) = \frac{f(x+h) – f(x)}{h} + + \mathcal{O}(h) $$

Now to the central difference quotient:

We establish that:

$$ f(x+h) = f(x) + hf'(x) + \mathcal{O}(h^2) $$

and

$$ f(x-h) = f(x) – hf'(x) + \mathcal{O}(h^2) $$

Moving on:

$$ f(x+h) – f(x-h) = 2hf'(x) $$

$$ \frac{f(x+h) – f(x-h)}{2h} = f'(x) $$

Where did I make a mistake ? Because it seems like the error disappeared completely. Seems like we can differentiate any function exactly with this technique ( for any $h$).

So my question is:

  1. How does the error of the forward difference quotient compare to the central difference quotient ?

  2. Also another more general question: What does that error exactly mean ?

I'm a little bit confused because it seems smaller $i$ for $\mathcal{O}(h^i)$ would be better, yet some sources say a bigger $i$ is better.

A short precise sentence about that error term woudl be much appreciated

Best Answer

I'll try to be as concise as possible. The point is that $O(h^2)$ isn't a "real" algebric term that you can cancel out that simply. I hope this example clarify the big-O notation: call $D_h^Cf(x)$ the centered finite difference approximation, you can say that:

$f'(x)-D_h^Cf(x) = O(h^2)$

This means that exists $h_0$ and $C$ such that for $0<h<h_0$, $ |f'(x)-D_h^Cf(x)| \leq C\cdot h^2$. Concluding, the big-O notation tells you that the difference $f'(x)-D_h^Cf(x)$ is bounded by something which is proportional to $h^2$.

This is the reason why you can't cancel out the $O(h^2)$ in your calculations.

Answering your two questions:

1)The difference between the centered (C) and forward (F) finite difference lies in the fact that those two errors has two different orders, C has order 2, while F has order 1. What does this mean? It means that when $h \rightarrow 0$ (which is generally what we do, we are interested in a small $h$) the C finite difference gives an approximation which is more accurate than the F finite difference. In other terms, if you fix $h$ small (say, $10^{-2}$) the C finite difference is closer to the exact value of $f'(x)$ rather than the F finite difference. It means you can use $h$ which is not ludicrously small to get the same degree of accuracy.

2)That error means how far off you are from the "actual" derivative, using a finite difference. You can think of it geometrically: take a function and draw the tangent to the graph in a point. Then you can construct the approximate tangent using finite difference method. What you find is something similar to the figure: Forward finite difference.

The error tells you, for a given $h$, the discrepancy between the coefficient of the approximated tangent and the actual first derivative, in a given point $x$.

I hope this clarifies.

Related Question