[Math] Why is central difference preferred over backward and forward difference in convolution

Question

It is mentioned in some literature that we should always use central difference when computing the derivatives of an image instead of forward or backward difference. Does anyone knows why is that?

Central difference = $\frac{df(x)}{dx} = \frac{f(x+h) – f(x-h)}{2h}$

Forward difference = $\frac{df(x)}{dx} = \frac{f(x+h) – f(x)}{h}$

Backward difference = $\frac{df(x)}{dx} = \frac{f(x) – f(x-h)}{h}$

Answer 1

A few things come to mind:

• For smooth $f$, the central difference scheme is second order in $h$, whereas the other two you mentioned are first order in $h$. In other words, if $f$ is smooth, the (real space) error for the centered difference scheme is $O(h^2)$ whereas for the forward/backward schemes it is $O(h)$. Thus at fixed sufficiently small $h$, the central difference will have a better (real space) error than the other two. (Specifically, if $f \leq c_1 h$ and $g \leq c_2 h^2$ then $g<f$ once $h<\frac{c_2}{c_1}$. In this context, $c_1$ is proportional to a bound on $f''$ while $c_2$ is proportional to a bound on $f'''$.)
• Central differencing uses the same number of points as the other two you mentioned, so there is no loss in efficiency compared to those.
• There are higher order methods even than central differencing, and actually some of these may be even better. But very high order methods are not practical in floating point arithmetic, because they get their high order from subtraction of large nearly equal numbers, which causes loss of precision. "Middle order" methods (of order 2 through 6) are generally preferred when using double precision, because they do a good job of balancing accuracy in exact arithmetic with roundoff error when the calculations are done in double precision.

There are some other aspects of this that are more specific to signal processing. If this doesn't answer your question by itself then I can mention a little bit about that.

Upon request, here is a figure comparing the (signed) error in the central, forward, and backward derivative estimates for $e^x$ at $x=1$. The centered difference is in black, the forward difference is in blue, and the backward difference is in red.