While trying to approximate derivatives in my numerical methods class, we were taught about forward and central difference approximations, however apart from questions when the method to be used is stated i have no idea which is to be used in different scenarios.
While researching online i found that you can also use backwards difference approximations, yet we had not been taught this method, is this due to it being less useful?
Formulas:
Forward difference approximation:
$$f'(x)\approx \frac{f(x+h)-f(x)}{h}$$
Central difference approximations
$$f'(x)\approx\frac{f(x+h)-f(x-h)}{2h}$$
Backward difference approximations:
$$f'(x)\approx \frac{f(x)-f(x-h)}{h}$$
It seems to me like forward and back are essentially the same but used depending on whether the behind or forward of $x$ will give a better approximation of the gradient. However i can't think of situation were central would produce a more accurate approximation, surely using a larger "interval" to approximate would make the gradient less accurate, even if you then divide the gradient by $2h$ instead of $h$ surely that makes the result smaller but not more accurate. can anyone explain the use of the different methods and when they should be used? And perhaps find an example curve that is approximated better with the central method?
Best Answer
Assuming you have a nice enough $f$ (lets say thrice continuously differentiable around $x$), the central difference has error on the order of $h^2$, which is smaller than the error from forward/backward differences (which is on the order of $h$). You can see this by plugging in a Taylor polynomial of degree 3 into the definitions of the forward/backward/central difference, and see that the error is on the order stated prior. Or read about it here.
So, the central difference is more accurate than forward/backward. The computational complexity is the same, but depending on the application, it may not be usable. For example, if you have data arriving in time, and you need the time derivative at the current time and can't look into the future, you have to use something like a backward difference.