We want to see the total error in approximating
$$ f'(x) \approx \frac{ f(x+h)-f(x) }{h} $$
where $f: R \to R$ is differentiable. We can find $\theta \in [x,x+h]$
by Taylor's to that$$ f(x+h) = f(x) + f'(x) h + f''( \theta ) h^2 /2 $$
If the error in function values is bounded by $\epsilon$, prove that
the rounding error is bounded by $2 \epsilon /h$ and the truncation
error is bounded by $Mh/2$ where $M$ is a bound for $|f''(t)|$ for $t$
near $x$.
Try
We know truncation error is difference between true result and the result that would be produced by algorithm. By the result given above, we see that
$$ \frac{ f(x+h) – f(x) }{h} = f'(x) + f''(\theta)h/2 $$
So that
$$ \underbrace{ \frac{ f(x+h) – f(x) }{h} }_{approx} – \underbrace{f'(x)}_{true \; result} = f''(\theta)h/2 $$
So that trucantion error $E_T$ is absolute value of the above:
$$ E_T = |f''(\theta) | h/2 \leq Mh/2 $$
So we have our first result. However, for the rounding error I dont see how it is $2 \epsilon / h $. Can someone explain what they really mean? Perhaps am I misunderstanding this part.
Best Answer
The rounding error? Take that upper bound $\epsilon$ for the error in function values. We're subtracting two instances of it, which doubles that (at most $\epsilon - (-\epsilon)$). Then we divide by $h$, for $\frac{2\epsilon}{h}$. That's how far our calculated value of the difference quotient $\frac{f(x+\epsilon)-f(x)}{\epsilon}$ can be from the true value.