The Wikipedia article on numerical differentiation mentions the formula
$$
h=\sqrt \epsilon \times x
$$
where $\epsilon$ is the machine epsilon (approx. $2.2\times 10^{-16}$ for 64-bit IEEE 754 doubles), to calculate the optimum "small number" $h$ to be used in differentiation, such as
$$
\frac{f(x+h)-f(x)}{h}
$$
But what if $x$ is zero? Then $h$ will be zero too, and division by zero is certainly not a way to do numerical differentiation. Is the article wrong? Is it otherwise correct, except that near zero (how near?) some small enough constant (how small?) should be used?
Best Answer
If you click through to the reference given for the Wikipedia piece, you'll find an answer. The formula given there is to take $h$ to be roughly $\sqrt{\epsilon_f}x_c$, where $\epsilon_f$ isn't necessarily "machine epsilon," but more to the point, where $x_c$ isn't necessarily $x$.