I read the error propagation formula scanario said that, the connection between the variance of a random variable $x$ and $f(x)$ is $\frac{var(f(x))}{|\partial_xf(x)|_{x=\bar{x}}|^2}=var(x)$. While I'm confused in the prove of it in the process of Taylor series:
The proof of it states that, if we define the expected value of random variable $x$ and variance of it as follows:
$$
\begin{aligned}
\bar{x} &=\sum_{i=1}^{N} p_{i} x_{i} \\
(\Delta x)^{2} &=\sum_{i=1}^{N} p_{i}\left(x_{i}-\bar{x}\right)^{2} \\
&=\overline{x^{2}}-\bar{x}^{2}
\end{aligned}
$$
And the same for $f(x)$:
$$
\begin{aligned}
\overline{f(x)} &=\sum_{i=1}^{N} p_{i} f\left(x_{i}\right) \\
(\Delta f)^{2} &=\sum_{i=1}^{N} p_{i}\left(f\left(x_{i}\right)-\bar{f}\right)^{2}
\end{aligned}\tag{1}
$$
Then we can Taylor expanse it as:
$$
\begin{aligned}
f\left(x_{i}\right) &=\left.\sum_{n !}^{\infty} \frac{1}{n !}\left(x_{i}-\bar{x}\right)^{n} f^{(n)}(x)\right|_{x=\bar{x}} \\
&\approx f(\bar{x})+\left(x_{i}-\bar{x}\right) f^{(1)}(\bar{x})
\end{aligned}\tag{2}
$$
ignoring the square and higher-order term. Then $\overline{f(x)}$ can be written as
$$
\begin{aligned}
\overline{f(x)} &\approx f(\bar{x})+\overline{\left(x_{i}-\bar{x}\right)} f^{(1)}(\bar{x}) \\
&=f(\bar{x})
\end{aligned}\tag{3}
$$
Then replace eq (3) and (2) into (1)， we can get $\frac{var(f(x))}{|\partial_xf(x)|_{x=\bar{x}}|^2}=var(x)$.
My question is, $x_i-\bar{x}$ is not always a small term!!! Why should we just discard it? Or, are there more formal references of this error propagation formula, because all I search online is some other kind of form.
Best Answer
Indeed $x_i-\bar{x}$ is not always small but the average of it $\overline{x_i-\bar{x}}$ is zero. You could write is as $$\overline{x_i-\bar{x}} = \overline{x_i} - \overline{\bar{x}} = \bar{x_i} - \bar{x}=0$$
Sidenote: when you apply this linear approximation then $x_i-\bar{x}$ should actually be small. In this question about a log transformation, you see how the linear approximation works when the deviation are relatively 'small'. (It is also shown how it does not work and how a better approximation can be made)
In that question the following image was used, to show that a normal distribution (or something that looks like a normal distribution) becomes nearly linear transformed with a log transformation, when the deviations are not large.
You can see this similar for the error propagation. With the linear approximation we approximate the curve of the function by a straight line and we use the slope of the function to compute how the deviations changing with an approximately constant factor