[Math] Iteratively Updating a Normal Distribution

normal distributionstatistics

Is there a way to update a normal distribution when given new data points without knowing the original data points? What is the minimum information that would need to be known? For example, if I know the mean, standard deviation, and the number of original data points, but not the values of those points themselves, is it possible?

Best Answer

It is certainly possible. The best way, avoiding some numerical precision issues, is to track the following two values, using the new $n$th observation $a_n$ each time to update the following:

$$m_n = m_{n-1} + \frac{a_{n}-m_{n-1}}{n}$$

$$s_n = s_{n-1} + (a_n - m_{n-1})(a_n - m_n)$$

starting with $m_0=s_0 =0$. Then the mean of the first $n$ values is $m_n$ while the standard deviation is $\sqrt{\frac{s_n}{n}}$ or $\sqrt{\frac{s_n}{n-1}}$ depending on what denominator you usually use to calculate the standard deviation. If you would prefer to just track the standard deviation you can calculate $s_{n-1}=(n-1)\sigma_{n-1}^2 \text{ or } (n-2)\sigma_{n-1}^2 $ each time.

Related Solutions

[Math] incrementally calculate gaussian distribution

Recall that the standard deviation $\sigma$ for some sample $X$ with results $x_{1},x_{2},\dots,x_{i}$ can be calculated by:

$$\sigma=\sqrt{\frac{\sum_{i}x_{i}^{2}}{n}-\left(\frac{\sum_{i}x_{i}}{n}\right)^{2}}=\sqrt{E(X^{2})-E^{2}(X)}$$

Therefore, if we add a new result $x_{\text{new}}$ to the sample, we can calculate the new mean $\bar{x}_{\text{new}}$ using:

$$\bar{x}_{\text{new}}=\frac{n\bar{x}+x_{\text{new}}}{n+1}$$

And then, rearranging $\sigma^{2}=E(X^{2})-E^{2}(X)=\frac{\sum_{i}x_{i}^{2}}{n}-\bar{x}^{2}$ gives $\sum_{i}x_{i}^{2}=n(\sigma^{2}+\bar{x}^{2})$, therefore, we find that:

$$\sigma^{2}_{\text{new}}=\frac{n(\sigma^{2}+\bar{x}^{2})+x_{\text{new}}^{2}}{n+1}-\bar{x}_{\text{new}}^{2}$$

Or, in pseudocode:

mean[n+1] = ((n * mean[n]) + value)/(n+1)
var[n+1] = (n * (var[n] + mean[n]^2) + value^2)/(n+1) - mean[n+1]^2

Statistics – Finding Mean and Standard Deviation of Normal Distribution Given Two Points

Let's take the example in question. Assume that the mean is $\mu$ and that the standard deviation is $\sigma$. If we have two z-values $z_1$ and $z_2$ corresponding to our two observations, 19.76 and 18.3 then we can solve the following equations for $\mu \ \text{and} \ \sigma$. $$\frac{19.76 - \mu}{\sigma} = z_1 \\ \frac{18.3 - \mu }{\sigma} = z_2$$ We have two equations in two unknowns, solving which, we can find $\mu$.

Best Answer

Related Solutions

[Math] incrementally calculate gaussian distribution

Statistics – Finding Mean and Standard Deviation of Normal Distribution Given Two Points

Related Question