I want to calculate the standard error for an experimental measurement. The data is stored as a 2D image which is circularly symmetric about a center point. To reduce the data we radially integrate to get counts/intensity as a function of distance from the center point (radius).
The algorithm for doing the radial integration uses a weighted binning method to determine how to bin each pixel. This is for two reasons, the desired distance grid is much finer than the pixel width and also because the image has square pixels and not semi-circular. The way this works is for every radius grid point, the algorithm essentially draws a circle, determines which pixels that circle passes through, then weights each of pixels depending on where in the pixel that circle passes through. The counts/intensity for each of these pixels is then multiplied by the assigned weight and summed to get the intensity/counts at that radius grid point.
I am want to determine the error for each pixel but I am unsure about how to factor in the weight factor. Here are the formula I am using for the variance, standard deviation, and normal error,
where I is intensity (\bar{I} is the average intesity), Q is the radius, and N is the sum of all the weight factors.
Initially I set it up to loop through all the radius grid points to calculate I(Q) as the sum of all the weighted intensity values, N(Q) as the sum of all the weight factors, and an array of I(Q,i) as the weighted intensity values for each pixel. I define \bar{I} = I(Q)/N(Q).
I think this gives a incorrect variance because each I(Q,i) is a partial contributor to the sum, and not a complete pixel value. It does have the weight factor but I am not sure that is complete. My error bars seem much smaller than I expect. This if fine, as long as I am confident they are correct.
Thanks for any insights. I am happy to clarify any details. I also posted this question here as I was not sure which community would be more appropriate.
Best Answer
I just found this wikipedia page discussing data of equal significance vs weighted data. The correct way to calculate the biased weighted estimator of variance is: