I'm having a hard time understanding what the authors of this paper (pdf) want to tell me with this graph (Fig. 2,3 (shown below) and 4, right):
[Caption: A comparison between the standard deviation of the differences (green line) and the standard deviation of all GOMOS (red line) and LIDAR (blue line) ozone profiles.]
Note: contrary to convention, the measured quantity is plotted on the x-axis, not y-axis.
The problem at hand is as following: Two instruments measure the same physical quantity (in this case vertical distribution of ozone in the atmosphere). Their corresponding (instrument characterized) standard deviations (averaged over if I understand correctly) are displayed in red and blue. The difference between the two datasets is computed and the standard deviation of the difference data set is plotted in green.
What does this graph mean? Is this a way to say that the two datasets are in agreement? If yes, on what basis?
What would other extremes mean?
- stdev of the difference is much larger than the measurement stdevs
I think: measurement stdevs are too optimistic. - stdev of the difference is much smaller than the measurement stdevs
I think: measurement stdevs are too conservative.
Maybe an example would help me to understand this better. Thanks
Best Answer
This appears to be an unconventional way to report correlation (or lack thereof). It focuses more on the variability of the measurements (across the earth at each fixed altitude) than on the correlation among them. As such the graphic may be of physical interest but it's an obscure way (at best) of comparing two measurement systems.
At each vertical position (an estimated altitude based on a pressure reading) the plots summarize between 3 and 18 pairs of data obtained over fixed stations on the earth's surface. The summaries consist of sample standard deviations normalized by the LIDAR readings (the reference measurement).
When comparing measurements, one is usually interested in assessing their correlations. We need to do a little math to relate this graphic to those correlations. Let $(X,Y)$ be a random variable representing the (LIDAR, GOMOS) readings. Let the variance of $X$ be $\sigma^2$, the variance of $Y$ be $\tau^2$, and their correlation equal $\rho$. Then
$$Var(X-Y) = Var(X) + Var(Y) - 2Covar(X,Y) = \sigma^2 + \tau^2 - 2 \rho \sigma \tau.$$
Consequently we can recover the correlation from the covariances:
$$\rho = \frac{1}{2\sigma \tau}(Var(X) + Var(Y) - Var(X-Y)).$$
Let the LIDAR mean be $m$. The plots depicts estimates of $\sigma/m$ (relative LIDAR SD): call this $s$; $\tau/m$ (relative GOMOS SD): call this $t$; and $\sqrt{Var(X-Y)}/m$ (relative SD of difference): call this $r$. Plug the estimates in to the preceding formula:
$$\rho = \frac{1}{2(s m)(t m)}((s m)^2 + (t m)^2 - (r m)^2;$$
$$\rho = \frac{s^2 + t^2 - r^2}{2 s t}.$$
These are, of course, estimates of $\rho$, subject to sampling uncertainty.
We can now qualitatively identify several portions of the plot:
$s = t = r$, approximately, between 35 and 45 km. From the formula we estimate $\rho \sim 1/2$. This is modest correlation--not very good for two measurements of the same thing.
One of $s$ and $t$ is small relative to the other and $r$ is comparable to the larger. This occurs from 20 to about 25 km and 45 to 50 km. The formula indicates $\rho \sim 0$. This is lack of correlation.
$r$ is small and $s$ and $t$ are comparable (between 28 and 35 km). Now we estimate $\rho \sim 1$. This is what one hopes to see for two consistently comparable measurements.
In short, good correlation occurs when the green line lies substantially to the left of the red or blue lines and there is lack of correlation wherever the green line approximates (or exceeds) either or both of the red and blue lines. Overall, correlation is poor except between 27 and 34 km.