Solved – relationship between the median of a function of random variables and the function of the median of random variables

mathematical-statisticsmeanmedianprobabilityrandom variable

Background

notation: RV= random variable, $\mu=$ mean $m=$ median

Jensen's Inequality considers the relationship between the mean of a function of an RV and the function of the mean of an RV.

If $f(x)$ strictly convex:

$$\mu (f(x)) > f(\mu (x))\mathrm{\hspace{20mm}(1)}$$

Conversely if -f(x) is strictly convex:

$$\mu (f(x)) < f(\mu (x))$$

An analogous property of the median has been presented (Merkle et al 2005, pdf).

motivation

I have a nonlinear function of positive random variables.

In practice, I find that the function of the medians provides a much better estimate of the median of the function than does the estimate of the mean of the function from the function of the means. I am interested in learning the conditions for which this is true.

question

Under what conditions will the function of a median be closer to the median of a function than the mean of a function is to a function of the mean?

Specifically for what types of $f(x)$ and $x$ is

$$\mu (f(x)) – f(\mu (x)) > m (f(x)) – f(m (x))$$

simulation results

I used an empirical approach (the one I know) to investigate this question for a function of a single variable:

Interestingly, for $x>0$,

$$m(x^2)\simeq m(x)^2$$

set.seed(1)
x<-cbind(rlnorm(100, 1), rbeta(100, 1, 5), rgamma(100,0.5,0.5))
quad <- function(x)x^2

median.x <- apply(x,2,quantile,0.5)
mean.x <- apply(x,2,mean)

colMeans(quad(x))
quad(mean.x)

apply(quad(x), 2, quantile, 0.5)
quad(median.x)

For a slightly more complicated function, my proposal (equation 1) is true

miscfn <- function(x) 1 + x + x^log(x^2) - exp(-2(x)*5^x 

colMeans(miscfn(x))
miscfn(mean.x)

apply(miscfn(x), 2, quantile, 0.5)
miscfn(median.x)

abs(apply(miscfn(x),2,mean)-miscfn(mean.x)) > abs(apply(miscfn(x), 2, quantile, 0.5) - miscfn(median.x))

However, before I begin to use this observation in my work, I would like to know more about its conditions.

References

Merkle et al 2005 Jensen's inequality for medians. Statistics & Probability Letters, Volume 71, Issue 3, 1 March 2005, Pages 277-281

Best Answer

Let the cdf of $x$ be denoted by $F_X(x)$. Thus, the median of $X$ denoted by $m_x$ satisfies:

$F_X(m_x)=0.5$

Consider $Y = X^2$. Thus, the cdf of $Y$ is given by:

$P(Y \le y) = P(X^2 \le y)$

In other words, the cdf of $Y$ is given by:

$F_Y(y) = F_X(\sqrt{y}) - F_X(-\sqrt{y})$

The median for $Y$ denoted by $m_Y$ satisfies:

$F_Y(m_y)=0.5$

In other words, it should satisfy:

$F_X(\sqrt{m_y}) - F_X(-\sqrt{m_y}) = 0.5$

If $m_y = (m_x)^2$ then it must be that:

$F_X(m_x) - F_X(-m_x) = 0.5$

The above with the first equation suggests that the relationship $m(x^2) = m(x)^2$ will only hold if $F_X(-m_x) = 0$. Thus, the relationship holds only if the support of $X$ is positive.

The examples you examined in your code have a positive support and hence you find that $m(x^2) = m(x)^2$. If you try a uniform distribution (e.g., U(-1,1)) you will find that $m(x^2) \ne m(x)^2$

Related Solutions

Distributions – How to Understand the Relationship Between Mean and Median in Left Skewed Data?

It's a nontrivial question (surely not as trivial as the people asking the question appear to think).

The difficulty is ultimately caused by the fact that we don't really know what we mean by 'skewness' - a lot of the time it's kind of obvious, but sometimes it really isn't. Given the difficulty in pinning down what we mean by 'location' and 'spread' in nontrivial cases (for example, the mean isn't always what we mean when we talk about location), it should be no great surprise that a more subtle concept like skewness is at least as slippery. So this leads us to try various algebraic definitions of what we mean, and they don't always agree with each other.

If you measure skewness by the second Pearson skewness coefficient, then the mean ($\mu$) will be less than the median ($\stackrel{\sim}{\mu}$ -- i.e. in this case you have it backwards).

The (population) second Pearson skewness is $$\frac{3(\mu-\stackrel{\sim}{\mu})}{\sigma}\,,$$ and will be negative ("left skew") when $\mu<\stackrel{\sim}{\mu}$.

The sample versions of these statistics work similarly.

The reason for the necessary relationship between mean and median in this case is because that's how the skewness measure is defined.

Here's a left-skewed density (by both the second Pearson measure and the more common measure in (2) below):

enter image description here

The median is marked in the lower margin in green, the mean in red.

So I expect the answer they want you to give is that the mean is less than the median. It's usually the case with the sorts of distributions we tend to give names to.

(But read on, and see why that's not actually correct as a general statement.)

If you measure it by the more usual standardized third moment, then it is often, but by no means always, the case that the mean will be less than the median.

That is, it's possible to construct examples where the opposite is true, or where one skewness measure is zero while the other is non-zero.

Which is to say, there's no necessary relationship between the locations of the mean, median and the moment-skewness.

Consider, for example, the following sample (the same example can be constructed as a discrete probability distribution):

  2.7 15.0 15.0 15.0 30.0 30.0

mean: 17.95
median: 15

The mean is larger than the median, yet the third-moment skewness coefficient is negative (i.e. by its lights, we have left-skew data) since the sum of the cubes of the deviations from the mean is negative.

So in that sense, left-skew, but mean>median.

(On the other hand, if you change 2.7 in the above example to 3, then you have an example where the moment-skewness is zero, yet the mean exceeds the median. If you make it 3.3, then the moment-skewness is positive, and the mean exceeds the median - i.e. is finally in the 'anticipated' direction.)

If you use the first Pearson skewness instead of either of the above definitions, you have a similar issue to this case - the direction of the skewness does not pin down the relation between mean and median in general.

Edit: in answer to a question in comments -- an example where the mean and median are equal, but the moment-skewness is negative. Consider the following data (as before, it also counts as an example for a discrete population; consider writing the numbers on the faces of a die).

 1  5  6  6  8 10

the mean and the median are both 6, but the sum of cubes of deviations from the mean are negative, so the third moment skewness is negative.

Solved – the mean and variance of the median of a set of i.i.d normal random variables

The median is the central order statistic when the number of observations is odd. If $n$ is even then the median is either an order statistic, or the mean of 2 order statistics (or something else) depending on which definition of median you use.

So the exact distribution of the median can be worked out based on the distribution of order statistics. For odd $n$ where all the $x$'s are iid from a pdf $f$ with cumulative distribution $F$ the distribution of the median is:

$\binom{n-1}{(n-1)/2} F(x)^{\frac{n-1}2} f(x) (1-F(x))^{\frac{n-1}2}$

You can google "distribution of order statistics" to get more details and derivation.

For the normal we don't have a closed form solution for $F(x)$, but there are computational tools that can help estimate the above (see the distr package for R for one possibility).

If your main goal is just an estimate of the variance of the median, then a simpler approach is just to simulate a bunch of datasets and compute the variance of their medians (and the variance of their means for comparison).

The Wikipedia article on "Median" also has information that may be of interest.

Best Answer

Related Solutions

Distributions – How to Understand the Relationship Between Mean and Median in Left Skewed Data?

Solved – the mean and variance of the median of a set of i.i.d normal random variables

Related Question