Descriptive Statistics – Is the Median of a Large Number of Medians Equal to the True Median?

Question

Take a non-symmetric distribution and assume its median exists and you know what it is in advance (call it $\mu$). Then, draw a very large number of independent samples from it (a large multiple of some integer, $n$). Now, divide into groups of $n$ (first $n$ samples in the first group, next $n$ samples in the second group and so on). Find the median for each of these groups. Then, take the median across all these calculate medians. Will this resulting median of medians always be very close to $\mu$ for all (or most) distributions. It is well known that the mean of these medians will be far from $\mu$. However, simulations show me that the median of medians is very very close for a variety of distributions. But can this be proven?

See plot below where I took the median of 100,000 medians where each individual median was from 11 samples for the LogLogistic distribution. The black line ($\mu$) is plum behind the orange line (median of medians). Seems to hold for many other distributions as well.

Code used for generating the plot: https://gist.github.com/ryu577/420d6a9e56b21d3e7038660e3a9a88f5

Answer 1

Let's for simplicity deal with samples of odd size $2k+1$, then the median of a sample is the the $(k+1)$-th order statistic.

We can model this order statistic as drawing a quantile $p$ from a beta distribution and transforming with the quantile function, or the inverse of the cumulative distribution

$$\text{sample median} = Q(p) = F^{-1}(p) \qquad \text{where $p \sim Beta(k+1, k+1)$}$$

The median of medians is then like drawing several $p$ from that beta distribution, take the median $p_m$ of those $p$ and transform it with the quantile function $Q(p_m)$

For increasing sample size, the median $p_m$ of a sample taken from a $Beta(k+1,k+1)$ distribution will approach $0.5$ and so the median of medians will approach $Q(0.5)$, provided that this point exists*.

---

It is well known that the mean of these medians will be far from $\mu$. However, simulations show me that the median of medians is very very close for a variety of distributions.

Why are the 'mean of medians' and the 'median of medians' different?

• Both the mean and median of the sample of $p$ will approach $0.5$.
• However there is a difference. For the median $M(\cdot)$ we have the following transformation rule: when we take the median of a transformed variable (where the transformation $f$ is a monotonic function) then this is the same as transforming the median of the variable. $$M(f(p)) = f(M(p))$$ For the mean this is not true in general.

---

^{* An example when this occurs is a Bernoulli distribution with equal probability for $0$ and $1$ $$P(X=x) =\begin{cases}0.5&\qquad \text{if $x=0$}\\ 0.5&\qquad \text{if $x=1$}\\ 0&\qquad \text{else}\\\end{cases}$$ In this case the median of medians must be either $0$ or $1$ when the sample size is odd. Due to symmetry this is with equal probability and there is no point towards which the median of medians converges if the number of medians is increased.}

^{Similar examples can be made with other discrete distributions. When we consider only the median of odd samples then this 'problem' is resolved.}