Let $X_1,\ldots,X_n$ be $n$ iid random variables. Suppose they are arranged in increasing order
$$X_{(1)}\leq\cdots\leq X_{(n)}$$
The first ordered statistic is always the minimum of the sample
$$Y_1 \equiv X_{(1)}=\min\{\,X_1,\ldots,X_n\,\}$$
For a sample of size $n$, the $n$th order statistic is the maximum, that is,
$$Y_n \equiv X_{(n)}=\max\{\,X_1,\ldots,X_n\,\}$$
According to Wolfram Mathworld (here), if $X$ has a probability density function $f(x)$ and cumulative distribution function $F(x)$, then the probability $Y_r$ is given $$f_{Y_r} = \frac{N!}{(r-1)!(N-r)!} [F(x)]^{r-1} [1-F(x)]^{N-r}f(x)$$
My Questions
- Is an ordered statistic merely the ordering of a collection of random variables from highest to lowest? But I thought a random variable was a collection of values (e.g. $\lbrace 1,2,3,4,5,6 \rbrace$), so how can we rank them?
- How do I derive that formula for $f_{Y_r}$? I googled it several times and I don't get how I found a would be derivation here I think on slide 4, but I don't follow what's going on.
Best Answer
I believe that that ordered statics are the ordering of realizations of $X_1,\dotsc, X_n$ from lowest to highest. A random variable is a function.
The thing you posted for $f_{Y_r}$ is the formula for the $r$th order statistic. I do not advocate the memorization of this formula. And for a long time, I didn't even know it existed. To me, it is better to understand what is happening. I provide a simple example with a very well known result. In the first part, I talked through it, but it is essentially the formula you provide. You can see how it would generalize to an arbitrary number $n$, with a generic distribution $G$. In the second part, I approach using the cdf.
Let $Y_1,Y_2,Y_3$ iid exponential distributions with mean $1/\lambda$. Then, to find distribution of the minimum $M:=Y_{(1)}$, we must consider \begin{align*} P(M\in dm) &= P(M\in [m,m+\epsilon])\tag 1\\ &=P(\text{one of the $Y$'s } \in [m,m+\epsilon]\text{ and all others }>x)\tag 2 \end{align*} where in $(1)$ and $(2)$ I use the notation provided in the slides.
Since there are $3$ choices for the minimum
the number of ways to choose a minimum is $$\binom{3}{1} = 3$$ where $\binom{n}{k}$ is the counting factor called the binomial coefficient. Notice that I am not invoking the binomial distribution. In other words, there are $3$ choices to fall in the interval $[m,m+\epsilon]$.
Once we have chosen the smallest, then there is only $$\binom{2}{2} = 1$$ way to choose the other two larger ones. In other words, there is one way have the rest fall into the interval $(m,\infty)$. Recall that the $Y_i$ are iid $Y\sim\text{Exp}(\lambda)$, by which I mean that the average is $1/\lambda$. So it must be the case that \begin{align*} P(M\in dm) &= P(M\in [m,m+\epsilon])\\ &=P(\text{one of the $Y$'s } \in [m,m+\epsilon]\text{ and all others }>m)\\ &=P(\text{one of the $Y$'s } \in [m,m+\epsilon])P(\text{All others }>m)\tag 3\\ &=\binom{3}{1}[f_Y(m)\epsilon]\binom{2}{2}[1-F_Y(m)]^2 \\ &=3[\lambda e^{-\lambda m}\epsilon][1-\{1-e^{-\lambda m}\}]^2 \\ &= 3\lambda\epsilon e^{-3\lambda m} \end{align*} where in $(3)$ I use independence. This gives that the pdf is $$f_M(m) = 3\lambda e^{-3\lambda m}.$$
Notice that the minimum $M$ (or $Y_{(1)}$) follows an exponential distribution with mean $\frac{1}{3\lambda}$.
Alternatively, for the $n$ case $$P(Y_{(1)}\leq m) = 1-P(Y_{(1)}>m) =1- (e^{-\lambda m})^n = 1- e^{-n\lambda m}.$$ Then the pdf is $n\lambda e^{-n\lambda m}$. Notice that the interpretation is that there is $\binom{n}{1}$ options for the smallest. Then the rest $\binom{n-1}{n-1} = 1$ have to fall in the interval $(m,\infty)$, hence $n\lambda e^{-n\lambda m}$.