Random Variable Realisation – Difference Between a Realisation of a Random Variable and the Random Variable Itself

definitionexpected valuenotationprobability

I am having a hard time distinguishing random variables from their realisations. (Please note that for the sake of simplicity of my question I use discrete values in my example below.) Usually, a random variable is shown with an upper case letter such as "$\boldsymbol{X}$" and its realisations are shown with a lower case letter such as "$\boldsymbol{x}$". For instance, if the experiment is to weigh a group of five objects then the random variable $X$ will be a set of five realisations of the weight random variable. So we can write $X_1:\{x_1 = 80, x_2 = 83, x_3 = 67, x_4 = 72, x_5 = 90\}$ in Kg. (Note that I wrote $X_1$ instead of just $X$ to emphasise that this is my first random variable and other random variables will be introduced below, shortly.)

If one would like to add another random variable such as "temperature" of the weighted objects we could, simply, write that as "$X_2$". Let us assume that realisations of the temperature variable (i.e. $X_2$) are $X_2:\{x_1 = 12, x_2 = 30, x_3 = 45, x_4 = 23, x_5 = 9\}$ in Kelvin. And we could go on and add as many random variables as we would like to (e.g. $X_3$: velocity of the objects, $X_4$: colour of the objects and so on where each of the variables will have five realisations). Things are fine until we come across statements such as "If $X_1, X_2, X_3, \dots, X_n$" are $n$ random variables, we find that"

$$E[\sum_{i=1}^{n} X_i] = \sum_{i=1}^{n} E[X_i]$$

I have come across this and similar equations in different textbooks (where there is some kind of a sum over "$n$" random variables)! The problem is that (given my example above) the units of random variables do NOT match so the whole sum does not make sense! Given my interpretation of random variable, I would write the right hand side of the equation above as follows,

$$ \sum_{i=1}^{n} E[X_i] = E[X_1] + E[X_2] + E[X_3] + … + E[X_n].$$

In my example above $n = 2$ so we only have to deal with $E[X_1]+E[X_2]$. And I run into the problem of different random variables having different units (i.e. $E[X_1]$ in Kg and $E[X_2]$ in Kelvin). There is another question relevant to mine here where my interpretation of the answer to that question is that $X_1$, $X_2, \dots, X_n$ are not random variables but realisations! But if that is the case then $E[X] = \mu$ does not make sense! How can expected value of one realisation ($X$, that is a single observation/measurement) equal to the mean of the entire population ($\mu$)? If I were to summarise my question I would ask the followings,

What is the difference/connection between $x_i$, $X_i$, and a vector of random variables $\boldsymbol{X}$?
What is the connection between random variables {$x_i, X_i, \boldsymbol{X}$} and sample mean $\overline{X}$ and population mean $\mu$?

Best Answer

The realization of a random variable is the value that was observed (though, as noticed in the comments, you can have random variables for non-observable things). For example, you treat the result of throwing a fair dice as a random variable $X$. Say that the result is five dots, $x=5$ is the realization. The “five objects” that you call “realizations” are all random variables that together form multivariate random variable. In this framework, it doesn’t make sense to discuss a single random variable with multiple realizations.

You can throw a dice $n$ times and treat the results as $X_1,X_2,\dots,X_n$ random variables with $n$ observed realizations accordingly. $E[X_1]$ could be an expected value of the random variable for the result of first throw $X_1$, where the realization $x_1$ would be a number, for example, $3$. So

$$ \bar X = \frac{1}{n} \sum_{i=1}^n X_i $$

is a random variable, as a function of $n$ random variables, and in

$$ \bar x = \frac{1}{n} \sum_{i=1}^n x_i $$

$\bar x$ is a realization of $\bar X$ calculated from realizations $x_i$ random variables $X_i$.

A random vector $\mathbf{X} = (X_1,X_2,\dots,X_n)$ is just a shorthand for writing them all each time.

Finally, you would see different notations used by different authors and in different contexts, so each time you need to make sure what is described rather than assuming things from notation alone.

You should probably refresh your knowledge on random variables to make things clearer. Given the multiplicity of issues mentioned by you, I'd recommend also a probability and statistics handbook or leactures.

Related Solutions

Max-Min Random Variable – PDF/CDF of Max-Min Type Random Variable

I will illustrate with the example in the question, because a general answer is too complicated to write down.

Let $F$ be the common distribution function. We will need the distributions of the order statistics $x_{[1]} \le x_{[2]} \le \cdots \le x_{[n]}$. Their distribution functions $f_{[k]}$ are easy to express in terms of $F$ and its distribution function $f=F^\prime$ because, heuristically, the chance that $x_{[k]}$ lies within an infinitesimal interval $(x, x+dx]$ is given by the trinomial distribution with probabilities $F(x)$, $f(x)dx$, and $(1-F(x+dx))$,

$$\eqalign{ f_{[k]}(x)dx &= \Pr(x_{[k]} \in (x, x+dx]) \\&= \binom{n}{k-1,1,n-k} F(x)^{k-1} (1-F(x+dx))^{n-k} f(x)dx\\ &= \frac{n!}{(k-1)!(1)!(n-k)!} F(x)^{k-1} (1-F(x))^{n-k} f(x)dx. }$$

Because the $x_i$ are iid, they are exchangeable: every possible ordering $\sigma$ of the $n$ indices has equal probability. $X$ will correspond to some order statistic, but which order statistic depends on $\sigma$. Therefore let $\operatorname{Rk}(\sigma)$ be the value of $k$ for which

$$\eqalign{ x_{[k]} = X = \max&\left( \min(x_{\sigma(1)},x_{\sigma(2)},x_{\sigma(3)}),\min(x_{\sigma(1)},x_{\sigma(4)},x_{\sigma(5)}), \right. \\ & \left. \min(x_{\sigma(5)},x_{\sigma(6)},x_{\sigma(7)}),\min(x_{\sigma(3)},x_{\sigma(6)},x_{\sigma(8)})\right). }$$

The distribution of $X$ is a mixture over all the values of $\sigma\in\mathfrak{S}_n$. To write this down, let $p(k)$ be the number of reorderings $\sigma$ for which $\operatorname{Rk}(\sigma)=k$, whence $p(k)/n!$ is the chance that $\operatorname{Rk}(\sigma)=k$. Thus the density function of $X$ is

$$\eqalign{ g(x) &= \frac{1}{n!} \sum_{\sigma \in \mathfrak{S}_n} f_{k(\sigma)}(x) \\ &= \frac{1}{n!}\sum_{k=1}^n p(k)\binom{n}{k-1,1,n-k} F(x)^{k-1} (1-F(x))^{n-k} f(x) \\ &=\left(\sum_{k=1}^n \frac{p(k)}{(k-1)!(n-k)!}F(x)^{k-1} (1-F(x))^{n-k} \right)f(x) .}$$

I do not know of any general way to find the $p(k)$. In this example, exhaustive enumeration gives

$$\begin{array}{l|rrrrrrrrr} k & 1 & 2 & 3 & 4 & 5 & 6 & 7 & 8 & 9\\ \hline p(k) & 0 & 20160 & 74880 & 106560 & 92160 & 51840 & 17280 & 0 & 0 \end{array}$$

The figure shows a histogram of $10,000$ simulated values of $X$ where $F$ is an Exponential$(1)$ distribution. On it is superimposed in red the graph of $g$. It fits beautifully.

The R code that produced this simulation follows.

set.seed(17)
n.sim <- 1e4
n <- 9
x <- matrix(rexp(n.sim*n), n)
X <- pmax(pmin(x[1,], x[2,], x[3,]),
          pmin(x[1,], x[4,], x[5,]),
          pmin(x[5,], x[6,], x[7,]),
          pmin(x[3,], x[6,], x[8,]))

f <- function(x, p) {
  n <- length(p)
  y <- outer(1:n, x, function(k, x) {
    pexp(x)^(k-1) * pexp(x, lower.tail=FALSE)^(n-k) * dexp(x) * p[k] /
      (factorial(k-1) * factorial(n-k))
  })
  colSums(y)
}

hist(X, freq=FALSE)
curve(f(x, p), add=TRUE, lwd=2, col="Red")

Solved – How to calculate the probability that one random variable is bigger than a second one

To be brutally mindless about it, we may begin with the full five-dimensional integral and then proceed to evaluate it. Because this is carried out over a region in $\mathbb{R}^5,$ I will not attempt to sketch it :-).

As a simplification of the notation (and to reveal the ideas), let the joint density of $(X_1,X_2,X_3)$ be $f_{123} $ and the joint density of $(X_4,X_5)$ be $f_{45}.$ Then, with $P = \Pr(\min(X_1,X_2,X_3) \gt \max(X_4,X_5)),$

$$P = \iint f_{45}(x_4,x_5) \iiint_{\max(x_4,x_5)} f_{123}(x_1,x_2,x_3)\,\mathrm{d}x_1\mathrm{d}x_2\mathrm{d}x_3\ \mathrm{d}x_4\mathrm{d}x_5.$$

The first (double) integral extends over all $\mathbb{R}^2$ while the second (triple) integral extends only over those points $(x_1,x_2,x_3)$ in $\mathbb{R}^3$ where all three coordinates exceed both $x_4$ and $x_5.$

It is usually easiest to deal with a maximum in an integral's endpoint by breaking the integral into parts: almost surely either $X_4$ or $X_5$ will be the larger of those two and these two events (namely, $\mathcal{E}_4:X_4=\max(X_4,X_5)$ and $\mathcal{E}_4:X_5=\max(X_4,X_5)$) are mutually exclusive. Therefore we may compute the probabilities of these two events and add them.

Because $X_4$ and $X_5$ are iid, they are exchangeable, implying $\mathcal{E}_4$ and $\mathcal{E}_5$ have the same probability. Consequently, taking the case $X_4\gt X_5$ (event $\mathcal{E}_4$), we obtain

$$P = 2\int\int_{x_5} f_{45}(x_4,x_5) \iiint_{x_4} f_{123}(x_1,x_2,x_3)\,\mathrm{d}x_1\mathrm{d}x_2\mathrm{d}x_3\ \mathrm{d}x_4\mathrm{d}x_5.$$

Specializing now to iid uniform$[0,1]$ variables we may compute this integral using the most elementary techniques as

$$\begin{aligned} P &= 2\int_0^1\int_{x_5}^1\int_{x_4}^1\int_{x_4}^1\int_{x_4}^1\,\mathrm{d}x_1\mathrm{d}x_2\mathrm{d}x_3\mathrm{d}x_4\mathrm{d}x_5 \\ &= 2\int_0^1\int_{x_5}^1 \left(\int_{x_4}^1\mathrm{d}x_1\right)\left(\int_{x_4}^1\mathrm{d}x_2\right)\left(\int_{x_4}^1\mathrm{d}x_3\right)\,\mathrm{d}x_4\mathrm{d}x_5 \\ &= 2\int_0^1\int_{x_5}^1(1-x_4)^3\mathrm{d}x_4\mathrm{d}x_5 \\ &= 2\int_0^1 \frac{1}{4}(1-x_5)^4\,\mathrm{d}x_5 \\ &= 2\left(\frac{1}{4}\right)\left(\frac{1}{5}\right) = \frac{1}{10}. \end{aligned}$$

This gives the answer for any continuous iid variables with common density $f$ because the Probability Integral Transform

$$u(x) = \int^x f(t)\,\mathrm{d}t,$$

converts the variables $(X_1,\ldots, X_5)$ into variables $U_i = u(X_i)$ that are iid with a Uniform$[0,1]$ distribution without changing the order statistics, thereby leading to the calculation of $P$ that was just performed.

Best Answer

Related Solutions

Max-Min Random Variable – PDF/CDF of Max-Min Type Random Variable

Solved – How to calculate the probability that one random variable is bigger than a second one

Related Question