Let's say I have an experiment where I repeatedly toss a coin; each toss is independent. I would like to define a random variable $X: \omega \rightarrow R$. $X$ is the number of failures before the first success. I would like to visualize the sample space of this experiment, but I'm having trouble. Is the sample space an infinite set containing potentially infinitely long sequences?
[Math] Sample Space of a Geometric Distribution
probability distributionsprobability theorystatistics
Related Solutions
When you would use which really depends on the random variable you're interested in. If you're interested in the number of trials needed to obtain the first success, use the first kind of geometric distribution. If you're interested in the number of successes you are able to achieve before the first failure occurs then use the second kind of geometric distribution. (Inverting the interpretation of the second version like this also requires you to redefine the success probability as $1-p$ and the failure probability as $p$.)
As the Wikipedia page says, "Which of these one calls 'the' geometric distribution is a matter of convention and convenience."
I find the two different versions confusing myself. When I read somewhere that $X$ has a "geometric distribution," the writer isn't always careful to specify which one is meant, and then I have to spend time figuring out which one it is. This confusion also spreads to the negative binomial distribution, which is a generalization of the geometric distribution. The exponential distribution, which is a continuous version of the geometric distribution, and the gamma distribution (a generalization of the exponential), have more than one definition, too. It's too bad that these weren't standardized with one definition for each, but part of the reason they weren't is that the different versions are useful in different scenarios.
In response to Josh Guffin's comment: Yes, in many contexts it is easy to figure out which geometric distribution a writer is referring to. However, in some it is not. Wikipedia's page on moment-generating functions is a classic example. The table there gives the mgf's for the negative binomial (and thus geometric), exponential, and gamma distributions, but it doesn't specify which convention for each one is being used.
Our sample space $\Omega$ is the set of all possible infinite sequences (tuples) of Heads (hereafter shorted as $H$) and Tails (hereafter shortened as $T$). A sequence can be thought of like a set with the following major differences: elements can be repeated and order of elements matters.
Consider a smaller example. Consider the following sample space of all possible results of three coin flips in sequence:
$$X = \{(\omega_1,\omega_2,\omega_3)~:~\omega_i\in \{H,T\}\}$$
The notation above is interpreted as meaning in words as "$X$ is the set of all possible triples of the form $(\omega_1,\omega_2,\omega_3)$ where each element of a triple is either an $H$ or a $T$. This could have been written explicitly as the following:
$$X = \left\{(H,H,H),(H,H,T),(H,T,H),(H,T,T),(T,H,H),(T,H,T),(T,T,H),(T,T,T)\right\}$$
"But this contradict with my understanding. Because in the example, both ω1 and ω2 can be H, and duplicate values are not allowed in a set."
Note that for example $(H,H,T)$ is a valid triple despite the fact that there are multiple $H$'s appearing and further it is considered a different triple than $(H,T,H)$ despite the fact that both have two $H$'s and one $T$ since the order in which they occur is different. Since the elements of our sample space are themselves sequences or tuples, we do not care if there are repeats within the sequences or tuples, just so long as the sequences or tuples themselves aren't repeated (and even that wouldn't be such a bad thing., we would just consider them as occurring only once).
Note, the difference in how these are enclosed. Sets are enclosed with curly brackets like so: $\{~~~\}$ while tuples and sequences are enclosed with circular brackets like so: $(~~~)$
Going back to your original example,
$$\Omega = \{(\omega_1,\omega_2,\omega_3,\dots)~:~\omega_i\in\{H,T\}\}$$
This, similarly to before is the set of all possible sequences where each element in the sequences come from the set $\{H,T\}$.
One such sequence might begin $(H,T,H,T,H,T,H,T,\dots)$ while another might begin $(H,H,H,H,H,H,\dots)$, etc...
"In that case the example is the book is showing us one sample outcome. The sample space containing all sample outcome needs to be written as $\Omega = \{ \omega_1, \omega_2,..., : \omega_i = (\omega_{i1}, \omega_{i2}, \omega_{i3},...,) : \omega_{ij} \epsilon \{H, T\}\}$ Am I thinking right?"
You are getting closer however there is a property about $\Omega$ which your notation above gets wrong. When we begin to write elements and taper off with ellipses (the three periods in a row ...) this implies that the list of elements is not only infinite but
countably infinite. Your attempt at notating this would have people look at a glance and think there are only countably many such possible infinite sequences of heads and tails. This is incorrect.
There are in fact uncountably infinitely many infinite sequences of heads and tails. As such, we cannot even begin to list them in a pattern which would eventually list them all. A variation on Cantor's Diagonal Argument will prove that. Alternatively, consider replacing $H$'s by $1$'s and $T$'s by $0$'s and interpret each sequence as a sequence of binary numbers occurring after the decimal. You will have described every possible real number between $0$ and $1$ (some of which twice), again showing $\Omega$ has cardinality at least as great as the continuum.
As such, when referring to the tuples in our set, we should avoid using the ellipses and instead just use set builder notation as we had before. In the original notation, we had not decided to give an arbitrary sequence a label, opting to just refer to it as $(\omega_1,\omega_2,\dots)$, but if you insist on giving these labels then we could do it as the following:
$$\Omega = \{\omega~:~\omega = (\omega_1,\omega_2,\omega_3,\dots),~\omega_i\in\{H,T\}\}$$
In words that is $\Omega$ is the set of all elements $\omega$, where $\omega$ is itself a countably infinite sequence of elements each of whom are either $H$ or $T$.
Compare this to the way the original was phrased: $\Omega$ is the set of all countably infinite sequences of elements each of whom are either $H$ or $T$.
The descriptions are hardly different and the rewriting you proposed was not necessary.
Best Answer
There are always infinitely many valid ways to choose the sample space. Here are four natural ones:
The set of all infinite sequences of $T$ and $H$. The function $X$ gives the number of $T$s at the beginning of the sequence. In particular, $X(T,T,T,\dots)=\infty$.
The set of all infinite sequences, except for the all tails sequence. Now, $X$ is a finite number for all inputs.
The set of all finite sequences whose last entry is $H$ and whose other entries are all $T$. Here, we are taking examples $1$ or $2$ and ignoring some information.
The set of nonnegative integers. $X$ is the identity function. The probability measure is $P(\{n\})=(1-p)^{n}p$, where $p$ is the probability of heads. This is the same as example $3$, with the correspondence $T^nH\longleftrightarrow n$.
It does not matter whether we include the all tails sequence because the probability of it is zero. You can remove any probability zero event from a sample space without changing anything.