I have known the definition of classical probability as follows:
If a random experiment $A$ results in $n$ mutually exclusive and exhaustive and equally likely outcomes, out of which $m$ of them are favorable to the happening of event $A$ then the probability of event $A$ is denoted by $P(A)$ and defined as
$$P(A)=\frac{m}{n}$$
Easier texts just say that the sample space $S$ contains all possible outcomes of an experiment. And then they say if each of the outcomes are equally likely then probability is defined as $P(A)= \frac{|A|}{|S|}$. This definition simply avoids the use of the terms "mutually exclusive" and "exhaustive". I had seen this type of definition during the days of my middle school.
That the outcomes need to be exhaustive is clear to me. We are not supposed to leave out any outcome of an experiment from being included in the sample space.
But what I do not quite get the is meaning or use of the term "mutually exclusive" outcomes. When I had first seen the use of this term, I interpreted as follows suppose an experiment can result in each of the outcomes either in set $S_1=\{a,b,c\}$ or in set $S_2=\{b,c,d\}$ then the exhaustive sample space should be $S=\{a,b,c,d\}$ and the number of mutually exclusive outcomes are $n=|S_1|+|S_2|-|S_1 \cap S_2|$. i.e. I thought that we need to remove the possible overlap while considering the sample space as a whole.
Recently I came across a possible explanation of "mutually exclusive" outcomes as follows:
It means that the outcomes of an experiment are mutually exclusive [$\text{Outcome}_i \cap \text{Outcome}_j=\phi; \quad i\neq j$], i.e. an experiment cannot result in two outcomes simultaneously.
Can anyone point me out what is correct interpretation along with the intuition or mathematical logic/reasoning behind the same.
Best Answer
The sets $\{hh,ht\},\{th,tt\},\{hh,ht,th\}$ are called events, their elements $hh,ht,th,tt$ are called outcomes, and the single-outcome events $\{hh\},\{ht\},\{th\},\{tt\}$ are called elementary events.
When a probability experiment is performed, a single outcome happens; an event is said to happen if it contains this outcome.
By definition, events being mutually exclusive means that their pairwise joint probabilities are $0,$ that is, each pair҂ almost never or never both happen: $$\text{Events $X$ and $Y$ are mutually exclusive}\\\overset{\text{def}}{\iff}\\ P(X\cap Y)=0.$$
Clearly, $$\text{$X$ and $Y$ are disjoint (i.e., $X\cap Y=\emptyset)$}\\\implies\text{$X$ and $Y$ are mutually exclusive}.$$
On the other hand,$$\text{$X$ and $Y$ are mutually exclusive}\\\kern.6em\not\kern-.6em\implies\text{$X$ and $Y$ are disjoint, i.e., $X$ and $Y$ both happening is impossible}.$$ For example, the events $(1,2]$ and $[2,3)$ are mutually exclusive but not disjoint.
҂ On the other hand, (mutual) independence for multiple events requires more than just pairwise independence.
The author is referring to elementary events like $\{ht\}$ as “outcomes”.
As such, since they are disjoint, from #3, they are indeed mutually exclusive.
For finite sets, $$|X\cup Y|=|X|+|Y|-|X\cap Y|;$$ in particular, $$\text{$X$ and $Y$ are disjoint}\iff |X\cup Y|=|X|+|Y|.$$ Generally, $$P(X\cup Y)=P(X)+P(Y)-P(X\cap Y);$$ in particular, from #3, $$\text{$X$ and $Y$ are mutually exclusive}\iff P(X\cup Y)=P(X)+P(Y).$$