I'm currently trying to better understand the fundamental notions of statistics.
Navigating through multiple sites, I've found this formula for joint probability.
P(A ∩ B) = P(A)*P(B)
Namely, if we want to know what is the probability of A and B to co-occur, we just multiply their individual (marginal) probabilities.
As for the conditional probability, the formula stands like this:
P(A | B) = P(A∩B) / P(B)
So, if we want to know the probability of A when we already know B, we divide the joint probability of A and B by the probability of B.
If we develop this formula, we will get:
P(A | B) = P(A)*P(B)/P(B)
Further, we can simplify the fraction by dividing it by P(B), and we are left with
P(A | B) = P(A)
Suppose we want to estimate the probability of drawing from a deck of cards a red colored card of 4 (hearts or diamonds), while already knowing it's colored red.
Individual (marginal) probabilities
P(red) = 1/2 = 0.5
P(4) = 4/52 = 1/13
Joint Probability (to draw a red 4)
P(4 ∩ red) = P(A)*P(B) = 1/2 * 1/13 = 1/26
Conditional probability (prob to draw a 4 already knowing it is red)
P(4 | red) = P(4 ∩ red) / P(red) = P(4)*P(red)/P(red)
P(4 | red) = 1/13*0.5/0.5 = 1/13
If we know that the card we are about to draw is red, the probability of it being a red 4 is 1/13 (or 7.6%), which is just the probability of drawing a 4 from the deck. This makes sense because we have eliminated half of the possibilities (black cards), and we are actually drawing from a set of 26 cards that has 2 fours in it. So, 2 out of 26 (or 1 of 13) of the drawn cards will be a red 4.
The results look fine, but it really seems that I am missing something, because in the relationship P(A | B) = P(A) doesn't take into account the probability of B.
I will be highly grateful is someone could look into my question and give a response.
Thanks in advance!
Best Answer
$P(A\cap B)=P(A)P(B)$ is true if and only if $A$ and $B$ are independent. So, $P(A|B)\neq P(A)$ in general. In your example, it holds because the events are fortunately independent. You already informally found it in your reasoning paragraph, by thinking about the sample space.
For a simple counter example, consider the case where $A=B$. $P(A|B)=1$ clearly, because if you know that $B=A$ happened, the probability of $A$ happening is $1$.