Hi guys, can help me to understand the notation we used to represent V "Explaining the formulas, Visualization, …", I got the idea of Expectation Value E but, I did not get Conditional Variance. Thanks!
[Math] Conditional Variance For Discrete & Continous Random Variable X
statisticsvariance
Related Solutions
In the law of total variance $$\operatorname{Var}[X] = \operatorname{E}[\operatorname{Var}[X \mid T]] + \operatorname{Var}[\operatorname{E}[X \mid T]]$$ where I have used $T$ to denote the type of policy, with $$T \sim \operatorname{Categorical}(\pi_1 = 0.1, \pi_2 = 0.5, \pi_3 = 0.4), \\ \Pr[T = i] = \pi_i,$$ where the coding is $A \equiv 1$, $B \equiv 2$, and $C \equiv 3$, the first component $$\operatorname{E}[\operatorname{Var}[X \mid T]]$$ is what we call the "within-group" variance; this is the mean of the variability of $X$ that is attributable to each group. The second component $$\operatorname{Var}[\operatorname{E}[X \mid T]]$$ is what we call the "between-groups" variance; this is the variance of the conditional means of $X$ for each group; i.e., the variability of the means between groups.
How do we compute each of these? Since $X \mid T$ is Poisson, specifically $$X \mid T \sim \operatorname{Poisson}(\lambda_T)$$ where $\lambda_1 = 1, \lambda_2 = 2, \lambda_3 = 10$, and both the mean and variance of a Poisson distribution are equal to its rate parameter, we have $$\operatorname{E}[X \mid T] = \operatorname{Var}[X \mid T] = \lambda_T.$$ Consequently $$\operatorname{E}[\operatorname{Var}[X \mid T]] = \operatorname{E}[\lambda_T] = \lambda_1 \pi_1 + \lambda_2 \pi_2 + \lambda_3 \pi_3 = 5.1,$$ and $$\operatorname{Var}[\operatorname{E}[X \mid T]] = \operatorname{Var}[\lambda_T] = \operatorname{E}[\lambda_T^2] - \operatorname{E}[\lambda_T]^2 = (\lambda_1^2 \pi_1 + \lambda_2^2 + \pi_2 + \lambda_3^2 \pi_3) - (\lambda_1 \pi_1 + \lambda_2 \pi_2 + \lambda_3 \pi_3)^2 = 16.09.$$
Failing to take into account variability due to differences between group means--i.e, the second variance component--you are just computing how much variation arises from each individual group, ignoring that the groups may be located far apart from each other, as in this case where Type $C$ policies have a mean annual claim rate of $10$, far more than the other two types. Another way to see this is that if $\lambda_1, \lambda_2, \lambda_3$ are all clustered very "close together," i.e., there is very little variability of the mean claim rates by policy type, then the second component will be very small relative to the first component.
This equation $$\mathsf{Var}(A)=\mathbb{E}[XY]^2-(\mathbb{E}[XY])^2=\mathbb{E}[X^2Y^2]-(\mathbb{E}[X]\mathbb{E}[Y])^2=\mathbb{E}[X^2]\mathbb{E}[Y^2]=8*10=80$$ contains typographical errors. The corrected equation, with corrections in red, should look like this: $$\mathsf{Var}(A)=\mathbb{E}[\color{red}{(}XY\color{red}{)^2}]-(\mathbb{E}[XY])^2=\mathbb{E}[X^2Y^2]-(\mathbb{E}[X]\mathbb{E}[Y])^2=\mathbb{E}[X^2]\mathbb{E}[Y^2]=8*10=80.$$
The reason is akin to the reason why $f(x)^2$ is taken to mean $(f(x))^2$, rather than $f(x^2)$.
The conditional variance $$\mathsf{Var}(A \mid Y = 1)$$ is straightforward: Given that $Y = 1$, then $A = XY = X$, so $$\mathsf{Var}(A \mid Y = 1) = \mathsf{Var}(X \mid Y = 1) = \mathsf{Var}(X),$$ where the last equality that states that the conditional variance of $X$ given $Y = 1$ is equal to the unconditional variance of $X$, holds because $X$ and $Y$ are independent.
Best Answer
Are you familiar with the definition of variance? $$V(X) := E[(X-E[X])^2] = E[X^2] - E[X]^2.$$ It is the expected square distance of $X$ from its mean. The last expression $E[X^2] - E[X]^2$ is a common way to compute the variance.
Conditional variance extends this notion with conditioning on some event or random variable. Essentially, it is the same as variance, but conditioned on $A$. Note that the formula simply takes $E[X^2] - E[X]^2$ but replaces each expectation with the conditional expectation to get $E[X^2 \mid A] - E[X \mid A]^2$.
$$V(X \mid A) := E[(X-E[X])^2 \mid A] = E[X^2 \mid A] - E[X \mid A]^2.$$