[Math] calculate mutual information using joint probability distribution table

information theory

let us suppose we have following joint probability distribution table
enter image description here

i would like to calculate mutual information between two variable, first of all i have calculated marginal distributions
enter image description here

and i used following formula
enter image description here

which i have considered as this one
enter image description here

Using excel I have calculated following table
enter image description here

zero is written because it is accepted generally that

$0*log(0)=0$

so finally i got following result

0.334497797

because it is different from zero, that means that these two random variable are not independent right?

Best Answer

It's all right.

because it is different from zero, that means that these two random variable are not independent right?

Yes. That can also be seen by checking that the joint probability doesn't equal the product of the marginals (at least for some values). For example $P(X=0,Y=0)=0.2$ while $P(X=0)P(Y=0)= 0.5 \times 0.3 = 0.15$

Related Question