intuitionmultivariate analysisnormal distributionterminology
Intuitively, the mean is just the average of observations. The variance is how much these observations vary from the mean.
I would like to know why the inverse of the variance is known as the precision. What intuition can we make from this? And why is the precision matrix as useful as the covariance matrix in multivariate (normal) distribution?
Insights please?
Imagine $X$ and $Y$ are very highly correlated. Imagine $Y$ and $Z$ are very highly correlated. Imagine further, than aside from their mutual correlation with $Y$, $X$ and $Z$ have no further correlation. That is, their partial correlation is $0$.
In fact, let's construct some variables. I'll do it in R:
y <- rnorm(100) # generate 100 random standard normals
x <- .99*y+sqrt(1-.99^2)*rnorm(100) # x and y have population corr = 0.99
z <- .99*y+sqrt(1-.99^2)*rnorm(100) # z and y have population corr = 0.99
cor(x,z)
[1] 0.9830544
The population correlation between $X$ and $Z$ is $0.99^2 = 0.9801$; the sample correlation reflects that. The (population) partial correlation here is 0; the only source of correlation between $X$ and $Z$ is through the fact they they're both related to $Y$.
In your problem you have an additional variable, and all your variables are pairwise correlated with the next and previous variables in the list, but have no partial correlations with variables 'further away'. In effect something similar to what was just described above occurs, but with more variables involved. The partial correlation structure you have set up cause the variables each to be highly correlated with all the others, even the ones they're not directly related to.
Instead of jumping to the multivariate case in matrix form, look at the bivariate case first:
Can you recognize the portion of the denominator that is the determinant of the variance-covariance matrix below?
In the univariate case you don't have a determinant because $\sum$ consists of just one term. You don't have another variable, so you don't need to take into account any interaction between them.
Best Answer
Precision is often used in Bayesian software by convention. It gained popularity because gamma distribution can be used as a conjugate prior for precision.
Some say that precision is more "intuitive" than variance because it says how concentrated are the values around the mean rather than how spread they are. It is said that we are more interested in how precise is some measurement rather than how imprecise it is (but honestly I do not see how it would be more intuitive).
The more spread are the values around the mean (high variance), the less precise they are (small precision). The smaller the variance, the greater the precision. Precision is just an inverted variance $\tau = 1/\sigma^2$. There is really nothing more than this.