Correlation Analysis – Interpretation of Partial Correlation Explained in Detail

correlationinterpretationpartial-correlation

I was calculating a correlation between two variables (A and B) which revealed these variables are highly correlated. I know that one variable is also highly correlated with another one (C), therefore I did a partial correlation between A and B controlling for C. Now I receive a even higher correlation between A and B than I did before.
– How can I interpret this?

Best Answer

For understanding this I always prefer the cholesky-decomposition of the correlation-matrix.
Assume the correlation-matrix R of the three variable $X.Y.Z$ as $$ \text{ R =} \left[ \begin{array} {rrr} 1.00& -0.29& -0.45\\ -0.29& 1.00& 0.93\\ -0.45& 0.93& 1.00 \end{array} \right] $$ Then the cholesky-decomposition L is $$ \text{ L =} \left[ \begin{array} {rrr} X\\ Y \\ Z \end{array} \right] = \left[ \begin{array} {rrr} 1.00& 0.00& 0.00\\ -0.29& 0.96& 0.00\\ -0.45& 0.83& 0.32 \end{array} \right] $$ The matrix L gives somehow the coordinates of the three variables in an euclidean space if the variables are seen as vectors from the origin, where the x-axis is identified with the variable/vector X and so on.

Then the correlations of X and Y is $\newcommand{\corr}{\rm corr} \corr(X,Y)=x_1 y_1 + x_2 y_2 + x_3 y_3 $ and we see immediately it it $\corr(X,Y)=-0.29 $ because of the zeros and the unit-factor. We see also immediately the correlation $\corr(X,Z)=-0.45$ again because of the zeros and the unit-cofactor. However, the correlation between Y and Z is $\corr(Y,Z) = -0.29 \cdot -0.45 + 0.96 \cdot 0.83$ The partial correlation (after X is removed) is that part for which no variance in the X-variable is present, so $\corr(Y,Z)._X = 0.96 \cdot 0.83 $. Now imagine, the value $0.83$ would be $-0.83$ instead. Then the partial correlation would be negative and the correlation between Y and Z were $ 0.29 \cdot 0.45 - 0.96 \cdot 0.83$

What we see is, that the partial correlations are partly independent from the overall correlations (though within some bounds)