MATLAB: How to visualize the contributive factors and distribution of coefficients in the “coeff” matrix output by “pca”

Statistics and Machine Learning Toolbox

How can I visualize the distribution of weights/coefficients in the "coeff" matrix output by "pca" (used for Principal Component Analysis)? I have a large number of features / predictor variables, and I am interested in visualizing the contributive features for the principal components. For instance, can I visualize this via a heatmap?

Best Answer

One possible approach is to provide the "coeff" matrix as the input to the "heatmap" function. The x-axis would represent the principal components, while the y-axis would represent the predictor variables. The heatmap would indicate the distribution of the contributive features for each principal component.

In order to illustrate, here is an example:

load hald
coeff = pca(ingredients)
heatmap(coeff,'XLabel','Principal Components','YLabel','Variables');

For more details about the "heatmap" function, you can refer to the following link:

<https://www.mathworks.com/help/releases/R2018b/matlab/ref/heatmap.html#bvlbg64-1>

Note that the "biplot" function provides another way to visualize the magnitude and sign of each variable's contribution to the first two or three principal components:

<https://www.mathworks.com/help/releases/R2018b/stats/biplot.html>

<https://www.mathworks.com/help/releases/R2018b/stats/pca.html#btjpztu-1>

Related Solutions

MATLAB: Make Sample Covariance/Correlation Matrix Positive Definite

The fastest way for you to check if your matrix "A" is positive definite (PD) is to check if you can calculate the Cholesky decomposition (A = L*L') of it. You can calculate the Cholesky decomposition by using the command "chol(...)", in particular if you use the syntax :

[L,p] = chol(A,'lower');

you get a lower trianglular matrix "L"; if the decomposition exists (your matrix is PD) "p" will equal 0. If "A" is not positive definite, then "p" is a positive integer. For more details about this please refer to documentation page:

https://www.mathworks.com/help/matlab/ref/chol.html

Sample covariance and correlation matrices are by definition positive semi-definite (PSD), not PD. Semi-positive definiteness occurs because you have some eigenvalues of your matrix being zero (positive definiteness guarantees all your eigenvalues are positive). If you correlation matrix is not PD ("p" does not equal to zero) means that most probably have collinearities between the columns of your correlation matrix, those collinearities materializing in zero eigenvalues and causing issues with any functions that expect a PD matrix.

To fix this the easiest way will be to do calculate the eigen-decomposition of your matrix and set the "problematic/close to zero" eigenvalues to a fixed non-zero "small" value. That can be easily achieved by the following code, given your initial correlation matrix "A":

[V,D] = eig(A);       % Calculate the eigendecomposition of your matrix (A = V*D*V')                       % where "D" is a diagonal matrix holding the eigenvalues of your matrix "A"d= diag(D);           % Get the eigenvalues in a vector "d" d(d <= 1e-7) = 1e-7;  % Set any eigenvalues that are lower than threshold "TH" ("TH" here being                       % equal to 1e-7) to a fixed non-zero "small" value (here assumed equal to 1e-7)D_c = diag(d);        % Built the "corrected" diagonal matrix "D_c"A_PD = V*D_c*V';      % Recalculate your matrix "A" in its PD variant "A_PD"

Take note that due to issues of numeric precision you might have extremely small negative eigenvalues, when you eigen-decompose a large covariance/correlation matrix. These extremely small negative eigenvalues are "machine zeros". The work-around present above will also take care of them.

This work-around does not take care of the conditioning number issues; it does reduces it but not substantially. Additionally the Frobenius norm between matrices "A_PD" and "A" is not guaranteed to be the minimum. A more mathematically involved solution is available in the reference: "Nicholas J. Higham - Computing the nearest correlation matrix - a problem from finance", IMA Journal of Numerical Analysis Volume 22, Issue 3, p. 329-343 (pre-print available here: http://eprints.ma.man.ac.uk/232/01/covered/MIMS_ep2006_70.pdf )

MATLAB: Do I get an error when setting the LineStyle property for multiple lines in MATLAB 7.4 (R2007a)

The "LineStyle" property for a line object only accepts values of '-', '--', ':', '-.' or 'none'. You can use the "Marker" property to separately set marker symbols. To illustrate, here is an example:

h = plot(rand(10,6))
set(h, {'LineStyle'}, {'-';'--';'--';':';':';':'})
set(h, {'Marker'}, { 'none'; 'o'; 'x'; 'none'; 'none'; 'none'; })

For further help with Line properties in R2007a, refer to the corresponding documentation:

https://www.mathworks.com/help/releases/R2007a/techdoc/index.html?/help/releases/R2007a/techdoc/ref/line_props.html

For your reference, here is a link to the corresponding documentation for a more recent MATLAB version, e.g. R2018b:

https://www.mathworks.com/help/releases/R2018b/matlab/ref/matlab.graphics.chart.primitive.line-properties.html

Best Answer

Related Solutions

MATLAB: Make Sample Covariance/Correlation Matrix Positive Definite

MATLAB: Do I get an error when setting the LineStyle property for multiple lines in MATLAB 7.4 (R2007a)

Related Question