MATLAB: Should there be a residual when applying principal component analysis

Question

I am using Matlab function PCA (principal component analysis) to reduce the dimensionality of a data set with approximately 20 000 observations x 100 dimensions.

After having obtained the principal component coefficients of the data I recreated the input signal in the original coordinate system using the transformation matrix from the PCA function. This yielded a very large residual when comparing with the input signal. I have tried around with different data sets and sizes and it appears to be commonplace to have a large residual. I am not sure yet whether it is due to round-off errors or high SNR in the input data. The dimensionality reduction could of course still be useful, but is this something that one should be cautious about when performing principal component analysis? Or is there another metric that is better to assess the performance of the principal component analysis?

Answer 1

If you used all of the components, and you STILl have a large residual, then you are doing the reconstruction incorrectly. The residuals at that point should be on the order of eps. Have you verified that indeed, you did not get tiny numbers,and merely did not see the tiny power of 10 attached to the numbers? With no data and no indication of how you did the analysis or the reconstruction, we cannot know what you did incorrectly.

If I had to guess, you did not properly deal with the mean, or something silly like that.

My recommendation is you attach the data to a comment as a .mat file. Then I can do PCA, and the reconstruction, showing you what to do.