Solved – Problems with implementing PCA in Matlab

machine learningMATLABpca

I'm implementing PCA using eigenvalue decomposition in Matlab. I know Matlab has PCA implemented, but it helps me understand all the technicalities when I write code.
I've been following the guidance from here, but I'm getting different results in comparison to built-in function princomp.

Could anybody look at it and point me in the right direction.

Here's the code:

function [mu, Ev, Val ] = pca(data)

% mu - mean image
% Ev - matrix whose columns are the eigenvectors corresponding to the eigen
% values Val 
% Val - eigenvalues

if nargin ~= 1
 error ('usage: [mu,E,Values] = pca_q1(data)');
end

mu = mean(data)';

nimages = size(data,2);

for i = 1:nimages
 data(:,i) = data(:,i)-mu(i);
end

L = data'*data;
[Ev, Vals]  = eig(L);    
[Ev,Vals] = sort(Ev,Vals);

% computing eigenvector of the real covariance matrix
Ev = data * Ev;

Val = diag(Vals);
Vals = Vals / (nimages - 1);

% normalize Ev to unit length
proper = 0;
for i = 1:nimages
 Ev(:,i) = Ev(:,1)/norm(Ev(:,i));
 if Vals(i) < 0.00001
  Ev(:,i) = zeros(size(Ev,1),1);
 else
  proper = proper+1;
 end;
end;

Ev = Ev(:,1:nimages);

Best Answer

The line [Ev,Vals] = sort(Ev,Vals); probably does not do what you think it should. The sort command would have to be applied separately. Moreover, eig returns a diagonal matrix of eigenvalues, which you have to strip out to a vector. Probably you want to do the following:

[Ev, Vals]  = eig(L);
Vals = diag(Vals);        %strip out the eigenvalues;
[Vals,sidx] = sort(Vals); %do the sort
Ev = Ev(:,sidx);          %apply the sort to the eigenvectors as well.

I would guess this is actually unecessary, because eig returns the eigenvectors, -values, in ascending order anyway.

On a stylistic note, you can subtract out the mean using bsxfun as follows:

X = bsxfun(@minus,X,mu);

instead of calling the for-loop.

Related Question