Solved – the PCA representation of an image

image processingmachine learningpca

I've read a lot about how PCA is used to reduce the dimensionality of data, including this great answer and this mathy post. But I'm unclear what this does when applied to image data?

For example, given a set of vectors from images of faces, I could reduce the dimensions:

pca = PCA(n_components=100, svd_solver='randomized', whiten=True)
pca.fit(faces)
compressed = pca.transform(faces)

But when I look at the results, I see nothing like an image:

example = compressed[0]
example *= 255.0
example = example.astype('uint8')
example = example.reshape((10,10))
output = Image.fromarray(example)
output = output.resize((200,200))
output.save('ExamplePCA.jpg')

example of pca to image

Can someone describe for me (someone with limited matrix math/linear algebra knowledge) what is being represented here?

Best Answer

The transformed coordinates are not supposed to look anything like a natural image, even if you keep them all (number of components = number of pixels). In particular, all the different elements ("pixels") are uncorrelated.

When compressing an image this way, you are saying that each image in your dataset has many pixels, but that the images are different from each other in only 100 ways. You find 100 basis images that represent a typical image well, and then each reconstructed image is a linear combination of these 100 basis images, and the 100 numbers in the compressed vector are the coefficients multiplying each basis image. In order to understand what each "pixel" in the compressed vector means you must plot these basis images. To do this in python, try reshaping the rows of pca.components_

When dealing with face images, these basis images are sometimes called eigenfaces.

If you want to see your images projected on the low-dimensional principal subspace, then after applying:

compressed = pca.transform(faces)

you need to apply

decompressed = pca.inverse_transform(compressed)

and plot the decompressed images.

Related Question