I'm trying to get a basic understanding of AutoEncoders. Basically they are neural networks with a very low representation of the original input in some hidden layer, and then a final layer which has exact same dimension as the input but 'reconstructed' from the latent lower-dimensional representation. What I don't understand is what should we do with autoencoders, apart from denoising or compression. For example, would it be possible to use the latent representation as a dimensionality-reducted version of the original input, and then using it to train some Machine Learning Model?
Many thanks,
James
Best Answer
Yes, dimension reduction is one way to use auto-encoders. Consider a feed-forward fully-connected auto-encoder with and input layer, 1 hidden layer with $k$ units, 1 output layer and all linear activation functions. The latent space of this auto-encoder spans the first $k$ principle components of the original data. This can be useful if you want to represent the input in fewer features, but don't necessarily care about the orthogonality constraint in PCA. (More information: What're the differences between PCA and autoencoder?)
But auto-encoders allow a number of variations on this basic theme, giving you more options for how the latent space should be constructed than does PCA.