Model: Assume we want to learn patterns using an autoencoder neural network. In the simplest case, such a network is "shallow" with 1 hidden layer, takes a $d$-dimensional numerical input vector $x$, feeds it through the network and compares $x$ with the resulting $\tilde{x}$ at the output layer, where some information is lost between $x$ and $\tilde{x}$ due to e.g. a 'bottleneck' in the hidden layer (smaller number of neurons than in the input and output layer, forcing data compression). It is trained to reconstruct $x$ at the output layer by minimizing the mean-squared-error (MSE) between $x$ and $\tilde{x}$. After training, samples with small error would be "in line" with learned patterns, while larger errors would point to deviations.
Typical Application: This technique is often applied to images with a relatively large input vector dimension $d$. For example, the MNIST dataset of handwritten digits will have input vector size $d=28\times 28=784$ for every image.
Question: However, this technique could serve many other pattern recognition tasks, most of which have likely lower dimensional input size. Is there a minimum size of $d$ for the technique to work? For example, would an input layer with size $d\leq10$ still be feasible, where the data would be numerical time series data?
Best Answer
There is no minimum dimension for an autoencoder, you could have even 2 dimensions being embedded in one, and this would work.
Do note that the simplest autoencoder has 2 layers, not 3, one to connect the input to the embedded representation, and one to connect the embedded representation to the output.