Solved – Can the output layer size be bigger than input layer size in a neural network

lstmneural networks

Does the number of outputs of a neural network have to be smaller than the number of inputs? This seems to be the case in most of the applications that I have seen . At the same time, I am not sure how this might be the case in language translation models. It is not guaranteed that the output (the translation of a text) would be the same or smaller length as the input text. It can be longer or shorter.

Best Answer

No! There is no limit whatsoever on the size of the output relative to the size of the input.

But in most cases, a higher number of outputs is not necessary at all.

In the case of language translation models: this is either done fragment by fragment, or the output has a fixed maximum size (e.g. 2x input size).

Related Question