Solved – Whats the difference between a dense layer and an output layer in a CNN

conv-neural-networkmachine learningneural networks

All deeplearning4j CNN examples I have seen usually have a Dense Layer right after the last convolution or pooling then an Output Layer or a series of Output Layers that follow.

What is really the difference between a Dense Layer and an Output Layer in a CNN also in a CNN with this kind of architecture may one say the Fullyconnected Layer = Dense Layer+ Output Layer / Fullyconnected Layer = Dense Layer alone.

Best Answer

Short:
Dense Layer = Fullyconnected Layer = topology, describes how the neurons are connected to the next layer of neurons (every neuron is connected to every neuron in the next layer), an intermediate layer (also called hidden layer see figure)

Output Layer = Last layer of a Multilayer Perceptron

Long:
The convolutional part is used as a dimension reduction technique to map the input vector X to a smaller one. For example your input is an image with a size of (227*227) pixels, which is mapped to a vector of length 4096. $${\bf{X} : \mathbb{R}^{51529} \mapsto \mathbb{R}^{4096}}$$ This makes things easier for the second step, the classification/regression part. Therefore a classifier called Multilayer perceptron is used (invented by Frank Rosenblatt).
If you stack multiple layers on top you may ask how to connect the neurons between each layer (neuron or perceptron = single unit of a mlp). Indeed there are more options than connecting every neuron to every new one = dense or fullyconnected (other possible topologies: shortcuts, recurrent, lateral, feedback). In the most examples the intermediate layers are desely or fully connected.
The last neuron stack, the output layer returns your result. The output neurons are chosen according to your classes and return either a descrete vector or a distribution.

image

Related Question