Right now I am working on a problem where I have sets of 5 profile pictures of people. I am trying to classify those pictures using a CNN which feeds into an RNN. As a whole the set of 5 images should fall into one class although there may be some outliers within the 5 (like someone who posts a picture of their dog rather than themselves.) Ideally the network should be able to learn to ignore these. Right now this CNN->RNN approach works decently well. However, in an RNN the order in which the images are presented seems to matter. If I shuffle the profile images and feed them to the network the prediction probability tends to vary by about 5%. I was wondering if there is an architecture other than RNNs where the whole sequence can be considered yet where the ordering of the sequence does not matter?
Solved – Neural Networks, what is the best way to deal with sequence when order does not matter
classificationneural networksrecurrent neural network
Best Answer
In case of RNN order matters. hence your architecture will not work well when you shuffle the images and then tries to predict a sample. For this problem, RNN is not required at all and they are not even a better choice for such kind of problems. The best architecture for your problem is the use of CNNs only. It will give you an exceptional accuracy if you implement it correctly. In order to get an overview how it works, you can take a look at the example of CIFAR10 small images classification: Convolutional Neural Network (CNN) with realtime data augmentation given here