Assuming that I have a trained deep learning network that can detect 10 classes of objects (road, sky, tree, etc.) in images. It takes in RGB images and outputs a probability map of size (img_col, img_row, n_class), and the final segmentation will be a argmax operation over the last dimension.
Now I want to add a new class to the network, for example, pedestrians, so that after training, the network will be able to detect pedestrians in images.
But I no longer have the old training data. Instead, I've got a new dataset which also contains pedestrians, but only the pedestrians are labeled. Training with the new dataset (using the old weights as initialization) will be the most straightforward way, but I would like to hear some other approaches.
Could anyone share some thoughts on how to realize this?
Best Answer
You should, at least, re-train the classification layer. When you add another output, during learning the new class, the other class activation must shrink either explicitly (sigmoid) or implicitly (softmax). However, it may be better to learn, at least, last feature layer as there would be some useful features to recognize pedestrians.
Another approach can be feeding the new class to the network and collect confidence from the output. Low confidence can be indication of another class that is not belong to any of the classes learnt before. For sure this method can also give low confidence to another class other than pedestrian or any class the network learnt on. Also, NN is a non-local generalization method. It is prone to classify a totally garbage image with high confidence (See adversarial examples if you are curious).