MATLAB: How to handle NaNs in the response data in deep neural networks

Question

I want to train a deep neural network for classifying images. In my database some samples correspond to "unknown" or "don't-care" classes. For the classic neural networks, the following link

https://www.mathworks.com/help/nnet/ug/choose-neural-network-input-output-processing-functions.html#bss33zw-8

mentions that such classes can be represented by NaN. However, for deep neural networks that is not an option as the responses cannot contain any NaN as per:

https://www.mathworks.com/help/nnet/ref/trainnetwork.html#mw_d0b3a2e4-09a0-42f9-a273-2bb25956fe66

How to handle NaNs in the response data in deep neural networks if my output response contains unknown values?

Answer 1

If you have data for "unknown" classes that you do not care about, you should remove them from the training set when using the deep neural networks.

The link mentioned in the question is for the classic neural networks and not deep neural networks. That page is referring to situations where you have a network with multiple outputs, but an instance only has a label for one those outputs. Then you can pass NaN for the labels for the other outputs, and the instance will only contribute to the loss for one of the outputs. This situation does not occur in deep learning yet because we do not support deep networks with multiple outputs.