Solved – Prediction vs. Classification in neural networks

classificationneural networksprediction

I am learning the backpropagtion algorithm, and would like to clarify some concepts.

Suppose my training data set consists of 20-dimensional bit strings that are classified into 5 different classes. Then my neural net has 20 inputs, and 5 outputs: 

10000
01000
00100
00010
00001

Is this correct? What if some new test data yields something like $11000$. How does one interpret this? The data can be in class 1 or 2?

Now let's consider a slightly different problem: Suppose I have inputs consisting of 20-dimensional bit strings, and their outputs are 5-dimensional bit strings. For example, 

00000000001111111111 --> 01001
11111000001111100000 --> 11000

etc…

I want a network to make these sorts of predictions. So my network should still have 20 inputs, but can I still just use 5 outputs? Following the first example, it looks like I should technically have $2^5$ outputs, each corresponding to one of the possible outputs.

What is the relationship between these classification/prediction problems? Are my setups correct?

Best Answer

11000 is an invalid class if your classes are mutually exclusive and if you have only five classes consisting of 10000,01000,00100,00010, and 00001.

11000 is no longer an invalid class once you define your class space to be all digits between 0 and 2^5. Ultimately it's up to you to decide what types of classes you would like to classify. Likewise for inputs: you can use however many inputs you would like as long as you are consistent between training and test/predicting.

Your dataset sounds very abstract and that could be what is clouding your thinking. If you're just starting out with machine learning and neural networks, go with a well known dataset first so that you only need to think about the algorithm. A good example is the MNIST handwritten digits dataset. All the classes are digits 0-9 (10 total) and so there is no confusion about classes falling outside of 0,1,2,3,4,5,6,7,8,9.

Google's Tensorflow library has a good MNIST tutorial here: https://www.tensorflow.org/versions/r0.7/tutorials/mnist/pros/index.html.

Related Question