In a particular application I was in need of machine learning (I know the things I studied in my undergraduate course). I used Support Vector Machines and got the problem solved. Its working fine.
Now I need to improve the system. Problems here are
-
I get additional training examples every week. Right now the system starts training freshly with updated examples (old examples + new examples). I want to make it incremental learning. Using previous knowledge (instead of previous examples) with new examples to get new model (knowledge)
-
Right my training examples has 3 classes. So, every training example is fitted into one of these 3 classes. I want functionality of "Unknown" class. Anything that doesn't fit these 3 classes must be marked as "unknown". But I can't treat "Unknown" as a new class and provide examples for this too.
-
Assuming, the "unknown" class is implemented. When class is "unknown" the user of the application inputs the what he thinks the class might be. Now, I need to incorporate the user input into the learning. I've no idea about how to do this too. Would it make any difference if the user inputs a new class (i.e.. a class that is not already in the training set)?
Do I need to choose a new algorithm or Support Vector Machines can do this?
PS: I'm using libsvm implementation for SVM.
Best Answer
For (1), as ebony1 suggests, there are several incremental or on-line SVM algorithms you could try, the only thing I would mention is that the hyper-parameters (regularisation and kernel parameters) may also need tuning as you go along as well, and there are fewer algorithmic tricks to help with that. The regularisation parameter will almost certainly benefit from tuning, because as the amount of training data increases, the less regularisation that is normally required.
For (2) you could try fitting a one-class SVM to the training data, which would at least tell you if the data were consistent with the classes you do know about, and then classify as "unknown" if the output of the one-class SVM was sufficiently low. IIRC libsvm has an implementation of one-class SVM.
For (3) if you just use 1-v-1 class component SVMs, then you just need to make three new SVMs, one for each of the unknown-v-known class combinations, and there is no need to retrain the others.
HTH