classificationmachine learningregressionsupervised learningunsupervised learning
I'm newbie in AI
I know that Supervised Learning algorithms are divided into Classification and Regression algorithms.
But is that true of all machine learning algorithms, not just Supervised Learning? Are there any other categories than Classification and Regression?
It seems as if deep learning might be very interesting for you. This is a very recent field of deep connectionist models which are pretrained in an unsupervised way and fine tuned afterwards with supervision. The fine tuning requires a much less samples than the pretraining.
To wet your tongue, I recommend [Semantig Hashing Salakhutdinov, Hinton. Have a look at the codes this finds for distinct documents of the Reuters corpus: (unsupervised!)
If you need some code implemented, check out deeplearning.net. I don't believe there are out of the box solutions, though.
Machine learning (ML) in practice depends on what the goal of doing ML is. In some situations, solid pre-processing and applying a suite of out-of-the-box ML methods might be good enough. However, even in these situations, it is important to understand how the methods work in order to be able to troubleshoot when things go wrong. However, ML in practice can be much more than this, and MNIST is a good example of why.
It's deceptively easy to get 'good' performance on the MNIST dataset. For example, according to Yann Le Cun's website on MNIST performance, K nearest neighbours (K-NN) with the Euclidean distance metric (L2) also has an error rate of 3%, the same as your out-of-the-box random forest. L2 K-NN is about as simple as an ML algorithm gets. On the other hand, Yann, Yoshua, Leon & Patrick's best, first shot at this dataset, LeNet-4, has an error rate of 0.7%, 0.7% is less than a fourth of 3%, so if you put this system into practice reading handwritten digits, the naive algorithm requires four times as much human effort to fix its errors.
The convolutional neural network that Yann and colleagues used is matched to the task but I wouldn't call this 'feature engineering', so much as making an effort to understand the data and encode that understanding into the learning algorithm.
So, what are the lessons:
Best Answer
All unsupervised algorithms, e.g.
Some of them might internally use regression or classification elements, but the algorithm itself is neither.