Solved – Understanding the difference between Supervised and unsupervised learning

nonparametric-bayessupervised learningunsupervised learning

I have been reading about the Supervised and Unsupervised learning. What I came to know through this link is that in case of Supervised learning you have a set of input and a set of labels which are then used in the training phase whereas in case Unsupervised learning there are no labels. Now my basic question is:

  1. Since there are no labels available in case of Unsupervised learning so does that mean that there is no training phase in unsupervised learning?

  2. If there is a training phase in Unsupervised learning and we do not use any labels then what is the importance of training phase and how is it performed since we do not have any error or reward signal to evaluate a potential solution in training phase? Isn't it equal to the normal testing phase where we have the set of data but no labels and we predict the labels?

  3. I was reading about a Nonparametric bayesian modeling method and the introduction said that-"The method is unsupervised and has no free parameter that require tuning". In this what is the meaning of no free parameter that require tuning? Is it that there is no requirement of training phase?

Please bear with my lack of understanding about these topics as I am new to this and would love if you could provide some explanation to clear out my doubts.

Best Answer

1.

Lets look at a simple example of trying to predict housing prices. Assume we have a dataset that looks like

Cost |  Sq Ft  | N bedroom
 100K    1,800     4
 120K    1,300     3
 220K    2,200     5

In the case of supervised learning we would know the cost (these are our y labels) and we would use our set of features (Sq ft and N bedrooms) to build a model to predict the housing cost. The formula would look like

Cost ~ Sq Ft + N bedrooms

Now in unsupervised learning we would not know the cost of the house but we still would know the features. Therefore, we would train a model and try to group the types of houses together that are similar. For an example of this look at k-means clustering (http://scikit-learn.org/stable/modules/clustering.html#clustering)

This is a great, free, book which covers this very nicely (http://web.stanford.edu/~hastie/local.ftp/Springer/OLD/ESLII_print4.pdf)

2.

Each type of learning method (may) have a set of parameters which are called model parameters. The training phase is used to find out the optimal set of parameters which generalizes you data the best. That book also gives very nice information on different learning methods are there parameters.

For example, in the leaning algorithm called SVM there is a term that looks like $\exp(-\gamma|x-x^{2}|)$. In this example the $\gamma$ parameter is what we try to optimize using the training data.

Related Question