Solved – Can we say that RNN for time series is an example of semi-supervised learning

neural networkssemi-supervised-learningsupervised learning

I am learning neural nets, esp. focusing on RNN for my research problem. This question has nothing exactly to do with my research.

With my understanding of RNN, I can think of it as an example of semi-supervised learning rather than supervised learning.
It is because we do not have an exact data set (unsupervised, since no actual labels), but we use the shifted value of the input as the data set (makeshift labels). Hence this makes RNN a semi-supervised learning algorithm (at least for time series).

Am I correct in this understanding?

Best Answer

Semi-supervised refers to using a combination of a labeled dataset (usually quite small) and a (usually much larger) unlabeled dataset.

When the labels are automatically derived from the data itself, this is these days called self-supervised learning or self-supervision. Examples in time series include predicting the next time step, or filling in a gap in the sequence.

This website by Lillian Weng is a good reference for some of the many possible strategies. https://lilianweng.github.io/lil-log/2019/11/10/self-supervised-learning.html

Related Solutions

Solved – Supervised learning : Hebb learning rule doubts

Hebbian learning is unsupervised learning.

There have been multiple interpretation of Hebbian Theory in terms of which equation is given.

There are a lot of supervised learning algorithms! But for artificial neural networks, we mostly do error-backpropagation.

In evolutionary computing the fittest survive - according to some criterion of fitness, which can be defined in either a supervised or an unsupervised manner.

Solved – Understanding the difference between Supervised and unsupervised learning

Lets look at a simple example of trying to predict housing prices. Assume we have a dataset that looks like

Cost |  Sq Ft  | N bedroom
 100K    1,800     4
 120K    1,300     3
 220K    2,200     5

In the case of supervised learning we would know the cost (these are our y labels) and we would use our set of features (Sq ft and N bedrooms) to build a model to predict the housing cost. The formula would look like

Cost ~ Sq Ft + N bedrooms

Now in unsupervised learning we would not know the cost of the house but we still would know the features. Therefore, we would train a model and try to group the types of houses together that are similar. For an example of this look at k-means clustering (http://scikit-learn.org/stable/modules/clustering.html#clustering)

This is a great, free, book which covers this very nicely (http://web.stanford.edu/~hastie/local.ftp/Springer/OLD/ESLII_print4.pdf)

Each type of learning method (may) have a set of parameters which are called model parameters. The training phase is used to find out the optimal set of parameters which generalizes you data the best. That book also gives very nice information on different learning methods are there parameters.

For example, in the leaning algorithm called SVM there is a term that looks like $\exp(-\gamma|x-x^{2}|)$. In this example the $\gamma$ parameter is what we try to optimize using the training data.

Best Answer

Related Solutions

Solved – Supervised learning : Hebb learning rule doubts

Solved – Understanding the difference between Supervised and unsupervised learning

Related Question