Solved – Regression vs. classification and generative vs. discriminative

classificationdeep learninggenerative-modelsmachine learningregression

I recently touch the idea of Generative adversarial networks, which is a competition between a generative network and a discriminative network.

This idea makes me think of replacing the word "network" into a general machine learning model or algorithm. One thing comes to my mind is the difference between regression and classification, which seems to me that it is comparable to the difference between a generative model and a discriminative model.

I know there are many clarification questions asked on this site such as:

Why not approach classification through regression?

So this question is merely asking for confirmation or clarification of the idea:

Is it legitimate to say that regression is a data generative task and classification is a data discriminative task?

Best Answer

There are no discriminative or generative tasks, but discriminative and generative models, for both regression and classification. There is a very nice paper that discusses this difference: On Discriminative vs. Generative classifiers: A comprarison of logistic regression and naive Bayes.

Basically, discriminative models attempt to estimate the conditional probability $P(y|x)$, where $y$ is the output conditioned on the input $x$. Generative models on the other hand estimate $P(x, y) = P(y|x)P(x)$. This allows you to sample from the generative model (and hence the name), because you also model how samples $x$ are generated.

Related Question