I have an AI course in the which we studied and worked on logistic regression. After having read and studied a bit about neural networks I'm starting to ask myself why on earth would we use logistic regression instead of neural networks.
Is it just an academic standard way of introducing the topic of loss-function minimization based ML techniques ?
Best Answer
Logistic regression is basically a neural network with one layer.
So you can equivalently ask, why do people use networks with 12 layers and not 24 layers ? or why do people use 5-th order polynomials for curve fitting and not 10-th order ?
The question of how to choose to right model for your data is very broad. But in general, a model with more parameters is not necessarily better, and in many cases will be worse (think over-fitting). This is why for example convolutional networks work better for image data then fully-connected networks. It all depends on what type of data you have.