Solved – Gradient Ascent vs Gradient Descent in Logistic Regression

gradient descentlogisticmachine learning

I've been going trough Machine Learning in Action book from manning (https://www.manning.com/books/machine-learning-in-action)
In the logistic regression chapter it uses gradient ascent to calculate the best weights.
Why do we pick gradient ascent instead of gradient descent ?

Best Answer

https://en.wikipedia.org/wiki/Gradient_descent:

To find a local minimum of a function using gradient descent, one takes steps proportional to the negative of the gradient (or of the approximate gradient) of the function at the current point.

If instead one takes steps proportional to the positive of the gradient, one approaches a local maximum of that function; the procedure is then known as gradient ascent.

In other words:

  • gradient descent aims at minimizing some objective function: $\theta_j \leftarrow \theta_j-\alpha \frac{\partial}{\partial \theta_{j}} J(\theta)$
  • gradient ascent aims at maximizing some objective function: $\theta_j \leftarrow \theta_j+\alpha \frac{\partial}{\partial \theta_{j}} J(\theta)$