[Math] Why are numerical solutions preferred to analytical solutions

calculusnumerical methodsoptimization

I'm just learning about optimization, and the difference between an analytical solution, and a numerical one. Suppose there is a cost function f(x), and we want to find the value of x which minimizes this. In an analytical solution, we would differentiate with respect to x, i.e. find df(x)/dx. Whereas in a numerical solution, we would try values of x, and see in which direction we need to change x in order to move towards the solution (gradient descent).

It seems to me that the analytical solution is always preferred because it gives you an exact answer. You do not get stuck in local minima and you can be sure that your solution is correct. So why do we not always use analytical solutions?

I understand that numerical solutions make things easier because you don't have to work out by hand the analytical solution, but it seems to me that solving something by hand to get a precise answer is highly preferred to doing it computationally for an approximate answer.
Can somebody clear this up for me? Thanks!

Best Answer

  • Some equations have no finitely expressible analytic solution ($x^5+x+1=0$, for example).
  • Symbolic algebraic manipulation is computationally expensive, even when it can produce a usable solution.
  • For some functions, even taking the derivative analytically is too difficult.
  • You don't always need an exact solution: sometimes you just want bounds on the answer.