I'm a little confused on how to approach the following problem.
Use the least square method to fit a curve of the form $y=a\cdot b^x$ to a collection of $n$ data points $(x_1,y_1),…,(x_n,y_n)$ Find $a$ and $b$.
From what I understand. Given data {$(x_1,y_1),…,(x_n,y_n)$}, we may define the error associated to $y=a\cdot b^x$ by
\begin{equation}
E(c,d)=\sum_{i=1}^{n}(y_i-a\cdot b^{x_i})^2
\end{equation}
The goal is to find values of $a$ and $b$ that minimize the error. In multivariable calculus we learn that this requires us to find the values of $(a,b)$ such that
\begin{array}
d \frac{\partial E}{\partial a}=0 & & \frac{\partial E}{\partial b}=0
\end{array}
Differentiating $E(a,b)$ yields:
\begin{eqnarray}
\frac{\partial E}{\partial a}&=&\sum_{i=1}^n2(y_i-a\cdot b^{x_i})(-b^{x_i})=0\\
\frac{\partial E}{\partial b}&=&\sum_{i=1}^n2(y_i-a\cdot b^{x_i})(-a\cdot b^{x_i-1}x_i)=0\\
\end{eqnarray}
I know that the least square method tries to minimize the error, but I can't help but feel that I doing this somewhat wrong since this isn't a polynomial – but an exponential – function.
The alternative was for me to take the natural log of $y=a\cdot b^x$ to get:
\begin{equation}
\ln{y}=\ln{a}+x\ln{b}
\end{equation}
Then defining $\ln{y}=Y$,$\ln{a}=A$ and $\ln{b}=B$ to get:
\begin{equation}
Y=A+xB
\end{equation}
Then from here apply the least square method to my new linear equation.
Any suggestions or feedback on what I can do to solve this problem?
Thank you for your time.
Best Answer
The standard approach to solving this question is indeed to linearize by taking the logarithm. The same trick is used for the power law, $y=a\cdot x^b$, among others. This is probably the approach expected by your professor.
Anyway this is a twist as the nonlinear transform does not provide the true minimum solution of the original problem, as the errors themselves are rescaled non-uniformly.
Your formulation is indeed the correct one, but you end-up with a nasty system of equations that has no analytical solution.
So you must resort to a numerical approach such as that of Levenberg and Marquardt which solves these nonlinear equations iteratively. This method requires a good initial approximation, for which one will naturally chose the solution obtained from linearization.