Suppose that we have a set of points $(x_1,y_1), \ldots (x_n,y_n)$, and we want to fit a function of the form $f(x) = ae^{2x} + be^x + c$ to those points. If we make $z=e^x$, then our function becomes a second-degree polynomial in $z$: $f(z) = az^2+bz+c$. My question is: Which transformations (if any) do I have to make on the points $(x_i,y_i)$ in order to be able to fit the second-degree polynomial $f(z)$ to the set of transformed points, by the usual method of least squares?
[Math] fitting by linear combination of exponential functions
exponential functionleast squaresnumerical methods
Related Solutions
Assume we have the linear model $$y=X\beta$$ where \begin{align*} y_{n\times1} =\begin{bmatrix} y_1\\ y_2\\ \vdots\\ y_n \end{bmatrix} \hspace{2cm} X_{n\times2} = \begin{bmatrix} 1 & x_1\\ 1& x_2\\ \vdots & \vdots\\ 1 & x_n \end{bmatrix} \hspace{2cm} \beta_{2\times1} = \begin{bmatrix} b_0\\ b_1 \end{bmatrix} \end{align*} and so using linear algebra we have (all of my sums are with respect to $i$ and go to $n$, i.e., $\sum_{i=1}^n$) \begin{align*} \beta &= (X'X)^{-1}X'y\\ &=\left( \begin{bmatrix} 1 & 1&\cdots & 1\\ x_1 & x_2 & \cdots & x_n \end{bmatrix} \begin{bmatrix} 1 & x_1\\ 1& x_2\\ \vdots & \vdots\\ 1 & x_n \end{bmatrix}\right)^{-1} \begin{bmatrix} 1 & 1&\cdots & 1\\ x_1 & x_2 & \cdots & x_n \end{bmatrix} \begin{bmatrix} y_1\\ y_2\\ \vdots\\ y_n \end{bmatrix}\\ &=\left(\begin{bmatrix} n&\sum x_i\\ \sum x_i & \sum x_i^2 \end{bmatrix}\right)^{-1}\begin{bmatrix} \sum y_i\\ \sum x_iy_i \end{bmatrix}\\ & \hspace{-1.45in}\text{taking the inverse is not hard since it is a }2\times2 \text{ matrix}\\ &=\frac{1}{n\sum x_i^2-\left(\sum x_i\right)^2}\begin{bmatrix} \sum x_i^2 & -\sum x_i\\ -\sum x_i & n \end{bmatrix}\begin{bmatrix} \sum y_i\\ \sum x_iy_i \end{bmatrix}\\ &=\begin{bmatrix} \frac{\sum x_i^2\sum y_i-\sum x_i\sum x_iy_i}{n\sum x_i^2-\left(\sum x_i\right)^2}\\ \frac{n\sum x_iy_i-\sum x_i\sum y_i}{n\sum x_i^2-\left(\sum x_i\right)^2} \end{bmatrix} \end{align*} and so, \begin{align*} b_0 = \frac{\sum x_i^2\sum y_i-\sum x_i\sum x_iy_i}{n\sum x_i^2-\left(\sum x_i\right)^2} \end{align*} and \begin{align*} b_1 = \frac{n\sum x_iy_i-\sum x_i\sum y_i}{n\sum x_i^2-\left(\sum x_i\right)^2} \end{align*}
Now we have $b_0$ and $b_1$ and so for any value of $x$ we can figure out the corresponding $y$ value. The cool thing about the least squares line is that it WILL ALWAYS pass through the point that corresponds to the mean of $x$ and the mean of $y$. Why is that true? Plug in $\bar x$ to $y=b_0+b_1x$ and after some algebra it's easy to see.
Let's plug in $\bar x$ for $x$ in $y=b_0+b_1x$, so
\begin{align*} b_0+b_1\bar x &=\frac{\sum x_i^2\sum y_i-\sum x_i\sum x_iy_i}{n\sum x_i^2-\left(\sum x_i\right)^2}+\frac{n\sum x_iy_i-\sum x_i\sum y_i}{n\sum x_i^2-\left(\sum x_i\right)^2}\times\frac{1}{n}\sum x\\ &=\frac{\sum x_i^2\sum y_i-\sum x_i\sum x_iy_i}{n\sum x_i^2-\left(\sum x_i\right)^2}+\frac{n\sum x_iy_i\sum x_i-\sum x_i\sum y_i\sum x_i}{n^2\sum x_i^2-\left(\sum x_i\right)^2}\\ &=\frac{\sum x_i^2\sum y_i-\sum x_i\sum x_iy_i}{n\sum x_i^2-\left(\sum x_i\right)^2}+\frac{\sum x_iy_i\sum x_i}{n\sum x_i^2-\left(\sum x_i\right)^2}-\frac{\left(\sum x_i\right)^2\sum y_i}{n^2\sum x_i^2-\left(\sum x_i\right)^2}\\ &=\frac{\sum x_i^2\sum y_i-\sum x_i\sum x_iy_i+\sum x_iy_i\sum x_i}{n\sum x_i^2-\left(\sum x_i\right)^2}-\frac{\left(\sum x_i\right)^2\sum y_i}{n^2\sum x_i^2-\left(\sum x_i\right)^2}\\ &=\frac{\sum x_i^2\sum y_i}{n\sum x_i^2-\left(\sum x_i\right)^2}-\frac{\left(\sum x_i\right)^2\sum y_i}{n^2\sum x_i^2-\left(\sum x_i\right)^2}\\ &=\frac{\sum x_i^2\sum y_i-\left(\sum x_i\right)^2\sum y_i}{n\sum x_i^2-\left(\sum x_i\right)^2}\\ &=\frac{\sum y_i\left(\sum x_i^2-\left(\sum x_i\right)^2\right)}{n\sum x_i^2-\left(\sum x_i\right)^2}\\ &=\frac{1}{n}\sum y_i\\ &\bar y \end{align*}
The standard approach to solving this question is indeed to linearize by taking the logarithm. The same trick is used for the power law, $y=a\cdot x^b$, among others. This is probably the approach expected by your professor.
Anyway this is a twist as the nonlinear transform does not provide the true minimum solution of the original problem, as the errors themselves are rescaled non-uniformly.
Your formulation is indeed the correct one, but you end-up with a nasty system of equations that has no analytical solution.
So you must resort to a numerical approach such as that of Levenberg and Marquardt which solves these nonlinear equations iteratively. This method requires a good initial approximation, for which one will naturally chose the solution obtained from linearization.
Best Answer
Least squares folution to overdetermined system $Ax = b$ is given by $$ x = \left(A^\mathsf{T} A\right)^{-1} A^\mathsf{T} b $$ There's no need to do a transform $z = e^x$, it is solvable as it is. Rewrite the overdetermined system $$ e^{2x_1} a + e^{x_1} b + c = f_1\\ e^{2x_2} a + e^{x_2} b + c = f_2\\ \vdots\\ e^{2x_n} a + e^{x_n} b + c = f_n\\ $$ in matrix form $$ A = \begin{pmatrix} e^{2x_1} & e^{x_1} & 1\\ e^{2x_2} & e^{x_2} & 1\\ &\vdots\\ e^{2x_n} & e^{x_n} & 1\\ \end{pmatrix}, \qquad f = \begin{pmatrix} f_1\\f_2\\\vdots\\f_n \end{pmatrix}. $$ Now, $$ A^\mathsf{T} A = \begin{pmatrix} \sum_i e^{4x_i}& \sum_i e^{3x_i} & \sum_i e^{2x_i}\\ \sum_i e^{3x_i}& \sum_i e^{2x_i} & \sum_i e^{x_i}\\ \sum_i e^{2x_i}& \sum_i e^{x_i} &1 \end{pmatrix}\\ A^\mathsf{T} b = \begin{pmatrix} \sum_i e^{2x_i} f_i\\ \sum_i e^{x_i} f_i\\ \sum_i f_i \end{pmatrix} $$