I am trying to find the gradient vector and Hessian of the following summation:
$$
f(x) = \sum_{i=1}^n [(x_i +b_i)^4 + e^{2x_i}]
$$
Trying to find the gradient vector:
$\nabla f = (\frac{\partial f}{\partial x_1}, \frac{\partial f}{\partial x_2}, \ldots, \frac{\partial f}{\partial x_n})$, so we need to work out $\frac{\partial f}{\partial x_i}$
$\frac{\partial f}{\partial x_i} = \sum_{i=1}^n [4(x_i +b_i)^3 + 2e^{2x_i}]$
= $ 4\sum_{i=1}^n (x_i +b_i)^3 + 2\sum_{i=1}^ne^{2x_i}$
From here, I don't really know how to continue finding the gradient vector.
Thanks a lot.
Best Answer
It might be clearer to calculate the gradient vector if you wrote out the sum explicitly: $$ f(x) = \sum_{i=1}^{n}[(x_i+b_i)^4+e^{2x_i}] $$ $$= (x_1+b_1)^4+e^{2x_1}+(x_2+b_2)^4+e^{2x_2}+\ldots+(x_j+b_j)^4+e^{2x_j}+\ldots+(x_n+b_n)^4+e^{2x_n} $$ for some $1\leq j\leq n$. Computing $\frac{\partial f}{\partial x_j}$, we arrive at: $$\frac{\partial f}{\partial x_j} = 4(x_j+b_j)^3+2e^{2x_j}$$ Since the gradient is the vector of partial derivatives, we have: $$\nabla f = \left(4(x_1+b_1)^3+2e^{2x_1},\ldots, 4(x_j+b_j)^3+2e^{2x_j},\ldots, 4(x_n+b_n)^3+2e^{2x_n} \right)$$ The Hessian will be an $n\times n$ matrix, where the $ij$th entry is found according to $H_{ij} = \frac{\partial^2 f}{\partial x_i \partial x_j} = \frac{\partial}{\partial x_i}\left(\frac{\partial f}{\partial x_j}\right)$. Notice that for $i\neq j $, we have $\frac{\partial}{\partial x_i}\left(\frac{\partial f}{\partial x_j}\right) = 0$, since in the $j$th component of the gradient, there is no dependency on $x_{i}$. For $i=j $, we have that $$ \frac{\partial^2 f}{\partial x_j^2} = \frac{\partial}{\partial x_j}\left(4(x_j+b_j)^3+2e^{2x_j}\right) = 12(x_j+b_j)^2+4e^{2x_j}$$ Then you end up with: $$\begin{pmatrix} 12(x_1+b_1)^2+4e^{2x_1} & 0& 0 & \cdots & 0\\ 0 & 12(x_2+b_2)^2+4e^{2x_2} & 0 & \cdots & 0 \\ \vdots & \vdots& \vdots & \ddots & \vdots \\ 0 & 0 & 0 & \cdots & 12(x_n+b_n)^2+4e^{2x_n} \end{pmatrix}$$