[Math] How to find gradient and hessian of summation

Question

I am confused because of the summation and its implicit form. Could you please help me to find gradient and Hessian of the function given below?

Thanks a lot.

$$
f(x) = \Vert x\Vert
= \left(\sum_{i=1}^nx_i^2\right)^\frac 12
$$

Answer 1

Once you have convinced yourself that $$\frac{\partial f(x)}{\partial x_i}=\frac{\partial \|x\|}{\partial x_i}=\frac{x_i}{\|x\|}$$ Then, recall that, the Hessian matrix of $f$ is the $n\times n$ matrix $\textbf H$ with the $(i,j)$-entry given by $$\textbf{H}_{ij}=\frac{\partial^2 f(x)}{\partial x_j\partial x_i}=\frac{\partial}{\partial x_j}\bigg(\frac{\partial f(x)}{\partial x_i}\bigg)$$ thus, we will have to calculate the latter in order to give the general input of the matrix.

Using the quotient rule, we see that $$\begin{align} \frac{\partial}{\partial x_j}\bigg(\frac{\partial f(x)}{\partial x_i}\bigg) &= \frac{\partial}{\partial x_j}\bigg(\frac{x_i}{\|x\|}\bigg)=\cfrac{\cfrac{\partial x_i}{\partial x_j}\cdot\|x\|-x_i\cdot\cfrac{\partial \|x\|}{\partial x_j}}{\|x\|^2} \\ &= \frac{1}{\|x\|}\frac{\partial x_i}{\partial x_j}-\frac{x_ix_j}{\|x\|^3} \end{align}$$ Note that, $$\frac{\partial x_i}{\partial x_j} = \delta_{ij} =\begin{cases} 1 & \textrm{if } i=j \\ 0 & \textrm{if } i\neq j \end{cases}$$ Hence, we have $$\mathbf{H}_{ij}=\frac{1}{\|x\|}\delta_{ij}-\frac{x_ix_j}{\|x\|^3} = \frac{\delta_{ij}\|x\|^2-x_ix_j}{\|x\|^3}$$ as, for example, if $n=2$, then $f$ is given by $f(x,y)=\|(x,y)\|=\sqrt{x^2+y^2}$ and then $$\mathbf{H}=\begin{pmatrix} \cfrac{\delta_{11}\|(x,y)\|^2-x^2}{\|(x,y)\|^3} & \cfrac{\delta_{12}\|(x,y)\|^2-xy}{\|(x,y)\|^3} \\ \cfrac{\delta_{21}\|(x,y)\|^2-yx}{\|(x,y)\|^3} & \cfrac{\delta_{22}\|(x,y)\|^2-y^2}{\|(x,y)\|^3} \end{pmatrix}=\begin{pmatrix} \cfrac{y^2}{(x^2+y^2)^{3/2}} & -\cfrac{xy}{(x^2+y^2)^{3/2}} \\ -\cfrac{xy}{(x^2+y^2)^{3/2}} & \cfrac{x^2}{(x^2+y^2)^{3/2}} \end{pmatrix} = \frac{1}{(x^2+y^2)^{3/2}}\begin{pmatrix} y^2&-xy \\ -xy&x^2 \end{pmatrix}$$