[Math] Is the Hessian Equal to the Outer Product of the Score with Itself

Question

Recall for a function $f: \mathbb{R}^n \to \mathbb{R}$, the Gradient is
\begin{equation}
\nabla f(\mathbf{x}) =
\begin{bmatrix} \frac{\partial f(\mathbf{x})}{\partial x_1} & \frac{\partial f(\mathbf{x})}{\partial x_2} & \cdots & \frac{\partial f(\mathbf{x})}{\partial x_n}
\end{bmatrix}
\end{equation}
and the Hessian is
\begin{equation}
Hf(\mathbf{x}) =
\begin{bmatrix}
\frac{\partial^2 f(\mathbf{x})}{(\partial x_1)^2} & \frac{\partial^2 f(\mathbf{x})}{\partial x_1 \partial x_2} & \cdots & \frac{\partial^2 f(\mathbf{x})}{\partial x_1 \partial x_n} \\
\frac{\partial^2 f(\mathbf{x})}{\partial x_2\partial x_1} & \frac{\partial^2 f(\mathbf{x})}{(\partial x_2)^2} & \cdots & \frac{\partial^2 f(\mathbf{x})}{\partial x_2 \partial x_n} \\
\vdots & \vdots & \ddots & \vdots \\
\frac{\partial^2 f(\mathbf{x})}{\partial x_n\partial x_1} & \frac{\partial^2 f(\mathbf{x})}{\partial x_n\partial x_2} & \cdots & \frac{\partial^2 f(\mathbf{x})}{(\partial x_n)^2}
\end{bmatrix}
\end{equation}
To me, this looks like an outer product matrix. So, this brings me to my question: given the gradient of $f$, if I wanted to compute the Hessian, is it true that
\begin{align}
Hf(\mathbf{x})&=
\begin{bmatrix} \frac{\partial f(\mathbf{x})}{\partial x_1} \\ \frac{\partial f(\mathbf{x})}{\partial x_2} \\ \vdots \\ \frac{\partial f(\mathbf{x})}{\partial x_n}
\end{bmatrix}
\begin{bmatrix} \frac{\partial f(\mathbf{x})}{\partial x_1} & \frac{\partial f(\mathbf{x})}{\partial x_2} & \cdots & \frac{\partial f(\mathbf{x})}{\partial x_n}
\end{bmatrix} \\
&=\nabla f(\mathbf{x})^{\top}\nabla f(\mathbf{x}) \\
&=sf(\mathbf{x})sf(\mathbf{x})^{\top}
\end{align}
where $sf(\mathbf{x})=\nabla f(\mathbf{x})^{\top}$ denotes the score?
Thank you!

Answer 1

I think you're just confused by notation.

$\frac{\partial^2 f}{\partial x_1 \partial x_2}(\mathbf{x}) \neq \frac{\partial f}{\partial x_1}(\mathbf{x}) \cdot \frac{\partial f}{\partial x_2}(\mathbf{x})$