"Curvature of a function" is not a standard mathematical term as far as I know, though it may be something found in a calculus textbook. If you mean the curvature of the graph of a function $y=f(x)$, then it involves the first derivative as well as the second. One can make the first derivative go away by changing the system of coordinates so that one of the axis is tangent to the curve at the point of interest. Then indeed, the second derivative $f''$ gives the curvature of the graph.
For the surface $z=f(x,y)$, assuming $\nabla f$ vanishes at the point of interest, we get the principal curvatures (plural) as the eigenvalues of the Hessian. The determinant of the Hessian gives Gaussian curvature, while the trace (that is, Laplacian $\Delta f$) gives mean curvature times two.
The relation is more complicated at the points where $\nabla f$ is nonzero, and the entire concept of curvature gets much more complex when we move from surfaces to higher-dimensional manifolds. Then the curvature cannot be adequately measured by a single number. An accessible reference is Curvatures of Hypersurfaces.
As for the last question: otherwise, you don't have a critical point and there is nothing to test. :-) Think with one variable: would you look for a maximum or minimum if $f'(x_0) \neq 0$?
Your intuitive understanding of the Hessian points in the right direction. The point is: how to "sum up" all the data $f_{xx}, f_{xy} = f_{yx}, f_{yy}$ in just one single fact?
Well, thinking about the quadratic form that the Hessian defines. Namely,
$$
q(x,y) =
\begin{pmatrix}
x & y
\end{pmatrix}
\begin{pmatrix}
f_{xx} & f_{xy} \\
f_{yx} & f_{yy}
\end{pmatrix}
\begin{pmatrix}
x \\
y
\end{pmatrix}
=
f_{xx}x^2 + 2 f_{xy}xy + f_{yy}y^2 \ .
$$
If this quadratic form is positive-definitive, that is $q(x,y) > 0$ for all $(x,y) \neq (0,0)$, then $f$ has a local minimum at the point where this happens (just as in the one-variable case, $f''(x_0) > 0$ implies $f$ has a local minimum at $x_0$).
It's more or less obvious that for $q(x,y)$ to be positive or not at all points doesn't depend on the coordinate system you're using, isn't it?
Right, then do the following experiment: you have a nice quadratic form like
$$
q(x,y) = x^2 + y^2
$$
which is not ashamed to show clearly that she is positive-definite, is she?
Then, do to her the following linear change of coordinates:
$$
\begin{pmatrix}
x \\ y
\end{pmatrix}
=
\begin{pmatrix}
1 & 1 \\
1 & 0
\end{pmatrix}
\begin{pmatrix}
\overline{x} \\
\overline{y}
\end{pmatrix}
$$
and you'll get
$$
q(\overline{x}, \overline{y}) = 2\overline{x}^2 + 2 \overline{x}\overline{y} + \overline{y}^2 \ .
$$
Is now also clear that $q(\overline{x}, \overline{y}) > 0$ for all $ (\overline{x}, \overline{y}) \neq (0,0)$?
So, we need some device that allows us to show when a symmetric matrix like $H$ will define a positive-definite quadratic form $q(x,y)$, no matter if the fact is disguised because we are using the wrong coordinate system.
One of these devices are the eigenvalues of $H$: if all of them are positive, we know that, maybe after a change of coordinate system, our $q(x,y)$ will have an associate matrix like
$$
\begin{pmatrix}
\lambda & 0 \\
0 & \mu
\end{pmatrix}
$$
with $\lambda, \mu > 0$. Hence, in some coordinate system (and hence, in all of them), our $q > 0$.
Best Answer
Are you familiar with the Taylor expansion of a single variable function?$$f(x)=f(x_0)+f'(x_0)\Delta x+\frac12f''(x_0)\Delta x^2+\dots$$We can also find Taylor expansions for multivariable functions; the first two terms should be familiar:$$f(\mathbf{x})=f(\mathbf{x}_0)+J(\mathbf{x}_0)\Delta\mathbf{x}+\frac12\Delta\mathbf{x}^TH(\mathbf{x}_0)\Delta\mathbf{x}+\dots$$... so we see that while the Jacobian corresponds to our original idea of the derivative, the Hessian instead serves a purpose like our second derivative -- it indeed captures local curvature information. In fact, if you look at its elements, this should become very clear.