I am trying understand the following identity in two dimensions:
$$ H_{XY} = J^TH_{xy}J$$
Here $x,y$ and $X,Y$ are different coordinates for $\mathbb R^2$, the $J$ is the Jacobian matrix and the $H$ are the Hessian matrices in the coordinates.
My thoughts are the following: the $J$ is locally a coordinate transformation from $XY$ to $xy$ coordinates. At least is how I thought of Jacobians until today. The problem is that then $J^T$ should be $J^{-1}$. I checked and Jacobians are not necessarily unitary. Now I wonder: how do I understand the equation above and in general the Jacobian matrix? In particular why is it $J^T$ and not $J^{-1}$?
Best Answer
Try to compute it directly. Assume $(x,y)\rightarrow(X,Y)$ is linear transformation, we have $$(H(X))_{ij}=\frac{\partial^2f}{\partial X_i\partial X_j}=\sum_k\sum_l\frac{\partial x_k}{\partial X_i}\frac{\partial^2f}{\partial x_k\partial x_l}\frac{\partial x_l}{\partial X_j}=\sum_k\sum_l J_{ki}(H(x))_{kl}J_{lj}$$ Write in matrix form precisely $$H(X)=J^TH(x)J$$
Now you may still be unclear why there's a $J^T$ instead of $J^{-1}$. The reason is that Hessian is not a linear transformation as what the other matrices be.
Let's look closer to change of basis of a matrix A by a linear transformation $P$.
If matrix $A$ is a linear transformation, that is it maps a vector to a vector. $$y=Ax$$ Then change basis means an another linear transformation $A'$ so that $$Py=A'Px$$ This is precisely $A'=P^{-1}AP$. We say linear transformation is both covariant (so $P$) and contravariant (so $P^{-1}$).
However, if matrix $A$ is a bilinear form, that is it maps two vectors to a scalar. $$c=A(x,y)$$ Then the change of basis becomes $$c=A'(Px,Py)$$ So by computation I wrote above, $A'=J^TAJ$. We then say this mapping is covariant(so both $P$)
You may say I can also regard linear transformation as a mapping from two vectors to scalar defined as $$c=x^TAy$$ Yes, that's the problem. Actually, here $x$ is no longer a vector but covector. You will further check $$c=(Px)^TA'(Py)=x^T(P^TA'P)y$$ Also transpose instead of inverse right? We can conclude the form of transformation depends on type of vector and matrix. In detail, contravariant, covariant or mix-type.