Please do not use notation like this. If $f=f(u)$ and $u=g(x)$, please write $$\frac d{dx}f(g(x)) = \frac{df}{du}\Big|_{u=g(x)}\cdot \frac{dg}{dx}.$$ You then need to use the product rule and chain rule to take the derivative again. In particular,
$$\frac d{dx}\left(\frac{df}{du}\Big|_{u=g(x)}\right) = \frac{d^2f}{du^2}\Big|_{u=g(x)}\cdot\frac{dg}{dx}.$$ Now you finish up with the product rule.
This is what I did, but I'm not sure if this is right.
First I find the second-order partial derivatives, by using the chain rule:
$\frac{\partial u}{\partial \theta}=\frac{\partial u}{\partial x}\frac{\partial x}{\partial \theta}+\frac{\partial u}{\partial y}\frac{\partial y}{\partial \theta}\Rightarrow\frac{\partial u}{\partial x}\frac{\partial x}{\partial \theta}=\frac{\partial u}{\partial \theta}-\frac{\partial u}{\partial y}\frac{\partial y}{\partial \theta}\Rightarrow\frac{\partial u}{\partial x}=\frac{\partial u}{\partial \theta}\frac{\partial \theta}{\partial x}-\frac{\partial u}{\partial y}\frac{\partial y}{\partial \theta}\frac{\partial \theta}{\partial x}$
$\frac{\partial u}{\partial x}=-\frac{\partial u}{\partial \theta}\cdot\frac{y}{x^2+y^2}+\frac{\partial u}{\partial y}r\cos(\theta)\cdot\frac{y}{x^2+y^2}$
$\frac{\partial u}{\partial r}=\frac{\partial u}{\partial x}\frac{\partial x}{\partial r}+\frac{\partial u}{\partial y}\frac{\partial y}{\partial r}\Rightarrow\frac{\partial u}{\partial y}\frac{\partial y}{\partial r}=\frac{\partial u}{\partial r}-\frac{\partial u}{\partial x}\frac{\partial x}{\partial r}\Rightarrow\frac{\partial u}{\partial y}=\frac{\partial u}{\partial r}\frac{\partial r}{\partial y}-\frac{\partial u}{\partial x}\frac{\partial x}{\partial r}\frac{\partial r}{\partial y}$
$\frac{\partial u}{\partial y}=\frac{\partial u}{\partial r}\cdot\frac{y}{(x^2+y^2)^\frac{1}{2}}-\frac{\partial u}{\partial x}\cos(\theta)\cdot\frac{y}{(x^2+y^2)^\frac{1}{2}}$
$\frac{\partial^2 u}{\partial y \partial x}=\frac{\partial}{\partial y}(\frac{\partial u}{\partial x})=-\frac{\partial u}{\partial \theta}\frac{\partial}{\partial y}(\frac{y}{x^2+y^2})+\frac{\partial u}{\partial y}r \cos \theta\frac{\partial}{\partial y}(\frac{y}{x^2+y^2})$
$\frac{\partial^2 u}{\partial y \partial x}=-\frac{\partial u}{\partial \theta}\cdot\frac{x^2-y^2}{(x^2+y^2)^2}+\frac{\partial u}{\partial y}r\cos(\theta)\cdot\frac{x^2-y^2}{(x^2+y^2)^2}$
$\frac{\partial^2 u}{\partial y \partial x}=-\frac{\partial u}{\partial \theta}\cdot\frac{r^2cos(2\theta)}{r^4}+\frac{\partial u}{\partial y}r\cos(\theta)\cdot\frac{r^2cos(2\theta)}{r^4}$
$\frac{\partial^2 u}{\partial y^2}=\frac{\partial u}{\partial r}\frac{\partial}{\partial y}(y\cdot(x^2+y^2)^{-\frac{1}{2}})-\frac{\partial u}{\partial x}\cos(\theta)\frac{\partial}{\partial y}(y\cdot(x^2+y^2)^{-\frac{1}{2}})$
$\frac{\partial^2 u}{\partial y^2}=\frac{\partial u}{\partial r}(\frac{1}{\sqrt{x^2+y^2}}-\frac{3}{2}2y^2(x^2+y^2)^{-\frac{3}{2}})-\frac{\partial u}{\partial x}\cos(\theta)(\frac{1}{\sqrt{x^2+y^2}}-\frac{3}{2}2y^2(x^2+y^2)^{-\frac{3}{2}})$
$\frac{\partial^2 u}{\partial y^2}=\frac{\partial u}{\partial r}(\frac{1}{r}-\frac{3}{2}2r^2\sin^2(\theta)r^\frac{1}{2})-\frac{\partial u}{\partial x}\cos(\theta)(\frac{1}{r}-\frac{3}{2}2r^2\sin^2(\theta)r^\frac{1}{2})$
Now using the fact that $x=r \cos(\theta)$ and $y=r \sin(\theta)$, I find $\frac{\partial u}{\partial x}$ and $\frac{\partial u}{\partial y}$ in terms of $r$ and $\theta$:
$\frac{\partial u}{\partial r}=\frac{\partial u}{\partial x}\frac{\partial x}{\partial r}+\frac{\partial u}{\partial y}\frac{\partial y}{\partial r}\Rightarrow\frac{\partial u}{\partial x}\cos (\theta)+\frac{\partial u}{\partial y}\sin(\theta)$
$\frac{\partial u}{\partial \theta}=\frac{\partial u}{\partial x}\frac{\partial x}{\partial \theta}+\frac{\partial u}{\partial y}\frac{\partial y}{\partial \theta}\Rightarrow\frac{\partial u}{\partial x}\left(-r\right)\sin(\theta)+\frac{\partial u}{\partial y}r \cos(\theta)$
Writing in terms of a matrix:
$\begin{bmatrix}
\frac{\partial u}{\partial r}\\
\frac{\partial u}{\partial \theta}
\end{bmatrix}=\begin{bmatrix}
\cos \theta & \sin \theta\\
-r \sin \theta & r \cos \theta
\end{bmatrix}\begin{bmatrix}
\frac{\partial u}{\partial x}\\
\frac{\partial u}{\partial y}
\end{bmatrix}$
$\begin{bmatrix}
\frac{\partial u}{\partial x}\\
\frac{\partial u}{\partial y}
\end{bmatrix}=\begin{bmatrix}
\cos \theta & \sin \theta\\
-r \sin \theta & r \cos \theta
\end{bmatrix}^{-1}\begin{bmatrix}
\frac{\partial u}{\partial r}\\
\frac{\partial u}{\partial \theta}
\end{bmatrix}$
$\begin{bmatrix}
\frac{\partial u}{\partial x}\\
\frac{\partial u}{\partial y}
\end{bmatrix}=\begin{bmatrix}
\cos \theta & -\frac{\sin \theta}{r}\\
\sin \theta & \frac{\cos \theta}{r}
\end{bmatrix}\begin{bmatrix}
\frac{\partial u}{\partial r}\\
\frac{\partial u}{\partial \theta}
\end{bmatrix}$
From here it's just a simple case of plugging in terms into the expression, which I am too lazy to do. Can anyone confirm that this is indeed correct?
Best Answer
The map $g : y \mapsto f(x(y),y)$ is only having one independent variable. Namely $y$. Hence $$\frac{\partial^2f(x(y), y)}{\partial x \ \partial y}$$ doesn't have any meaning.