[Math] An optimization problem in numerical linear algebra

linear algebrana.numerical-analysisoc.optimization-and-control

Provided two diagonal real matrix which has positive entries, $V$ and $U$.

Find a real matrix $A$, satisfying $A^TA=a^2I$ for some scalar $a$, to minimise

$\left|A^TVA-U\right|\quad\quad(*)$

where the matrix norm could be an induced one, or in form of $|M|^2_{F}=\mathrm{tr}(M^TM)$.

I believe the problem is quite useful, however I am not sure where I can find the related materials. A numerical approach is also welcome.

I found some related works , I think I can program the general framework for non-linear optimisation problem with unitary constraints. But since $(*)$ is only a quadratic form. I wonder if there are some improvements.

remark: there are two trivial cases, namely $V=U$ or $U=I$.

Best Answer

The set of matrices $A$ is a cone, smooth away from the origin. With the Frobenius norm $\sqrt{{\rm Tr}(M^TM)}$, you can use differential calculus. The minimum is achieved at some $A$. If $A\ne0_n$, that is $a\ne0$, the admissible variations are $\delta A=\rho A+AB$ with $\rho\in\mathbb R$ and $B$ skew-symmetric. When writing $$\delta|A^TVA-U|^2=0,$$ you obtain on the one hand that $V^TAV$ commuttes with $U$, and on the other hand that the trace of $(A^TVA-U)A^TVA$ vanishes. Let me assume for the sake of simplicity that $U$ has pairwise distinct diagonal entries $u_i$ (=eigenvalues). Then $A^TVA$ must be diagonal; its diagonal entries $w_i$ are equal to $a^2v_{\sigma(i)}$ for some permutation $\sigma$ (consider the eigenvalues). The second requirement gives us the value of $a$: $$a^2=\frac{\vec u\cdot\vec v_\sigma}{|\vec v|^2}.$$

The value of $|A^TVA-U|^2$ is then $|\vec u|^2-(u\cdot\vec v_\sigma)^2|\vec v|^{-2}$. To minimize it, we must choose the permutation $\sigma$ that puts $v_\sigma$ in the same ordering as $u$. Say that if $u_1\ge\cdots\ge u_n$ then we reorder $\vec v$ in descreasing form as well.

Related Question