Let $A$ be an $m\times n$-matrix and $B$ an $n \times m$-matrix over the same field. Consider the matrices $C=AB$ and $D=BA$. It is probably well known (and not difficult to show) that the only difference between the canonic rational forms of $C$ and $D$ are nilpotent blocks (blocks with minimal polynomial $x^k$). (In particular, these compensate the different dimensions of $C$ and $D$.)
I'm interested in the converse question:
Given an $m\times m$-matrix $C$ and an $n\times n$-matrix $D$, what are necessary and sufficient conditions that there exist matrices $A$ and $B$ such that $C=AB$ and $D=BA$?
One may assume without loss of generality that $C$ and $D$ are both nilpotent. I'm thinking of a characterization in terms of the Jordan normal forms of $C$ and $D$. These in turn are characterized by their block sizes. In fact, an equivalent version can be stated as a question on partitions:
Suppose $\lambda=(\lambda_1 \geq\lambda_2\geq \dotsc )$ and $\mu= (\mu_1 \geq \mu_2 \geq \dotsc)$ are partitions of the integers $m$ and $n$. When are there matrices $A$ and $B$ such that the blocks in the jordan normal forms of $AB$ and $BA$ belonging to the eigenvalue $0$ have sizes $\lambda_1, \dotsc$ and $\mu_1, \dotsc$, respectively?
These are not arbitrary, for example, the quotient of the minimal polynomials must be in $\{1, x^{\pm 1}\}$, meaning that $|\lambda_1-\mu_1| \leq 1$.
This problem seems so natural that I think it has been addressed somewhere (not in the linear algebra books I looked into, however), so in particular I would appreciate a reference.
EDIT: I have now seen that $|\lambda_i -\mu_i|\leq 1$ for all $i$ is sufficient (and this is easy, since we may assume $C$ and $D$ in Jordan form, and then reduce to the case of one Jordan block). I guess it's necessary, too. Does anyone know a reference for this? And are there any nontrivial mathematical applications of this situation?
Best Answer
"And are there any nontrivial mathematical applications of this situation?"
Yes, this is a very important construction in algebraic geometry and representation theory!
Algebraic geometry. The papers of Kraft and Procesi used this construction to analyze singularities of the closures of nilpotent orbits in the classical Lie algebras $\mathfrak{gl}_n, \mathfrak{sp}_{2n}, \mathfrak{o}_n.$ In particular, they proved that, in the case of $\mathfrak{gl}_n,$ these closures are normal varieties. Their proof is based on the relation $$ \bar{\mathcal{O}}_{\lambda}=r\circ\ell^{-1}(\bar{\mathcal{O}}'_{\mu}). \qquad (*) $$ Here $\bar{\mathcal{O}}_{\lambda}$ is the closure of the conjugacy class of nipotent $n\times n$ matrices with partition $\lambda$ and $\bar{\mathcal{O}}'_{\mu}$ is the closure of the the conjugacy class of nipotent $m\times m$ matrices with partition $\mu,$ where $\mu_i=\operatorname{max}(\lambda_i-1,0)$; the diagram of $\mu$ is obtained from the diagram of $\lambda$ by removing the first column, so that $n-m=\lambda'_1.$ The maps $r$ and $\ell$ are $$ r((A,B))=AB, \quad l((A,B))=BA $$ in the notation of the question. The papers of Daskiewicz, Kraskiewicz and Przebinda considered a more general situation: starting with $m$ and $n$ and a partition $\mu$ of $m,$ they proved that the formula (*) holds for a certain partition $\lambda$ of $n$. In other words, the algebraic variety $r\circ\ell^{-1}(\bar{\mathcal{O}}'_{\mu})$ is the closure of a single nilpotent orbit, without imposing additional assumptions on $m$ and $n;$ the proof involves careful combinatorial analysis, especially when $m>n.$
Representation theory. Without going into too much detail, this construction emerges in Roger Howe's theory of reductive dual pairs. The nilpotent orbits in question arise as the wave front sets or the associated varieties of representations of two classical groups occurring in the Howe duality correspondence. This is considered and exploited in various papers of J.-S. Li, T. Przebinda, P. Trapa, Nishiyama, Oshiai, and Taniguchi, and my own.