I want to prove $\text{rank}(I+AB) = \text{rank}(I+BA)$ without using property of for invertible matrix $A$, $\text{rank}(AB) = \text{rank}(BA)= \text{rank}(B)$. [If $A$ is invertible, $1+AB = A(A^{-1} +B)$ and since $A$ is invertible, $\text{rank}(A(A^{-1}+B)) = \text{rank}((A^{-1}+B)A) = \text{rank}(1+BA)$.
First of all, I am not sure this holds without the condition of invertibility of $A, B$.
My trial is as follows :
First I want to prove $N(I+AB) = N(I+BA)$. [Here $N$ denote the null space, and the rank is the dimension of null space]
Let $v \in N(I+AB)$, then $(I+AB) v = 0$ implies $v = – ABv$ so $Bv= – BABv$ which means $Bv \in N(I+BA)$.
Let $n = \text{rank}(I+AB)$ that means $v \in \{v_1, \cdots, v_n\}$ and I stuck at this moment…
Are there any straightforward or easy ways to prove this?
Best Answer
It's not difficult to verify that if $\lambda \neq 0$ is an eigenvalue of $AB$ corresponding to $v \neq 0$, then $Bv$ is an eigenvector for $BA$ corresponding to $\lambda$. If we have $ABv = \lambda v$, then since $\lambda \neq 0$, we have $\lambda v \neq 0$, and hence $Bv \neq 0$. Then $$BA(Bv) = B(ABv) = B(\lambda v) = \lambda Bv.$$ We can actually prove something stronger: if $E_1$ and $E_2$ are the eigenspaces of $AB$ and $BA$ respectively, with respect to eigenvalue $\lambda \neq 0$, then $$f : E_1 \to E_2 : v \mapsto Bv$$ is an isomorphism between $E_1$ and $E_2$, with inverse $$g : E_2 \to E_1 : v \mapsto \lambda^{-1}Av.$$ We have shown $f$ has a valid domain and codomain. By swapping the roles of $A$ and $B$, the same can be said about $\lambda g$, and hence $g$. It is clear they are linear, so we just need to show they are inverses.
We have, for any $v \in E_1$ (i.e. eigenvector of $AB$), $$(g \circ f)(v) = \lambda^{-1} ABv = \lambda^{-1}\lambda v = v.$$ Similarly, if $v \in E_2$, then $$(f \circ g)(v) = B(\lambda^{-1}Av) = \lambda^{-1}BAv = v.$$ Therefore, $f$ is an isomorphism, and $E_1$ and $E_2$ have the same dimension.
How does this help us? Suppose $A$ and $B$ are $n \times n$ matrices. Note that $N(I + AB)$ is the eigenspace of $AB$ corresponding to $\lambda = -1 \neq 0$, and similarly for $N(I + BA)$. If we denote them by $E_1$ and $E_2$ respectively, then by the rank-nullity theorem, $$\operatorname{rank}(I + AB) = n - \operatorname{null}(I + AB) = n - \dim E_1 = n - \dim E_2 = \operatorname{rank}(I + BA).$$