I was thinking about the problem that says:
If $A$ and $B$ are $3\times 3$ real matrices such that $\operatorname{rank}(AB)=1$, then $\operatorname{rank}(BA)$ can not be which of the following?
(a) $0$
(b) $1$
(c) $2$
(d) $3$.
My attempt: I have chosen suitable $3 \times 3 $ matrices for $A$ and $B$ keeping in mind that $\operatorname{rank}(AB)=1$. Say for example if I take $A$ and $B$ to be
$$A = \begin{pmatrix}
1 &2 &0 \\
0 & 0 &0 \\
0 & 0 &0
\end{pmatrix}$$
and
$$B = \begin{pmatrix}
-2 &1 &0 \\
1 & 0 &0 \\
0 & 0 &0
\end{pmatrix}$$
respectively, then I see $\operatorname{rank}(AB) = \operatorname{rank}(BA) = 1$. So, option (b) can not be correct. Do I have to keep choosing the matrices and then observe which of the option holds good. Is this kind of approach right to tackle the problem? I am looking for a direct way (e.g. application to some theorem) which can give me the result. I have also noticed that $AB$ and $BA$ are similar matrices as we see that $A^{-1}(AB)A=BA$. Is this observation going to help me in any way? Thanks in advance for your time.
Best Answer
If you only want to solve the problem, then as the others have pointed out, (d) is impossible and hence the answer is (d). It is helpful, however, to constuct some examples to convince yourself that (a)-(c) are really possible (this is also a sanity check against misprints). For this purpose, you may consider the matrices $$ A=\begin{pmatrix}1\\&1\\&&0\end{pmatrix},\ B=\begin{pmatrix}b&0&1\\0&0&0\\0&c&0\end{pmatrix}. $$ Then $\operatorname{rank}(AB)=1$. Now you may try to choose some appropriate values of $b$ and $c$ to make $\operatorname{rank}(BA)$ equal to $0,1$ or $2$.