This is a general question that came to my mind while listening to a lecture(although its framing may make it look like a textbook question).
Suppose that $A$ and $B$ be real matrices. $A$ is symmetric and positive semi-definite$(x^tAx\ge0\ \ \forall x\in \mathbb{R}^n)$.
Let $AB+BA=0$. Does this imply $AB=BA=0$? If yes can you give me some example?
Best Answer
Let $v$ be an eigenvector of $A$ with eigenvalue $\lambda$. Then $$ 0 = A(Bv) + B(Av) = A(Bv)+\lambda Bv\Longrightarrow A(Bv) = -\lambda Bv $$ what this means is that $Bv$ is also an eigenvector of $A$ with eigenvalue $-\lambda$. But since $A$ is positive semi-definite all eigenvalues of $A$ are $\geq 0$. If $\lambda=0$, so we have two possiblities:
Since all vectors can be written as a linear composition of these eigenvectors, we have $AB=BA = 0$ in these circumstances.
Extra: If you are also interested in what this $B$ can actually be if $AB+BA = 0$, note that if $w$ is any vector, and $P_0$ is the porjection linear transformation which sends $w$ to the subspace of vectors with eigenvalue zero. Then we know the following:
Write $w = P_0w + (1-P_0)w$, then by second observation above $B(1-P_0)w = 0$. So in the basis in which $A$ is diagonal (since $A$ is symmetric it is diagonalizable), $B$ is necessarily of the form $$ B=\begin{pmatrix} B_0 & 0\\ 0 & 0 \end{pmatrix} $$ In other words $B$ satisfies $AB+BA=0$ if and only if $P_0BP_0=B$.