Let A be an $n\times n$ matrix then,
$$trace (A^2)=trace(AA') \iff A= A'$$
where $A'$ is the transpose of $A$.
only if part is easy by observing that $A=A'$ implies that $A^2=AA'$ and trace of $A^2-AA'$ is zero, since all the entries are zero.
But the if part is ….
Best Answer
You sure this is correct for general $A$?
Let $A$ be a matrix such that $tr(A)=0$. Let $$A'=A+I$$ Then $$tr(AA')=tr(A^2)$$ but $$A \ne A'$$
ADDED: Let $$B=\frac{A+A^T}{2}$$ $$C=\frac{A-A^T}{2}$$
$$tr(A^2)=tr(AA^T)$$ $$tr((B+C)(B+C))=tr((B+C)(B-C))$$ $$tr(B^2+BC+CB+C^2)=tr(B^2-BC+CB-C^2)$$ $$tr(BC+C^2)=tr(-BC-C^2)=-tr((B+C)C)$$ $$=-tr(C(B+C))=-tr(BC+C^2)$$
Hence $$tr(BC+C^2)=tr(C^2)=0$$ $$tr(CC^T)=0$$ $$\sum_{ij}C_{ij}^2=0$$ $$C_{ij}=0$$ for all $i,j$. Hence, $$A=A^T$$.
Note: I have assumed real matrices. I fancy your $A'$ is in fact meant to be the Hermitian adjoint $A^\dagger$， for complex $A$ in general.