I gave the following problem to students:
Two $n\times n$ matrices $A$ and $B$ are similar if there exists a nonsingular matrix $P$ such that $A=P^{-1}BP$.
Prove that if $A$ and $B$ are two similar $n\times n$ matrices, then they have the same determinant and the same trace.
Give an example of two $2\times 2$ matrices $A$ and $B$ with same determinant, same trace but that are not similar.
Most of the ~20 students got the first question right. However, almost none of them found a correct example to the second question. Most of them gave examples of matrices that have same determinant and same trace.
But computations show that their examples are similar matrices. They didn't bother to check that though, so they just tried random matrices with same trace and same determinant, hoping it would be a correct example.
Question: how to explain that none of the random trial gave non similar matrices?
Any answer based on density or measure theory is fine. In particular, you can assume any reasonable distribution on the entries of the matrix. If it matters, the course is about matrices with real coefficients, but you can assume integer coefficients, since when choosing numbers at random, most people will choose integers.
Best Answer
If $A$ is a $2\times 2$ matrix with determinant $d$ and trace $t$, then the characteristic polynomial of $A$ is $x^2-tx+d$. If this polynomial has distinct roots (over $\mathbb{C}$), then $A$ has distinct eigenvalues and hence is diagonalizable (over $\mathbb{C}$). In particular, if $d$ and $t$ are such that the characteristic polynomial has distinct roots, then any other $B$ with the same determinant and trace is similar to $A$, since they are diagonalizable with the same eigenvalues.
So to give a correct example in part (2), you need $x^2-tx+d$ to have a double root, which happens only when the discriminant $t^2-4d$ is $0$. If you choose the matrix $A$ (or the values of $t$ and $d$) "at random" in any reasonable way, then $t^2-4d$ will usually not be $0$. (For instance, if you choose $A$'s entries uniformly from some interval, then $t^2-4d$ will be nonzero with probability $1$, since the vanishing set in $\mathbb{R}^n$ of any nonzero polynomial in $n$ variables has Lebesgue measure $0$.) Assuming that students did something like pick $A$ "at random" and then built $B$ to have the same trace and discriminant, this would explain why none of them found a correct example.
Note that this is very much special to $2\times 2$ matrices. In higher dimensions, the determinant and trace do not determine the characteristic polynomial (they just give two of the coefficients), and so if you pick two matrices with the same determinant and trace they will typically have different characteristic polynomials and not be similar.