Let's work on $\mathbb{R}^3$ for the time being and rotate our standard basis as
$$e_1 \to e_2 \\
e_2 \to -e_1 \\
e_3 \to e_3,$$
hence our transition matrix is
$$P = \begin{bmatrix}
0 & -1 & 0 \\
1 & 0 & 0 \\
0 & 0 & 1
\end{bmatrix}$$
, and consider the point $a = (1, 0 ,0)$ in standard basis. Now since we have rotated our, so called, x-y plane $90^{\circ}$ degree counterclockwise wrt z-axis, our point wrt to the new coordinate system should $a'=(0,-1,0)$. However, when we multiply $a$ with the matrix $P$ (from right, i.e P$\cdot $a), we get $(0, 1, 0)$, which is a wrong result, so where is my mistake ? I mean I have checked my logic and calculation over and over again, but couldn't find the mistake that I'm doing.
[Math] Change of basis with rotation matrices
change-of-basislinear algebrarotations
Best Answer
You've discovered the difference between covariant and contravariant. Column vectors are contravariant, so you must multiply by $P^{-1}$. Basically, you're not transforming your vector, but instead the coordinate system around the vector while the vector itself status the same. The consequence of this is the inversion of the transition matrix. This is easier to see intuitively if you, for instance, stretch $e_1$ by a factor of $a$ instead of rotating the coordinate system: the first component of any vector should be divided by $a$.
PS. Row vectors (i.e. linear maps from $\Bbb R^n$ to $n$) are covariant, so those you do multiply by $P$.
PPS. If you want to think through it, it helps to do it in two dimensions first. For instance, take two people trying to communicate where things in a city are in relation to a central square, using a map. One uses meters east and north, the other uses blocks to the right and up (the city grid isn't entirely aligned with the compass directions, and say the blocks are about $100\times 200$ meters, whatever gives them nice coordinates).
Take an arbitrary point in the city (concrete or genera, doesn't really matter). Write down the pair of numbers each of the two would use to describe it, and also how each of the two would describe the other person's basis vectors.
Now, without thinking about matrix multiplication and linear algebra, work through how each of the two would translate the other's description of the point to their basis. Recognize that what you actually do is a matrix-vector multiplication, and take note of which matrix is multiplied with which vector.
There really isn't anything deeper to it than the "it just happens this way" that you get from this example.