I am working through Georgi Shilov's Linear Algebra, and I am having trouble understanding the vector component transformation matrix definition he gives in section 5.31. I will describe this definition below.
Let $e_1, e_2, \dots, e_n$ and $f_1, f_2, \dots, f_n$ be two bases in a vector field of dimension $n$ such that for some quantities $p^{(j)}_i$ we have
$$f_j = p^{(j)}_1 e_1 + p^{(j)}_2 e_2 + \dots + p^{(j)}_n e_n$$
Shilov now defines the matrix of the transformation from basis $\{e\}$ to basis $\{f\}$ as
$$P =
\begin{bmatrix}
p^{(1)}_1 & p^{(2)}_1 & \dots & p^{(n)}_1 \\
p^{(1)}_2 & p^{(2)}_2 & \dots & p^{(n)}_2 \\
\vdots & \vdots & \ddots & \vdots \\
p^{(1)}_n & p^{(2)}_n & \dots & p^{(n)}_b
\end{bmatrix}
$$
So, $P$ is the matrix with components of $f_i$ in terms of the basis ${e}$ as columns.
Now, suppose we have some vector $x = \xi_1 e_1 + \xi_2 e_2 + \dots + \xi_n e_n = \eta_1 f_1 + \dots + \eta_n f_n$. Then, the author claims that the matrix describing the transformation from the components $\xi_1, \dots, \xi_n$ to the components $\eta_1, \dots , \eta_n$ is
$$S = (P^{-1})^T$$
In my understanding, the "matrix describing the transformation from the components $\xi_1, \dots, \xi_n$ to the components $\eta_1, \dots , \eta_n$" means that
$$S\begin{bmatrix} \xi_1 \\ \xi_2 \\ \vdots \\ \xi_n \end{bmatrix} = \begin{bmatrix} \eta_1 \\ \eta_2 \\ \vdots \\ \eta_n \end{bmatrix}$$
However, that doesn't seem to be the case. Consider an example where $e_1 = \begin{bmatrix} 1 \\ 0 \\ 0 \end{bmatrix}, e_2 = \begin{bmatrix} 0 \\ 1 \\ 0 \end{bmatrix}, e_3 = \begin{bmatrix} 0 \\ 0 \\ 1 \end{bmatrix}$ and $f_1 = \begin{bmatrix} 1 \\ 1 \\ 1 \end{bmatrix}, f_2 = \begin{bmatrix} 0 \\ 1 \\ 1 \end{bmatrix}, f_3 = \begin{bmatrix} 0 \\ 0 \\ 1 \end{bmatrix}$. Then, the matrix of the transformation from $\{e\}$ to $\{f\}$ and the respective component transformation matrix is
$$P = \begin{bmatrix} 1 & 0 & 0 \\ 1 & 1 & 0 \\ 1 & 1 & 1 \end{bmatrix} \quad S = \begin{bmatrix} 1 & -1 & 0 \\ 0 & 1 & -1 \\ 0 & 0 & 1 \end{bmatrix}^T = \begin{bmatrix} 1 & 0 & 0 \\ -1 & 1 & 0 \\ 0 & -1 & 1 \end{bmatrix}$$
However, if we try to use it to transform the vector $[5, 1, 2]_{\{e\}} = [5, -4, 1]_{\{f\}}$ we get
$$\begin{bmatrix} 1 & 0 & 0 \\ -1 & 1 & 0 \\ 0 & -1 & 1 \end{bmatrix}\begin{bmatrix} 5 \\ 1 \\ 2 \end{bmatrix} = \begin{bmatrix} 4 \\ -1 \\ 2 \end{bmatrix}$$
Which is not what I expected. However, if we instead take $S = P^{-1}$, it seems to work out:
$$\begin{bmatrix} 1 & -1 & 0 \\ 0 & 1 & -1 \\ 0 & 0 & 1 \end{bmatrix}\begin{bmatrix} 5 \\ 1 \\ 2 \end{bmatrix} = \begin{bmatrix} 5 \\ -4 \\ 1 \end{bmatrix}$$
So, am I misunderstanding what the component transformation matrix is supposed to do, or did I construct $P$ incorrectly? I did my best to write the definitions as they are written in the text, however, it is very possible I just mixed up my indices.
Best Answer
Let's set $\mathcal{E} = (e_1, \dots, e_n)$ and $\mathcal{F} = (f_1, \dots, f_n)$. Also, let's denote the coordinates of a vector $v$ with respect to a basis $\mathcal{E}$ (represented as a column vector) by $[v]_{\mathcal{E}}$. Then
$$ v = \begin{bmatrix} 5 \\ 4 \\ 2 \end{bmatrix}, \, [v]_{\mathcal{E}} = \begin{bmatrix} \xi_1 \\ \xi_2 \\ \xi_3 \end{bmatrix} = \begin{bmatrix} 5 \\ 4 \\ 2 \end{bmatrix}, \, [v]_{\mathcal{F}} = \begin{bmatrix} \eta_1 \\ \eta_2 \\ \eta_3 \end{bmatrix} = \begin{bmatrix} 5 \\ -4 \\ 1 \end{bmatrix}. $$
First of all, it seems you have the role of $S$ and $P^{-1}$ in your question reversed as you actually have $$ P^{-1} = \begin{bmatrix} 1 & 0 & 0 \\ -1 & 1 & 0 \\ 0 & -1 & 1 \end{bmatrix}, \,\,\, S = \left( P^{-1} \right)^T = \begin{bmatrix} 1 & -1 & 0 \\ 0 & 1 & -1 \\ 0 & 0 & 1 \end{bmatrix}.$$ The multiplications you perform later are also incorrect. However, your actual conclusion is true: In order to compute the coordinates of $[v]_{\mathcal{F}}$ from $[v]_{\mathcal{E}}$, you need to multiply $[v]_{\mathcal{E}}$ by $P^{-1}$, not $\left( P^{-1} \right)^T$. This is the meaning of equation $(12)$ Shilov writes on page 122 (where $Q = P^{-1}$).
However, Shilov is also not entirely wrong in claiming that the matrix "describing the transformation from the components $\xi_1,\dots,\xi_n$ to the components $\eta_1,\dots,\eta_n$ is $\left( P^{-1} \right)^T$". Why? If you write explicitly $Q [v]_{\mathcal{E}} = [v]_{\mathcal{F}}$ (equaton $(12)$) you get $$ \eta_1 = q_1^{(1)} \xi_1 + \dots + q^{(n)}_1 \xi_n, \\ \dots, \\ \eta_n = q_n^{(1)} \xi_1 + \dots + q^{(n)}_n \xi_n. $$
If you think of the components $\xi_1, \dots, \xi_n$ and $\eta_1, \dots, \eta_n$ as two "bases" and compare this equation to the equation you wrote in the beginning of the question, you will see that the "matrix of the transformation from the basis $\{ \xi \}$ to the basis $\{ \eta \}$" is actually $Q^T$ and not $Q$. To make this statement precise, one needs to discuss dual spaces and then the "components" of a vector with respect to a basis become the dual basis to the original basis and the matrix $\left( P^{-1} \right)^T$ becomes the change of basis matrix between the dual bases.