I'm currently taking a university class on linear algebra. In some proofs, a given matrix $A \in \mathbb{R}^{m\times n}$ is said to be able to be represented as a $1\times n$ row vector of $m \times 1$ column vectors, i.e.:
$$
A = [\vec{a_1}\quad\vec{a_2}\quad\dots\quad \vec{a_n}]
$$
with $\vec{a_i}$ the ith column of $A$. Naturally, the transpose of $A$, as used in most such proofs, would then be given by an $n\times 1$ column vector of $1\times m$ row vectors:
$$
A^T = [\vec{a_1^T}\quad\vec{a_2^T}\quad\dots\quad \vec{a_n^T}]^T
$$
When performing matrix-vector multiplication of the form $A\vec x$, I know the vector's amount of rows has to match the matrix's amount of columns. (In this case, $\vec x$ would be an $n \times 1$ column vector, and the result would be an $[m \times n][n \times 1] = [m \times 1]$ column vector.)
Now, when considering $A$ as a row vector as in the first equation, we can see that this holds up, as the result would be $[1 \times n][n \times 1] = [1 \times 1]$, though containing a sum of scaled $m \times 1$ column vectors, so, after scaling and adding, an $m \times 1$ column vector.
$$
A\vec x = [\vec{a_1}\quad\vec{a_2}\quad\dots\quad \vec{a_n}][x_1\quad x_2 \quad\dots\quad x_n]^T = \vec{a_1}\cdot x_1 + \vec{a_2}\cdot x_2 + \dots + \vec{a_n}\cdot x_n \in \mathbb{R}^{m \times 1}
$$
My problem, then, arises when considering the same product, but swapping $A$ with $A^T$ (Edit: For clarity, this is not transposing $Ax$, but rather interchanging $A$ to see what happens.), thus $A^T\vec x$ with $\vec x$ the same $n \times 1$ column vector. Conventionally, this would be the product of an $n \times m$ matrix with an $n \times 1$ vector: impossible. However, when considering $A^T$ as an $n \times 1$ column vector like in the second equation, this does seem to become possible, namely as the dot of two vectors of equal dimensions:
$$
A^T\boldsymbol{\cdot}\vec x = [\vec{a_1^T}\quad\vec{a_2^T}\quad\dots\quad \vec{a_n^T}]^T\boldsymbol{\cdot}[x_1\quad x_2 \quad\dots\quad x_n]^T = \vec{a_1^T}\cdot x_1 + \vec{a^T_2}\cdot x_2 + \dots + \vec{a^T_n}\cdot x_n \in \mathbb{R}^{1\times n}
$$
This, to me, seems to be a paradox, as there obviously is a mismatch in the dimensions of $\vec x$ (i.e. $n$) and the dimensions of the input space of $A^T$ (i.e. $m$), yet, rewriting $A^T$ as a vector, as is done with $A$, eliminates said mismatch, as it seems. Is it wrong to assume one can eliminate this mismatch? Is it wrong to assume the matrix-vector product of a matrix in $\mathbb{R}^{n\times 1}$ with a vector in $\mathbb{R}^n$ to be equal to the dot product of two vectors in $\mathbb{R}^n$? Is it wrong to write a matrix as a vector of vectors, or perhaps to transpose such a construction?
Best Answer
There are 2 wrong things:
1) Although you can write an $n \times m$ matrix as a table with $m $ columns each of which is a column vector of $n $ entries, this does not mean you are allowed to consider that its dimension is $1 \times m $. It is still $n \times m$.
2) the dot product of vectors $a $ and $b $ is carried out as $a^T \, b $ so dimensions must not be equal, the must be $1 \times n $ and $n \times 1$.