I'm going through the book Gilbert Strang – Introduction to Linear Algebra, Third Edition (2003) and in the section for transpose matrices right near the end, there is this paragraph: 
I completely understand the transformation, but I don't understand what the 'deep purpose' here really is.
Also, I don't understand in the last sentence how "$A$ moves from one side of the dot product to the other".
What i see in $(Ax)^Ty$ is: $A$ firstly linearly transforms the vector x and then the resulting vector is turned into a linear transformation on its own (thanks to the transpose operation). Now, y is linearly transformed with this new transformation and the end result is a single number.
My point is, there was never explicit dotting between x and y in the first place, so what does the author mean $A$ going from one side to the other?
Best Answer
By dotting, the author probably means that $\vec{a} .\vec{b} = a^Tb$, for vectors $a$ and $b$, so
$(Ax)^Ty = (Ax).y$,
and $x^T(A^T y ) = x.(A^Ty)$, note that $Ax$ and $A^Ty$ are vectors.