As a student in an introductory course in linear algebra, bases are one of the fundamental concepts that I've had the most troubles wrapping my head around.
$$
u_1=(1,1,1),\; u_2=(1,2,0),\; u_3=(2,1,3)
$$
Now let's say I construct three orthogonal vectors in the basis $u_1,u_2,u_3$
$$
w_1=a_1u_1=(a_1,0,0)\\
w_2=a_2u_2=(0,a_2,0)\\
w_3=a_3u_3=(0,0,a_3),
$$
$a_i\in\mathbb{R}.$ How can they be orthogonal in one basis but not in another? Can you even say the vectors $w_1,w_2,w_3$ are orthogonal in the basis $u_1,u_2,u_3$? I tried drawing it and drew the conclusion that you can't, but still, if I took the dot products of the vectors $w_1,w_2,w_3$, the conclusion would be that they are orthogonal. What am I missing here?
Best Answer
You are missing that when you change basis, if you want to keep the values of scalar products between given vectors unchanged (which is desirable because scalar products and lengths are there before we impose a coordinate system and thus shouldn't change when we change coordinate systems), then you also have to change the calculation of the inner product.
For a simpler example, if we have the vectors $u_1 = (2, 0)$ and $u_2 = (0,2)$ in $\Bbb R^2$ using the standard basis. Then we have that $u_1\cdot u_1 = 4$. If we change our basis to $u_1, u_2$, then suddenly all naively calculated inner products become four times smaller, such as $u_1\cdot u_1 = 1^2 = 1$.
The remedy (in this example multiply by $4$) is to insert a matrix between the row and column vectors when doing scalar products. Specifically, if $A$ is the change of basis matrix for the right direction (in this example $2I$), then you calculate the inner product between two vectors $v_1, v_2$ (expressed in the new basis) as $v_1^T A^TAv_2$. The effect of this is to transform $v_1$ and $v_2$ into their standard basis representations $Av_1$ and $Av_2$ before transposing one of them and matrix-multiplying them together.
In your example, we have $$ A = \begin{bmatrix}1& 1& 2\\1& 2& 1\\1& 0& 3\end{bmatrix} $$ and so the scalar product of $w_1$ and $w_2$ is calculated as $$ \left(\begin{bmatrix}a_1&0&0\end{bmatrix}\begin{bmatrix}1&1&1\\1&2&0\\2&1&3\end{bmatrix}\right)\left(\begin{bmatrix}1& 1& 2\\1& 2& 1\\1& 0& 3\end{bmatrix}\begin{bmatrix}0\\a_2\\0\end{bmatrix}\right)\\ = a_1a_2\begin{bmatrix}1&1&1\end{bmatrix}\begin{bmatrix}1\\2\\0\end{bmatrix} = 3a_1a_2 $$
Note that the matrix $B = A^TA$ is a symmetric (and positive-definite) matrix. In fact, any symmetric positive-definite matrix $B$ may be thought of as representing the scalar product over some basis in this way.