Before I begin, I understand why inner products need to maintain conjugate symmetry as seen in this post. This is not what my question is directly about. My motivation is to derive complex Fourier series through the complex inner product.
Suppose we have an inner product defined as $$\langle f,g\rangle=\int_a^b{f(x)\overline{g(x)}dx}$$
I have seen that the projection of a function $f$ onto $g$ is then defined as: $$\left(\frac{\langle f,g \rangle}{\langle g,g \rangle}\right)g=\left(\frac{1}{{\Vert g\Vert}^2}\int_{a}^{b}{f(x)}\overline{g(x)}dx\right)g$$
From this, I have previously taken the inner product (or really the dot product) to intuitively be the scaled norm of the projection of $f$ onto $g$ (that is, projecting $f$ onto $g$ and then scaling it by the norm of $g$). Combined with the fact the inner product of a vector with itself is the square of its norm, this means that the equation for projection is essentially dividing out the norm of $g$ from inner product of $f$ with $g$ in order to get the length of the component of $f$ along $g$, and then driving in the direction of $g$ by multiplying the component by a normalized version of $g$.
Something important to this intuition to me was that the inner product with purely real functions is symmetric. When dealing only with real valued vectors/functions, the operation of projecting $f$ onto $g$ and scaling it by $g$ is the same as projecting $g$ onto $f$ and scaling it by $f$. In other words, the order of the arguments don't matter.
$$\langle f,g\rangle = \int_a^b{f(x)g(x)dx} = \int_a^b{g(x)f(x)dx} = \langle g,f\rangle$$
However, this is not the case with complex valued function. Why, in the complex case, do we interpret the function that is conjugated in the integral ($g$) to be the function we are projecting onto? Why could we not interpret the function that we are taking the conjugate of to be the function we are projecting onto another function instead?
In essence, why is $$\langle f,g\rangle=\int_a^b{f(x)\overline{g(x)}dx}$$ interpreted projecting $f$ onto $g$ while $$\langle g,f\rangle=\int_a^b{g(x)\overline{f(x)}dx}$$ interpreted as projecting g onto f?
Where does conjugation fit into this geometric intuition behind projection (if it does)?
Best Answer
What I'm about to say works for finite dimensional spaces and might not extend perfectly to infinite dimensional spaces, but at least it builds intuition.
Given a basis $u_i$ of a vector space $V$ we want to find the components of any other vector $v$ is this basis. How do we accomplish that? In any vector space, there is a unique dual basis $\mu^i \in V^*$ such that $\mu^i(v_j) = \delta^i_j$. Once we have this then trivially $$ v = \sum_i\mu^i(v)u_i. $$
An inner product is a particular kind of antilinear isomorphism $\xi : V \cong V^*$ (a correlation between them if you will) in the sense that $$ \langle v, w\rangle = \xi(v)(w). $$ (Antilinear as you may guess means that $\xi(av) = \bar a\xi(v)$.) What does the dual basis correlate with? The vectors $u^i = \xi^{-1}(\mu^i)$ are called the reciprocal basis of the $u_i$; they can also be defined explicitly in terms of the inner product by the equations $$ \langle u^i, u_i\rangle = \langle u_i, u^i\rangle = \delta^i_j. \tag{$*$} $$ So we can use the reciprocal basis in the same way as the dual basis $$ v = \sum_i\langle u^i, v\rangle u_i. $$ Why does $u^i$ appear in the first slot? Actually, your attention should be on $v$: if we did it the other way then $$ av = \sum_i\langle av, u^i\rangle u_i = \bar a\sum_i\langle v, u^i\rangle u_i = \bar av $$ which is bad. So $v$ must appear in the linear argument of the inner product.
Now look again at the relations ($*$); they look an awful lot like orthonormality, doesn't it? This is no mistake: a basis $u_i$ is orthonormal if and only if $u^i = u_i$! More generally, it is orthogonal if $u^i \propto u_i$, or even better we can easily see that it must be that $$ u^i = \frac{u_i}{\langle u_i,u_i\rangle}. $$ This means that
We call this orthogonal-extension-independet quantity the orthogonal projection of $v$ onto $u$.