The inner product in a 'regular' vector space can be seen as:
$$ \langle \vec{v}, \vec{w} \rangle = \sum v_i w_i = |v| |w| \cos \alpha $$
So the 'sum' expression and the 'angle' expression are equal, but different ways to calculate the dot product.
The angle $\alpha$ is easily visualized as the angle between the vectors $v$ and $w$.
Now for function vector spaces we have the inner product:
$$ \langle f(x), g(x) \rangle =\int f(x)^*g(x)dx = |f||g| \cos \alpha ….(?)$$
The following connection is kind of intuitive, by going from a discrete sum to a continuous sum/integral:
$$ \sum v_i w_i \rightarrow \int f(x)^*g(x)dx $$
However is there any validity to:
$$ \langle f(x), g(x) \rangle = |f||g| \cos \alpha ….(?)$$
I think here $|f|$ is calculated with $\int |f|^2 dx $, however what does the $\alpha$ in this case express? Is there a way to intuitively understand the 'angle' between functions?
Best Answer
First of all, it's important to check that this angle can be properly defined. Supposing we want to write:
$$ \theta = \cos^{-1}\left(\frac{\langle f, g \rangle}{\|f\|\|g\|}\right),$$
we want to be sure that the quantity inside the $\cos^{-1}$ is in $[-1,1]$, so we can find such an $\theta$. This is guaranteed by the Cauchy-Schwarz inequality, which says that $|\langle f,g \rangle| \leq \|f\|\|g\|$, so that $-1 \leq \frac{\langle f, g \rangle}{\|f\|\|g\|} \leq 1$. Thus, we can say that there is an angle $\theta$ between $f$ and $g$.
As for interpretation, I think it's useful to think of projections. This figure below shows that, in Euclidean space, if I want to project $a$ onto $b$, the length of the projection is $\|a\| \cos \theta$. If $\theta = 0$, so that the $a$ an $b$ are parallel, the projection has length $\|a\|$. If $\theta = \pi/2$, so that $a$ and $b$ are perpendicular, the projection has length $0$.
The same intuition works for functions. $\cos \theta$ measures the extent to which $f$ shrinks when projected on to $g$, or vice versa. And if you're wondering how to define this projection, the defining property is that $f - \operatorname{proj}_{g}(f)$ should be perpendicular to $g$, or equivalently, that $\operatorname{proj}_{g}(f)$ is the closest point to $f$ on the line passing through $g$, which in the case of functions is simply $\{cg \mid c \in \mathbb{R}\}$.
So perhaps I will summarize things as follows: $\cos \theta$ tells us how much $f$ shrinks when we project it on to its closest constant multiple of $g$, or vice versa.