I was learning what Hilbert space was and this is the definition that I have:
$\mathcal{H}$ Hilbert Space is a vector space with $\langle \cdot , \cdot\rangle$ inner product and is complete with respect to the norm induced by that inner product $||\cdot||_{\langle \cdot , \cdot\rangle}$.
So the definition made sense to me because it just meant that I had some space of vectors $<a,b,c,…>$ that had a corresponding inner product and that vector space had "no holes" wrt to the norm defined from their inner product.
That made sense to me until further down the course they said the following about their orthonormal basis:
A Hilbert space has a countable orthonormal basis ($\phi_1, \phi_2, …,\phi_i,…$) if and only if it is separable and we can write every element in it as follow:
$$f = \sum^{\infty}_{i=1} \langle f, \phi_i \rangle \phi_i$$
for all $f \in \mathcal{H}$.
However, this kind of breaks what I thought a Hilbert space was. The reason is that I cannot seem to find a way to resolve the conflict in my head that this space is a vector space and a function space. I thought that the Hilbert space was some (vector) space with a inner product and that had a norm that was complete. A vector for me (at least intuitively) is a sequence of numbers that specifies some location (or direction to move in) or element in your set. If we think of a function as a algorithm, how is it possible to have a linear combination of algortihms/functions and get another one? How can it make sense to express a function as a vector? How does it make sense to add them and hope that they are in the space of functions you are?
Basically, is a Hilbert Space a vector space or a function space?
The instructors did give us some examples of what Hilbert spaces were:
1) $\mathbb{R}^n$
This one makes sense to me because its just a vector space in the usual way with no weird function being the set.
However they also give as an example, the space of functions that are square summable:
2) $L_2([0,1]) = \{f:[0,1] \rightarrow \mathbb{R} | \int^{1}_{0} f(x)^2 dx < \infty \}$
Which makes less sense.
I actually do not have a problem thinking of a set as a collection of functions with some properties. I am ok with that, I am just unclear of what that means precisely in the context of Hilbert spaces. Are the elements of the space, the functions themselves and the inter product defined in terms of these functions? How does it make sense to talk about basis vectors of a space of functions and how do you "add" elements in a space of functions.
NOTE: my background on analysis is limited and mostly self taught.
Best Answer
I think your is just a matter of definition.
A real vector space is an abelian group $(V, +)$ endowed with a scalar multiplication $\mathbb{R} \times V \longrightarrow V$, satisfying some axioms (associativity, distributivity, etc...).
Given a vector space, its elements are called vectors.
In particular, some function spaces (for example $L^2([0,1])$ and $ C^0(\mathbb{R})$) satisfy the vector space axioms, so they are vector spaces. You are asking what is the sum of two functions: the answer is pointwise sum. This means that given two functions $f(x)$ and $g(x)$ you can define $f+g$ as $x \mapsto (f(x)+g(x))$.
Note that $\mathbb{R}^n$ is a function space as well, since its elements $(x_1, \dots, x_n)$ can be thought as particular functions $\{1, \dots, n\} \longrightarrow \mathbb{R}$ by $i \mapsto x_i$.