I recently came across this question in the context of a course on functional analysis. This question was posed by a friend of mine, and the Wikipedia pages and existing MO threads are too dense for me to understand. Some SE threads we've looked at include this, this, and this – I feel none of these really answer the essence behind our question (the last one comes close but only briefly touches on the question here).
It is a basic result that separable Hilbert spaces are characterised by the existence of a countable basis, from which we can see that any (infinite-dimensional) separable Hilbert space is isometrically isomorphic to $\ell^2$. What we are wondering is why this result is helpful – we have tried to explain the build up to that below.
Classical PDE systems assume that the defining equations are smooth, but it turns out that this is a really strong condition to enforce (for example, take a square wave in the wave equation). By relaxing this assumption, there are certain measure theoretic issues that we run into, and so we instead have to consider the equations in their equivalence classes (e.g. in $L^2$, where for example the square wave is differentiable almost everywhere). In particular, the subspaces that these solutions live in are called Sobolev spaces, which are themselves Hilbert spaces.
It turns out that for some values of $p$, the Sobolev spaces are also separable, and so all the results from functional analysis can be applied to show existence and uniqueness of solutions to these linear PDEs. Additionally, it turns out that the same thing works for non-integral $p$ (Bessel spaces) which are also Hilbert space when $p = 2$.
The specific question that we still don't have a good answer to is
Why do we care about separable Hilbert spaces being equivalent to $\ell^2$? What property of $\ell^2$ makes it so useful in solving systems of PDEs?
It appears that the square summable sequences look similar to the Fourier series which are used to define Sobolev spaces, but this link is not made clear anywhere that we can find. In addition, even with such a link, we are unable to find an explanation for why/how this allows us to solve certain systems of PDEs.
A good answer for this would be one which explains this link and motivation (to a background strong in algebra and Riemannian geometry and understanding graduate-level functional analysis and measure theory, but with a much more elementary foundation of PDEs), or otherwise recommends some light, introductory resources and a short summary of what is really going on here.
Best Answer
This questions has two aspects which should be considered separately. Even non-separable Hilbert spaces are isomorphic to $\ell^2(S)$ for some set $S$ (which in that case is uncountable) and one can ask why this is useful. The second question is then why separability is interesting.
For the first question it may be helpful to have a careful look at the following analogy: Every finite dimensional (real) vector space is isomorphic to $\mathbb{R}^n$ for some $n$. Why do we care?
a) It is a satisfying classification result. There are no weird finite dimensional vector spaces besides the obvious ones.
b) If you can prove a certain property over $\mathbb{R}^n$ and know that it is invariant under basis changes, then the same property is true for a finite dimensional vector space. E.g. you could use this to prove $\det (AB)= \det A \cdot \det B$, first for matrices over $\mathbb{R}^n$, then for any finite dimensional vector space. (Though there are more elegant proofs of the identity). This only uses the fact that there does exist some isomorphism to $\mathbb{R}^n$.
c) Sometimes it is useful to choose a particular basis to understand a problem better. This amounts to finding a concrete isomorphism to $\mathbb{R}^n$ and then exploiting it.
Now, how does the situation look like for Hilbert spaces? The analogy of a) still holds and although it does not have any direct consequences to PDE's it is still worth knowing. Also b) remains valid: You can have a look at trace-class or Hilbert-Schmidt operators: In a sense one uses the isomorphism to $\ell^2$ in the very definition. But e.g. when showing that said operators are compact, one can do this for the respective operators on $\ell^2$ and then conclude for an arbitrary Hilbert spaces. Maybe c) is the most interesting one for PDE's so let me expand on that:
Say you want to solve the heat equation $\partial_t u = \Delta u$ on some domain $M$. In many cases the Laplace operator has a sequence of eigenfunctions $u_1,u_2,\dots$ satisfying $\Delta u_k = \lambda_k u_k$ for eigenvalues $\lambda_1\le \lambda_2\le \dots $. All of these functions are smooth (elliptic regularity) and indeed form a orthogonal basis of $L^2(M)$. Expressing functions in $L^2(M)$ in this basis amounts to using a particular isomorphism to $\ell^2(\mathbb{N})$ under which the heat equation turns into the decoupled system of equations $\partial_t a_k = \lambda_k a_k$ for a time dependent $(a_k)\in \ell^2$. This isomorphism is in analogy to the diagonalisation of matrices and e.g. in the case that $M$ is a torus, is implemented by the Fourier transform. However, in order to construct this particular isomorphism the general classification result of Hilbert spaces is of no use, unless that it maybe puts you in the right mindset.
The second question regarding separability makes sense to ask for a much wider class of topological vector spaces, but let's just talk about Banach spaces as you are familiar with them. One consequence of this is that weak*-topology becomes metrisable and thus amenable to testing with sequences (rather than nets). Another one is in measure theory - over separable Banach spaces stochastic processes have many desirable properties. There are probably even better answers to that, but I am sure you will find them yourself in MSE or elsewhere.