I understand the definition of a Hilbert space. But I do not understand why non-commutativity compels us to use Hilbert spaces.
It doesn't, but that's not what Scrinzi is saying.
The reason is doesn't is because we could work, for example, in Wigner quasiprobability representation:
$$\rho\mapsto W(x,p) = \frac{1}{\pi\hbar}\int_{-\infty}^\infty\langle x+x'|\rho|x-x'\rangle e^{-2ipx'/\hbar}\,\mathrm{d}x'\text{,}$$
where for pure states $\rho = |\psi\rangle\langle \psi|$ as usual, and Hermitian operators correspond to functions through the inverse Weyl transformation.
Note that $W(x,p)$ is a real function that's just like a joint probability distribution over the phase space, except that it's allowed to be negative. The uncertainty principle requires us to give up something, but it doesn't actually force Hilbert spaces on us.
However, what Scrinzi is saying is two things: (a) Hilbert spaces are very convenient to us in quantum mechanics, and (b) Hilbert spaces could be used in classical mechanics, but because non-commutativity doesn't exist in classical mechanics, it's "overkill" there, whereas it's "just the right amount of kill" in quantum mechanics. Both claims are correct.
The reason we can have used Hilbert spaces in classical mechanics is because they can represent very general algebras of observables, while the classical algebra of observables, being commutative, is actually simpler. (Cf. the Gel'fand–Naimark theorem for $C^*$-algebras in particular.)
The Hilbert space formulation of classical mechanics was done by Koopman and von Neumann in 1931-1932. But what their formulation actually does is generalize classical mechanics unless one imposes an artificial restriction that you're only ever allowed to measure observables in some mutually commutative set; only then is the classical (in the 19th century sense) mechanics is recovered exactly.
It is that artificial restriction that quantum mechanics lifts. Physically, non-commutativity observables corresponds to an uncertainty principle between them.
Let $H$ be a (separable) Hilbert space, such as we have in quantum mechanics. You can say that its inner product is a map $H\times H\to\mathbb{C}, (\lvert v\rangle,\lvert w\rangle)\mapsto \langle v\vert w\rangle$, and for $\lvert v\rangle\in H$ we can define $\langle v\vert \in H^\ast$ by the map $H\to \mathbb{C}, \lvert w\rangle\mapsto \langle v \vert w\rangle$. This is how the idea of dual spaces arises naturally for inner product spaces - the inner product gives you a natural map $H\to H^\ast, \lvert v\rangle \to \langle v\rvert$ from the space to its dual and the Riesz representation theorem establishes that this map is an (anti-)isomorphism.
Therefore, it is mathematically completely equivalent to view $\langle v\vert w\rangle$ either as the inner product of two vectors in $H$, or as the action of an element of $H^\ast$ on $H$. There is rarely any "need" to consider a particular of these viewpoints - since they are equivalent, you can rephrase every statement about the inner product as a statement about the dual and vice versa - but some things might be easier to express.
Note that the dual space and the map from kets to bras is completely fixed by $H$ and its inner product - there is no choice here, there isn't any additional information needed to talk about the dual space, so it's a bit strange to ask whether we "need" the dual space - it's just something that exists.
The notion of the dual becomes more subtle and more relevant as something distinct from the inner product when you try to formalize what certain objects like the "eigenstates" $\lvert x_0\rangle$ of the position operator are. They aren't elements of the Hilbert space, which is easiest to see when people write them in the position basis as $f(x) = \delta(x - x_0)$ - this isn't even a proper function, let alone an element of the Hilbert space of square-integrable functions $L^2$. However, the $\delta$-function is a tempered distribution $S^\ast$, which is the dual space to the space of Schwartz functions $S$. That is, if you want to be careful about it, $\langle x\rvert$ exists as a bra on the space of "Schwartz kets", but not as a ket, nor as a bra on the full Hilbert space. The sequence of spaces $S\subset L^2 \subset S^\ast$ is known as a Gel'fand triple and leads one to the notion of a rigged Hilbert space.
Best Answer
I think there are two distinct ways that one could interpret this question, so I'll try my hand at answering both.
This one is pretty straightforward. If you can add things together, multiply them by constants, and take inner products, then you're essentially$^\dagger$ working with a Hilbert space. The wavefunctions which you are comfortable solving for are elements of such a space, and the self-adjoint operators which represent observables are linear maps from one element of the space to another.
Linear algebra is just the study of vector spaces and linear maps between them, so it is in particular the backdrop of all of the calculations you are performing. When you solve an eigenvalue equation like the Schrodinger equation, you're doing linear algebra. When you expand a generic state as a superposition of eigenstates of some observable, you're doing linear algebra. When you are confident that such a set of eigenfunctions even exists and that the corresponding eigenvalues are real, it is because you have learned the spectral theorem for self-adjoint operators, which is a central result in (you guessed it) linear algebra (or functional analysis, which is essentially linear algebra in infinite dimensional spaces).
In that sense, it's not so much that linear algebra is useful in the standard formulation of quantum mechanics; its that the standard formulation of quantum mechanics is linear algebra, whether you'd like to call it that or not. To be sure, particular techniques, theorems, and general mindsets from linear algebra can be extremely useful for performing calculations, making up models, etc - but the fact remains that no matter how you slice it, what you are doing is linear algebra on a Hilbert space.
$^\dagger$Actually this describes what's called a pre-Hilbert space. To be a full Hilbert space, there's an additional technical requirement called completeness. Loosely, this means that sequences which "look like" they should converge actually do. This is important whenever you use a limit, which appears when you differentiate (e.g. the $\frac{d}{dt}$ in the Schrodinger equation) and whenever you expand a wavefunction as an infinite series of eigenvectors of some observable.
This is a much deeper question. At the deepest level, a physical theory is nothing more or less than a mechanism for assigning probabilities to the possible outcomes of measurements. The standard formulation of quantum mechanics accomplishes this in a somewhat peculiar way, wherein we devise a correspondence between measurable properties of the system and linear maps on some Hilbert space (and then proceed as you have learned).
This approach works, as has been shown by many thousands of experiments over the past hundred years, but it's far from obvious why this is the right path to take. Some insight may be obtained from the algebraic formulation of quantum mechanics, wherein the central object under consideration is the so-called algebra of observables, whose elements represent the various measurable properties of a given system.
On one hand, this is very nice - we're working and referring directly to things we intend to measure, and in many cases it's even possible to obtain this quantum algebra of observables by suitably fiddling with a corresponding classical algebra of observables (though I should say, the latter is a different variety of algebra). The downside is that this formulation of quantum mechanics is very abstract and very sophisticated - so much so that I would be willing to bet that the substantial majority of working physicists are at best only tangentially aware of its existence.
Luckily, for those of us who are not interested in studying $C^*$-algebras until our eyes bleed, there is an alternative to this heavy mathematical abstraction. According to the Gelfand-Naimark theorem, any such algebra of observables can be concretely realized as operators on some Hilbert space. In that way we are led back to the standard formulation of quantum mechanics, but with a fresh perspective: the seemingly arbitrary choice to model a quantum system around a Hilbert space is a choice born not of necessity but rather of convenience, because it provides a concrete realization of what would otherwise be a terribly abstract description of nature.