I had the same question, when studying the subject. Let me tell you, what I was told - it relates to the functional calculus:
Recall that in quantum mechanics, as we usually learn it, a measurement is a projective measurement, i.e. the outcomes of a measured observables are eigenvalues of the observable and we "update" the state according to the knowledge obtained (i.e. we project it into the Hilbert space). We can of course use the whole formalism of POVMs instead if you know about this, but still, projective measurements remain important special cases. For this reason, we don't actually need our Hermitian observables, but we need the spectral calculus and the spectral theorem. You want all spectral projections of an observable to belong to your space, since if you measure the observable, your results will be updated according to the eigenprojections. And here is the problem: C*-algebras in generally do not contain all their projections, von Neumann algebras do. So the "physical consequence" is that you actually have all your measureable quantities inside the algebra of operators you call "observables". I believe that is as physical as it gets. Since von Neumann algebras can always be seen as closures of C*-algebras in some topology, I would not expect there to be a much deeper reasons, although I'd love to know them myself, if there are.
Other reasons mentioned to me refer to the structure of von Neumann algebras (and its lattice of projections) and how this enters different scenarios in physics, but in this case, I would say that the reason to study von Neumann algebras is rather technical than physical.
Finally, let me point out that it is not a priori clear why we should study C*-algebras at all - I mean, the only physical quantities are the Hermitian operators, but generically, our algebras will contain many nonhermitian elements. In my view, this means there is no reason to study either C*-algebras or von Neumann algebras, but one would actually have to study Jordan algebras (the set of Hermitian elements of the bounded operators on some Hilbert space forms such a Jordan algebra, or more precisely, a Jordan operator algebra). Since these algebras are however nonassociative (which is inconvenient) and can nearly always be embedded into some associative algebra, we study the associative algebras. So, in a sense, studying C*-algebras is already "a technical thing".
If $\cal H$ is a complex Hilbert space, and $A :D(A) \to \cal H$ is linear with $D(A)\subset \cal H$ dense subspace, there is a unique operator, the adjoint $A^\dagger$ of $A$ satisfying (this is its definition)
$$\langle A^\dagger \psi| \phi \rangle = \langle \psi | A \phi \rangle\quad
\forall \phi \in D(A)\:,\forall \psi \in D(A^\dagger)$$
with:
$$D(A^\dagger) := \{ \phi \in {\cal H}\:|\: \exists \phi_1 \in {\cal H} \mbox{ with} \: \langle \phi_1 |\psi \rangle = \langle \phi | A \psi\rangle \:\: \forall \psi \in D(A)\}$$
The above densely-defined operator $A$ is said to be self-adjoint if $A= A^\dagger$. A densely-defined operator satisfying
$$\langle A \psi| \phi \rangle = \langle \psi | A \phi \rangle\quad \forall \psi,\phi \in D(A)$$
is said to be symmetric. It is clear that the adjoint of $A$, in this case, is an extension of $A$ itself.
A symmetric operator is essentially selfadjoint if $A^\dagger$ is self-adjoint. It is possible to prove that it is equivalent to say that $A$ admits a unique self-adjoint extension (given by $(A^\dagger)^\dagger$).
The operator $-i\frac{d}{dx}$ with domain given by Schwartz' space ${\cal S}(\mathbb R)$ (but everything follows holds also if the initial domain is $C_0^\infty(\mathbb R)$) is symmetric and essentially self-adjoint. Both $-i\frac{d}{dx}$ and the true momentum operator $p:= \left(-i\frac{d}{dx}\right)^\dagger$ are not bounded.
Both operators do not admit eigenvalues and eigenvectors.
The spectrum of $p$ is continuous and coincides with the whole real line.
Passing to Fourier-Plancherel transform, the operator $p:= \left(-i\frac{d}{dx}\right)^\dagger$ turns out to coincide with the multiplicative operator $k \cdot$.
Concerning the issue of unboundedness of most self-adjoint quantum operators, the point is that a celebrated theorem (one of the possible versions of Hellinger–Toeplitz theorem) establishes that:
a (densely-defined) self-adjoint operator
$A :D(A) \to \cal H$ is bounded if and only if $D(A)= \cal H$
and almost all
operators of QM, for various reasons, are not defined in the whole Hilbert space (unless the space is finite dimensional). These operators are not initially defined on the whole Hilbert space because they usually are differential operators. Differential operators need some degree of regularity to be applied on a function, whereas the generic element of a $L^2$ space is incredibly non-regular (it is defined up to zero-measure sets). The subsequent extension to self-adjoint operators exploits a weaker notion of derivative (weak derivative in the sense of Sobolev) but the so-obtained larger domain is however very small with respect to the whole $L^2$ space.
ADDENDUM. In view of a remarkable Andreas Blass' comment, I think it is worth stressing a further physical reason for unboundedness of some self-adjoint operators representing observables in QM.
First of all the spectrum $\sigma(A)$ of a self-adjoint observable represented by a self-adjoint operator $A$ has the physical meaning of the set of all possible values of the observable. So if the observable takes an unbounded set of values, the spectrum $\sigma(A)$ must be an unbounded subset of $\mathbb R$.
Secondly, if $A$ is a self-adjoint operator (more generally a normal operator), the important result holds true:
$$||A|| = \sup\{ |\lambda| \:|\: \lambda \in \sigma(A)\}$$
including the unbounded cases where both sides are $+\infty$ simultaneously.
So, if an observable, like $p$ or $x$ or the angular momentum, takes an unbounded set of values, the self-adjoint representing it has necessarily to be unbounded (and therefore defined in a proper dense subspace of the Hilbert space).
Best Answer
It doesn't really mean anything bad at all, in spite of the confused answers one sometimes receives about this matter. The big point is: there is no very direct physical meaning to applying an operator $H$ or $Q$, even if it is an observable, to a wave function $\psi$, even if $\psi$ is in the domain of $H$ or $Q$. (See also Operator versus Linear Transformation, https://physics.stackexchange.com/a/18933/6432 .
In my opinion, the only operator with very direct physical meaning is the time evolution operator $e^{iHt}$ for some $t$, or for all $t$, and as you seem to realise, by the Stone-von Neumann theorem, this exists even for discontinuous $\psi$ even for unbounded $H$.
But, also this is my opinion, the exact definition of Hilbert space is not that important and many mathematical physicists who worry (too much) about whether discontinuous wave functions, which are obviously outside the domain of $H$, are physical or not, manage to formulate Quantum Mechanics just fine on a dense subspace of a Hilbert space. Other, with the opposite worry, again, in my opinion, worrying too much about something which isn't really that bad, talk about rigged Hilbert spaces or nuclear spaces in order to somehow include infinite norm states and exclude non-differentiable wave functions....see Sudbery, Quantum Mechanics and the Particles of Nature. for this.
There is a mathematical reason for thinking that the exact choice of what domain or what space, whether the Hilbert space, or a subspace of smooth wavefunctions inside the Hilbert Space, or the Schwartz space of rapidly decreasing and smooth functions, or an extension of the Hilbert space to include some dual objects such as distributions, to think of as the domain of these operators, is....unimportant because no matter what space you choose, you get the same physics, and that reason is a theorem of Wilfrid Schmid, Henryk Hecht, and Dragan Milicic, or at least somebody or other, which says that if you have a semi-simple Lie group operating on the space, (if the QM is going to made relativistic you eventually have to assume the Lorentz group acts) and if the representation has a finite composition series (this excludes quantum fields), then the algebraic structure of that rep. is independent of which space you consider (within broad limits). Earlier versions of such results were proved by Nelson, Garding, and Harish-Chandra, and gave a very pleasant surprise to Hermann Weyl and everyone else involved at the time...
Now very concretely, even if $\psi$ is not in the domain of $H$, or any other observable $Q$, it is still true that the Hilbert space has a Hilbert basis of eigenstates of $H$ and hence even if $\psi$ is horrible and discontinuous and everything bad, it still holds that
$$\psi = \lim_{n\rightarrow \infty}\sum_{i=0}^{i=n} c_i v_i$$
where $v_i$ is the normalised eigenstate of $H$ (or $Q$) with eigenvalue, say, $\lambda_i$, and $c_i$ are complex numbers, the so-called Fourier--Bessel coefficients of $\psi$, and the convergence is not pointwise but in the L$^2$ norm. Now notice: each finite sum $$\sum_{i=0}^{i=n} c_i v_i$$ is an analytic function, if $H$ is hypo-elliptic, as is often true, e.g., the harmonic oscillator, and is at any rate smooth and in the domain of $H$.
And it is still true, by the axioms of QM, that the probability that $H$ (or $Q$) will, if measured, take the value of $\lambda_i$, is $\vert c_i \vert ^2$ whether or not $\psi$ is in the domain of $H$ (or $Q$).
Pedagogically, there is this widespread confusion that an observable, since it is an operator, ought to be applied to a function since it is an operator, but this is just a naive confusion. If anything should be applied to the wave function as an operator, it is the exponential of $iH$, which is always bounded. To repeat: just because $H$ is an operator, and $\psi$ is a function, doesn't mean you should apply $H$ to $\psi$. Although when you can, that may be a useful shortcut, it is not necessary to ever do it, and the axioms of QM, when stated carefully, never ask you to apply $H$ to $\psi$. What they ask you to do is, for the unitary time evolution to apply the exponential of $iH$, and for the Born rule probabilities, expand $\psi$ to get its Fourier--Bessel coefficients. The sloppy way of thinking, which one often sees, works fine for many simple QM problems, but leads to people asking precisely this OP, precisely since it is sloppy. The careful axiomatisation states things the way I formulated them.