The distinction to be made here is that, for the quantum harmonic oscillator system, there are no unbound states, only bound states thus, there is no benefit to insisting the states have negative energy, no reason to 'subtract infinity' in order to zero the potential at infinity.
However, in systems that permit both bound and unbound states, it is reasonable to zero the potential at infinity for the same reason that we do this classically.
For example, in the classical central force problem, there is a state in which particle can 'escape to infinity' where it will have zero kinetic energy (more precisely, the kinetic energy of the particle asymptotically approaches zero). If we set the potential energy to be zero at infinity, then the total energy 'at infinity' is zero. Thus, the particle with zero total energy 'sits on the boundary' between those particles with not enough energy to 'reach' infinity and those that do.
But, for the classical harmonic oscillator potential, no particle can escape to infinity. The kinetic energy of the particle will periodically and instantaneously be zero. In this case, it is reasonable that the state where the total energy is always equal to the potential energy (the state where the kinetic energy is always zero) be the zero total energy state; all other states having positive total energy.
So the conclusion is that nothing really exists according to Quantum
Mechanics... which can't be right, surely?
That's not remotely the correct conclusion to draw. One might conclude instead that
(1) The conception of bound state must be modified in the passage from classical mechanics to quantum mechanics and
(2) the physical (normalizable) unbound states are not eigenstates of the Hamiltonian, i.e., the physical unbound states are not states of definite energy but are, instead, a distribution of energy eigenstates, e.g., a wavepacket.
Ok, I have a solution for $\delta'(x)$ based on a very crude limit. I'm going to neglect factors of $\hbar$, $m$, etc for the sake of eliminating clutter.
Let $V_\epsilon(x)=\frac{\delta(x+\epsilon)-\delta(x)}{\epsilon}$. Then $\lim_{\epsilon\rightarrow 0}V_e(x)=\delta'(x)$. We'll solve the Schrodinger equation for finite $\epsilon$ and then take the limit afterwards.
We have $\Psi''(x) = (E-V_\epsilon(x))\Psi(x)$. If we want a bound solution, we must have $E<0$. Then in the ranges $[-\infty,0),(0,\epsilon),(\epsilon,\infty]$ we must have that $\Psi$ is some exponential function. In other words,
$$
\Psi(x) =\left\{\begin{array}{ll} e^{\sqrt{-E}x} &x\in [-\infty,0)\\
Ae^{\sqrt{-E}x}+Be^{-\sqrt{-E}x} &x\in (0,\epsilon)\\
Ce^{-\sqrt{-E}x}&x\in (\epsilon,\infty]\\ \end{array}\right\}
$$
From the fact that $\Psi$ must be continuous, we can replace $B$ and $C$ in terms of $A$ to get
$$
\Psi(x) =\left\{\begin{array}{ll} e^{\sqrt{-E}x} &x\in [-\infty,0)\\
Ae^{\sqrt{-E}x}+(1-A)e^{-\sqrt{-E}x} &x\in (0,\epsilon)\\
(1-A+Ae^{2\sqrt{-E}\epsilon})e^{-\sqrt{-E}x}&x\in (\epsilon,\infty]\\ \end{array}\right\}
$$
We can also write down the derivative of $\Psi$.
$$
\Psi'(x) =\left\{\begin{array}{ll} \sqrt{-E}e^{\sqrt{-E}x} &x\in [-\infty,0)\\
A\sqrt{-E}e^{\sqrt{-E}x}+(A-1)\sqrt{-E}e^{-\sqrt{-E}x} &x\in (0,\epsilon)\\
(A-1-Ae^{2\sqrt{-E}\epsilon})\sqrt{-E}e^{-\sqrt{-E}x}&x\in (\epsilon,\infty]\\ \end{array}\right\}
$$
Using the normal method of finding boundary conditions at a $\delta$ function barrier, we have that
$$
\begin{array}{c}
\Psi'_{+}(0)-\Psi'_{-}(0) = \Psi(0) \\
\Psi'_{+}(\epsilon)-\Psi'_{-}(\epsilon) = \Psi(\epsilon)
\end{array}
$$
The first boundary condition gives us
$$
(2A-1)\sqrt{-E} = \frac{1}{\epsilon}$$
or
$$
A=\frac{1}{2\epsilon\sqrt{-E}}+\frac{1}{2}
$$
The second boundary condition gives us
$$
-2A\sqrt{-E}e^{\sqrt{-E}\epsilon} = -\frac{1}{\epsilon}[Ae^{\sqrt{-E}\epsilon}+(1-A)e^{-\sqrt{-E}\epsilon}]
$$
or
$$
A=\frac{1}{e^{2\sqrt{-E}\epsilon}(2\epsilon\sqrt{-E}-1)+1}
$$
Putting the two conditions together gives us a constraint on $E$.
$$
\frac{1}{2\epsilon\sqrt{-E}}+\frac{1}{2} = \frac{1}{e^{2\sqrt{-E}\epsilon}(2\epsilon\sqrt{-E}-1)+1}
$$
We can expand both sides in a Laurent series to first order in $\epsilon$. The left hand side is already expanded. The right hand side becomes (to first order)
$$
\frac{1}{(2\epsilon\sqrt{-E}+1)(2\epsilon\sqrt{-E}-1)+1}=\frac{1}{4\epsilon^2(-E)}
$$
The two sides of the equation are then impossible to match in the limit $\epsilon\rightarrow 0$ since they occur at different orders in $\frac{1}{\epsilon}$. Thus, in that limit, no solution for $E$ exists, and so there is no bound state.
I'm sure there's some algebra mistakes in all that mess, but that's the general idea. You could do the same algebra to look at scattering states, if you wanted. One could also apply this method to higher derivatives. For example, $\delta''(0)=\lim_{\epsilon\rightarrow 0}\frac{\delta(x+2\epsilon)-2\delta(x+\epsilon)+\delta(x)}{\epsilon^2}$. Of course, each higher derivative demand one more boundary to account for, so the problem gets correspondingly messier. But doable, in principle.
Best Answer
In regard to your second question, I believe Griffiths is stating that, in general, the second integral is zero, and $d\psi/dx$ is continuous. However, when $V(x) \propto \delta(x)$, this statement no longer stands (i.e., $\psi$ is not continuous), and we clearly see that in equation (2.128).
If $\psi'$ were continuous, the value of $\Delta\psi′\equiv\lim_{\epsilon\rightarrow0}[\psi′(\epsilon)−\psi′(−\epsilon)]$ would be equal to zero: the value of $\psi′$ coming from negative values of $x$ should be equal to $\psi′$ coming from positive values of $x$ when approaching $x=0$ (the limit of $\epsilon\rightarrow0$ represents this). If they are equal in $x=0$, then the difference is 0, and we get $\psi′$ is, in fact, continuous.
For the delta-function potential, $\Delta\psi′\neq0$ due to the most important property of the delta function: $$\int_{a+\epsilon}^{a−\epsilon}𝑓(𝑥)\delta(x−a)𝑑x=𝑓(a).$$ Considering this property, one solves the RHS integral of equation (2.127) and gets the RHS of equation (2.128). Griffiths explains using this property by stating, "Equation (2.116) yields (...)".