Yes, $H^{n-1}(M-\{p\};\mathbb{Q}) \neq 0$ for any closed non-orientable manifold of dimension $n$. Long-story-short it's because $H^n(M)=0$ in this case so $H^{n-1}(M-\{p\})$ surjects onto the coefficient group.
Just for exposition I wanted to spell out the arguments to a few of your assertions. Consider the long exact sequence of the pair:
$$\dots \to H^{k}(M, M-\{p\}) \to H^k(M) \to H^k(M-\{ p\})$$$$ \to H^{k+1}(M, M-\{p\}) \to H^{k+1}(M)\to \dots $$
This sequence exists for all coefficients $R$, in particular $\mathbb{Q}$.
Since $M$ is a manifold of dimension $n$, the group $H^k(M, M-\{p\})$ is $R$ if $k = n$ and $0$ otherwise. It follows that
$$H^k(M) \cong H^k(M-\{p\})\text{ if $k < n-1$}$$
and there is an exact sequence
$$0 \to H^{n-1}(M) \to H^{n-1}(M-\{ p\}) \to R \to H^n(M) \to 0$$
where $H^n(M-\{p\}) = 0$ because it is an open manifold.
If $M$ is orientable then the last map is an isomorphism so $H^{n-1}(M) \cong H^{n-1}(M-\{p\})$. Otherwise $M$ is non-orientable, so if $R$ is such that $H^n(M) =0$ (for example $R=\mathbb{Z}, \mathbb{Q}$) then you get a short exact sequence with $H^{n-1}(M-\{p\})$ in the middle: if $R=\mathbb{Q}$ that means $H^{n-1}(M-\{p\})$ is a rational vector space of dimension $dim(H^{n-1}(M)) +1>0$. In fact as long as $H^n(M)=0$ and $R\neq 0$ we must have $H^{n-1}(M-\{p\};R)\neq 0$ because it surjects onto $R$.
One particularly concrete way to get at this issue is to fix real numbers $0 < r < R$, and to parametrize the solid torus of major radius $R$ and minor radius $r$ by
$$
X(\rho, \theta, \phi) = ((R + \rho\cos\theta)\cos\phi, (R + \rho\cos\theta)\sin\phi, \rho\sin\theta)
$$
for $0 \leq \rho \leq r$ and $0 \leq \theta, \phi \leq 2\pi$. (Geometrically, use polar coordinates $(R + r\cos\theta, r\sin\theta)$ to parametrize the disk of radius $r$ centered at $(R, 0)$, then revolve about the $z$-axis. Naturally, this map is defined on all of three-space.)
If $n$ is an integer, the image of
$$
x(\rho, \phi) = X(\rho, \tfrac{1}{2}n\phi, \phi),\qquad
-r \leq \rho \leq r,\ 0 \leq \phi \leq 2\pi,
$$
is a strip with $n$ half-twists. (Geometrically, we're taking the diameter from $(R - r, 0, 0)$ to $(R + r, 0, 0)$ in the longitudinal half-plane where $\phi = 0$ and revolving it $\frac{n}{2}$ times about the center of the disk (in the revolving longitudinal plane) as the disk revolves once about the $z$-axis.)
The upward-pointing unit vector $(0, 1, 0)$ at the center of the disk extends locally to a unit normal field to the strip.
If $n$ is even, the diameter rotates through a whole number of turns; the unit normal vector returns to its initial direction $(0, 1, 0)$, and the strip is orientable. The are two components to the complement of the zero section because each radius "joins back to itself". Each endpoint $(R \pm r, 0, 0)$ traces an $(n, 1)$ torus knot.
If $n$ is odd, the diameter rotates through a half-integer number of turns; the unit normal vector returns to the opposite direction $(0, -1, 0)$, and (mod details) the strip is consequently non-orientable. There is one component to the complement of the zero section because each radius "joins back to the other". The endpoints $(R \pm r, 0, 0)$ together trace a single $(n, 2)$ torus knot. (Equivalently, consider one endpoint and run $\phi$ two full turns, from $0$ to $4\pi$.)
In summary:
The trefoil is a $(3, 2)$ torus knot; generally, we get torus knots from the boundary curves of the strips, though there is "extra" twisting of the half-strips, loosely because we're seeing twisting around the central curves of the cut strip(s).
If we replace $n$ by $-n$, we're equivalently post-composing $x$ with the reflection $(x, y, z) \mapsto (x, y, -z)$, so your mirror-image conjecture is correct.
Best Answer
You can search on the web for images of the cross-cap; it is a model in 3 dimenions, with self intersections, of the projective plane.
The projective plane can also be seen as a Mobius band with a disc glued onto the boundary; but this cannot be dome in 3 dimensions. A representation of this gluing and its relation to rotations in 3 dimensions is shown in part of this presentation Out of Line.