I am not an expert but the question:
"Does there exist a simply-connected closed Riemannian Ricci flat $n$-manifold with $SO(n)$-holonomy?"
is a well-known open problem. Note that Schwarzschild metric is a complete Ricci flat metric on $S^2\times\mathbb R^2$ with holonomy $SO(4)$, so the issue is to produce compact examples; I personally think there should be many. The difficulty is that it is hard to solve Einstein equation on compact manifolds. If memory serves me, Berger's book "Panorama of Riemannian geometry" discusses this matter extensively.
I think it would be more accurate to say that the real reason why
Calabi-Yau, hyperkähler, $G_2$ and $\mathrm{Spin}(7)$ manifolds are of
interest in string theory is not their Ricci-flatness, but the fact
that they admit parallel spinor fields. Of course, in
positive-definite signature, existence of parallel spinor fields
implies Ricci-flatness, but the converse is still open for compact
riemannian manifolds, as discussed in this
question, and known to fail for noncompact manifolds as pointed
out in an answer to that question.
The similar question for lorentzian manifolds has a bit of history.
First of all, the holonomy principle states that a spin manifold
admits parallel spinor fields if and only if (the spin lift of) its
holonomy group is contained in the stabilizer subgroup of a nonzero
spinor. Some low-dimensional (i.e., $\leq 11$, the cases relevant to
string and M-theories) investigations (by Robert Bryant and myself,
independently) suggested that these subgroups are either of two types:
subgroups $G < \mathrm{Spin}(n) < \mathrm{Spin}(1,n)$, whence $G$ is
the ones corresponding to the cases 5-8 in the question, or else $G =
H \ltimes \mathbb{R}^{n-1}$, where $H < \mathrm{Spin}(n-1)$ is one of
the groups in cases 5-8 in the question. Thomas Leistner showed that
this persisted in the general case and, as Igor pointed out in his
answer, arrived at a classification of possible lorentzian holonomy
groups. Anton Galaev then constructed metrics with all the possible
holonomy groups, showing that they all arise. Their work is reviewed
in their
paper (MR2436228).
The basic difficulty in the indefinite-signature case is that the de
Rham decomposition theorem is modified. Recall that the de Rham
decomposition theorem states that if $(M,g)$ is a complete, connected
and simply connected positive-definite riemannian manifold and if the
holonomy group acts reducibly, then the manifold is a riemannian
product, whence it is enough to restrict to irreducible holonomy
representations. This is by no means a trivial problem, but is
tractable.
In contrast, in the indefinite signature situation, there is a
modification of this theorem due to Wu, which says that it is not
enough for the holonomy representation to be reducible, it has to be
nondegenerately reducible. This means that it is fully reducible
and the direct sums in the decomposition are orthogonal with respect
to the metric. This means that it is therefore not enough to restrict
oneself to irreducible holonomy representations. For example,
Bérard-Bergery and Ikemakhen proved that the only lorentzian holonomy
group acting irreducibly is $\mathrm{SO}_0(1,n)$ itself: namely, the
generic holonomy group.
It should be pointed out that in indefinite signature, the
integrability condition for the existence of parallel spinor fields is
not Ricci-flatness. Instead, it's that the image of the Ricci
operator $S: TM \to TM$, defined by $g(S(X),Y) = r(X,Y)$, with $r$ the
Ricci curvature, be isotropic. Hence if one is interested in
supersymmetric solutions of supergravity theories (without fluxes) one
is interested in Ricci-flat lorentzian manifolds (of the relevant
dimension) admitting parallel spinor fields. It is now not enough to
reduce the holonomy to the isotropy of a spinor, but the
Ricci-flatness equation must be imposed additionally.
Best Answer
This is a somewhat different take on Igor's answer, and I offer it just in case you are interested.
First, one doesn't need to have any continuous symmetries in order to have this kind of 'Wick rotation' exist. For example, if $(M,g)$ is a real-analytic Riemannian manifold that admits a nontrivial isometric involution $\iota:M\to M$ that fixes a hypersurface $H\subset M$, then, near $H$, one can write the metric in the form $g = dt^2 + h(t^2)$ where $t$ is the distance from the hypersurface and $h(a)$ is the induced metric on the level sets $t^2 = a$. Then the Wick rotation is just $g' = -d\tau^2 + h(-\tau^2)$ in the sense that this is the Lorentian metric induced on the slice $t = i\tau$ in the complexification $(M^\mathbb{C},g^\mathbb{C})$.
Second, one could generalize things a bit and say that two real-analytic (pseudo-)Riemannian metrics are 'Wick-related' if they are '$\mathbb{R}$-slices' of a common connected holomorphic Riemannian complex $n$-manifold $(M^\mathbb{C},g^\mathbb{C})$. By an '$\mathbb{R}$-slice', I mean a real $n$-manifold $N\subset M^\mathbb{C}$ such that the pullback of $g^\mathbb{C}$ to $N$ is real-valued and nondegenerate. In this terminology, I think that two $\mathbb{R}$-slices $N_1,N_2\subset M^\mathbb{C}$ should be said to be related by a 'Wick rotation' if $N_1\cap N_2$ is a submanifold of dimension $n{-}1$. This is certainly what happens in the case above generated by an isometric involution fixing a hypersurface. (Added note: In fact, I don't know an example of a Wick-rotation in this sense that isn't generated by such an isometric involution. It might be interesting to try to prove that this does give them all or else find a counterexample. I note that, to second order, it is true: The hypersurface $N_1\cap N_2$ is always totally geodesic in each of $N_1$ and $N_2$ (with their induced metrics), so reflection in the hypersurface is an isometry at least up to second order.)
Then the problem becomes how to tell when a given connected holomorphic Riemannian complex $n$-manifold $(M^\mathbb{C},g^\mathbb{C})$ has an $\mathbb{R}$-slice (and, of course, to determine them when they exist). Obviously, the complexification of a real-analytic pseudo-Riemannian $n$-manifold has at least one $\mathbb{R}$-slice, but generically, when $n>1$, this is the only one, so 'most' real-analytic pseudo-Riemannian manifolds are not Wick-related to any other, let alone possess a Wick-rotation.
The reason is that $\mathbb{R}$-slices are the integral manifolds of a very restrictive system of PDE for real submanifolds of $(M^\mathbb{C},g^\mathbb{C})$: If one lets $R\subset \mathrm{Gr}^\mathbb{R}_n(TM^\mathbb{C})$ denote the set of real $n$-planes $E\subset T_pM^\mathbb{C}$ to which $g^\mathbb{C}$ restricts to be real-valued (and, of course, nondegenerate), then $R$ is a smooth manifold of dimension $2n+\tfrac12n(n{-}1)$ with $n{+}1$ components (one for each possible index of $g$ when restricted to the $n$-plane $E$), and the basepoint projection $\pi:R\to M^\mathbb{C}$ given by $\pi(E) = p$ is a smooth submersion. It is easy to show that, for any given $E\in R$, there is at most one $\mathbb{R}$-slice that has $E$ as a tangent space. This is because there is a canonical $n$-plane field $H$ on $R$ with the property that the set of tangent spaces of any $\mathbb{R}$-slice is an $n$-manifold in $R$ that is tangent to $H$ everywhere.
The only time $H$ is Frobenius is when $(M^\mathbb{C},g^\mathbb{C})$ is the complexification of a (real) space form, i.e., a Riemannian manifold of constant sectional curvature. Generically, $H$ has no integral manifolds at all, and, generically, when it does have one, it has only one connected component.
For example, in the case $n=2$, if the Gauss curvature $K$ of $(M^\mathbb{C},g^\mathbb{C})$ is not constant, then it has at most a $1$-parameter family of positive definite $\mathbb{R}$-slices, and this happens only when the $\mathbb{R}$-slices are all isometric and have a symmetry vector field (so they are locally surfaces of revolution).
Now, in fact, each $\mathbb{R}$-slice (of whatever index) lies in the (real) hypersurface in $M^\mathbb{C}$ on which $K$ takes values in $\mathbb{R}$. Set $E = g^\mathbb{C}(\nabla K,\nabla K)$ (everything computed in the holomorphic category). For most metrics $g^\mathbb{C}$, the holomorphic functions $K$ and $E$ will not be functionally dependent, i.e., $\mathrm{d}K\wedge\mathrm{d}E$ will vanish only on a (possibly empty) complex-analytic subvariety $C\subset M^\mathbb{C}$ of complex dimension $1$. Go ahead and define $F = g^\mathbb{C}(\nabla K,\nabla E)$ and $G = g^\mathbb{C}(\nabla E,\nabla E)$.
Obviously, any $\mathbb{R}$-slice $N\subset M^\mathbb{C}$ must lie inside the locus on which the holomorphic functions $K$, $E$, $F$, and $G$ assume real values. Outside of $C$, the set $L\subset M^\mathbb{C}\setminus C$ on which $K$ and $E$ take real values is a (possibly empty) real submanifold of dimension $2$ and its components are the only possible $\mathbb{R}$-slices that lie outside of $C$ (of course, any $\mathbb{R}$-slice can only intersect $C$ in a (real) $1$-dimensional curve at most). A component of $L$ actually is an $\mathbb{R}$-slice if and only if $F$ and $G$ are real-valued on it. Two components are related by a Wick-rotation in the above sense if they intersect (in a real $1$-dimensional curve that necessarily lies inside $C$).
A similar, but slightly more involved analysis can be done for the case in which $K$ and $E$ are dependent everywhere on $M^\mathbb{C}$. Moreover, a similar, but more complicated analysis can be used in higher dimensions, with, say, the symmetric functions of the eigenvalues of the holomorphic Ricci tensor used in the place of $K$.