Here's another example.
First, imagine a short cylinder $S^1\times [0,1]$ and a long cylinder $S^1\times [0,10^{10}]$, but with the same radii. Smoothly cap off both ends of the cylinders in the same way using spaces homeomorphic to discs.
The resulting manifolds are both homeomorphic to $S^2$, are not isometric (since one has a much larger diameter than the other), but there is a curvature preserving diffeomorphism between them.
To see this, just convince yourself there is a diffeomorphism $f:S^1\times[0,1]\rightarrow S^1\times [0,10^{10}]$ with the property that $f$ is an isometry when restricted to $[0,\frac{1}{4}]$ and $[\frac{3}{4},1]$. This condition allows you to extend $f$ to a curvature preserving diffeo of both compact manifolds.
You do not need a Riemannian structure on $M$. However, you have to assume that $M$ is connected (if it has more than one component, you will find a non-identity deck transformation which this is the identity on all but one component).
$\tilde{M}$ can be defined as in Understanding the orientable double cover. $\pi : \tilde{M} \to M$ is a two-sheeted covering projection with $\tilde{M}$ connected. It has exactly two deck transformations: The identity and the map $r : \tilde{M} \to \tilde{M}, r(p,o) = (p,-o)$, where $-o$ denotes the reverse orientation for $o$. For each $(p,o) \in \tilde{M}$ we get an isomorphism $d_{(p,o)}\pi : T_{(p,o)}\tilde{M} \to T_pM$, and an orientation of $\tilde{M}$ is given by taking on $T_{(p,o)}\tilde{M}$ the unique orientation $\omega_{(p,o)}$ such $d_{(p,o)}\pi (\omega_{(p,o)}) = o$. We have
$$d_{(p,o)} \pi = d_{(p,o)} (\pi \circ r) = d_{r(p,o)} \pi \circ d_{(p,o)}r = d_{(p,-o)} \pi \circ d_{(p,o)}r,$$
hence
$$o= d_{(p,o)} \pi (\omega_{(p,o)}) = d_{(p,-o)}(d_{(p,o)}r(\omega_{(p,o)})).$$
Since $d_{(p,-o)} \pi (\omega_{(p,-o)}) = -o$, we must have $d_{(p,o)}r(\omega_{(p,o)}) = -\omega_{(p,-o)} = -\omega_{r(p,o)} $. This means that $r$ reverses the orientation of $\tilde{M}$.
Best Answer
For your first question: I do not know about "physical", but I know the following:
Theorem. Let $M$ be a finite volume noncompact hyperbolic manifold (i.e. a complete connected manifold of constant negative curvature). Then $M$ admits a metric of nonpositive curvature of bounded geometry which is complete and asymptotically flat.
See
B.Leeb, 3-manifolds with(out) metrics of nonpositive curvature, Inventiones Math., 1995.
Manifolds like this exist in all dimensions, for instance, every noncompact connected surface with finitely generated fundamental group and of negative Euler characteristic. (For instance, remove three points from the 2-sphere.)
As for your second question: Just take the Euclidean space. Or, if this is too simple, take one of the examples in part 1.