Yes, this is true. First, orient your $n$-manifold $M$ (your hypotheses imply that $M$ is contractible, so this is possible).
First, by your hypothesis, you obtain an increasing exhaustion $M_k \subset M$ of compact sets, diffeomorphic to the $n$-ball, so that each $M_k$ is contained in the interior of $M_{k+1}$.
This is enough; here is the idea. Write $B(k)$ for the unit ball of radius $k$ in $\Bbb R^n$. We may construct, for each $k$, some oriented diffeomorphism $\phi_k: M_k \to B(k)$. If we had $\phi_k \big|_{M_{k-1}} = \phi_{k-1}$, then by taking the increasing union of these $\phi_k$, we define a bijection $M \to \Bbb R^n$ which is a diffeomorphism on the interior of any compact set, and hence is a global diffeomorphism.
In practice, each successive $\phi_k$ has nothing to do with the previous one. Here is how we will resolve this.
Consider the map $g_k: \phi_{k+1}\phi^{-1}_k: B(k) \to B(k+1)$. All we know about this map is that it is a smooth oriented embedding into the interior.
Lemma: There is only one oriented embedding of the $n$-disc into any oriented smooth open $n$-manifold $M$, up to isotopy.
This is a lemma of Cerf and Palais, independently; see here. The idea is to take any smooth embedding to the linear embedding in a chart given by the derivative at zero. In particular, we may find an ambient isotopy $f_t: B(k+1) \to B(k+1)$ which is the identity near the boundary, so that $f_0 = \text{Id}$ and $f_t g_k$ is a smooth isotopy between $f_0 g_k = g_k$ above and $f_1 g_k$ the canonical inclusion map.
Therefore, the map $f_1 \phi_{k+1}$ restricts to $\phi_k$ on $M_k$. So we choose this to be our given diffeomorphism $M_{k+1} \to B(k+1)$. Proceeding inductively, we have our desired result.
Best Answer
In dimension 4, one has the following equivalence:
Proof. We construct maps both ways, which will be clearly inverse.
$C: \mathcal B_4 \to \mathcal S_4$. Pick $B \in \mathcal B_4$. By the resolution of the 3D Poincare conjecture, there is some oriented diffeomorphism $\varphi: \partial B \to S^3$. One may then "cap off the boundary": define $$C(B) = B \cup_\varphi B^4,$$ defined as $B \sqcup B^4$, modulo identifying $\partial B \cong S^3$ via the diffeomorphism $\varphi$.
$C$ is well-defined by Cerf's theorem that $\pi_0 \text{Diff}^+(S^3) = 1$: all diffeomorphisms are the same up to isotopy. Isotoping $\varphi$ above does not change the diffeomorphism type; so $C$ is a set map.
Conversely one has $D: \mathcal S_4 \to \mathcal B_4$, with $D(S)$ given by deleting the interior of some oriented embedding $\iota: B^4 \to S$. Oriented embeddings of balls into a connected manifold are unique up to isotopy (Palais, but straightforward); this is true in all dimensions.
Clearly $D(C(B)) = B$ (delete the ball you glued) and $C(D(S)) = S$ (glue in the ball you deleted). Therefore $C,D$ are inverse bijections.
However one has the following for $n \geq 6$.
Proof: Pick $B \in \mathcal B_n$. Delete a standard ball from its interior; then we are provided with a compact manifold $W$, with $\partial W = S^{n-1} \sqcup \partial B$, so that $W \cup_{S^{n-1}} B^n = B$.
$W$ is an h-cobordism (algebra). Therefore by the h-cobordism theorem there is a diffeomorphism $W \cong S^{n-1} \times [0,1]$, which sends $S^{n-1}$ to $S^{n-1} \times \{0\}$ by the identity map. Therefore $W \cup_{S^{n-1}} B^n \cong B^n$. Therefore $B \cong B^n$.
Of course there are no exotic n-balls n = 1,2,3. For n=5 I do not know. I think the answer is that $\partial: \mathcal B_5 \to \mathcal S_4$ is an injection (maybe bijection?) but I don't know off the top of my head. Some googling or looking on Math Overflow should help. Obviously "exotic balls" will get you nowhere but some buzzwords like diffeomorphism might.