Regarding the question:
Let $(M,g)$ be a complete Riemannian manifold and $p\in M$. Suppose $\exp_p$ is nonsingular everywhere on $B(0,R)\subset T_pM$. Does any curve on $M$ starting from $p$ with length $<R$ lift to a curve on $T_pM$ starting at $0$?
The answer to this in general is negative: Some curves do lift but some do not. However, the statement in your question is not what do Carmo is claiming. He only claims that if $\gamma: [0,a]\to M$ is a geodesic with $\gamma(0)=p$ then for all sufficiently small $T>0$ the restriction $\gamma|_{[0,T]}$ lifts to $T_pM$ via the exponential map to a radial line segment. This statement is obvious since $\exp_p$ is a local diffeomorphism at $0\in T_pM$.
Edit. I indeed misread the question. Here is the correct answer. One needs to assume that the curve $\alpha_0$ lifts. The map $\exp_p$ is a local diffeomorphism on $B(0, \pi/\sqrt{K_0})$. The same argument as in the proof of the covering homotopy theorem yields:
Lemma. Suppose that $f: X\to Y$ is a local homeomorphism between manifolds, $H:[0,1]^2\to Y$ is a continuous map and the curve $H(s,0), s\in [0,1]$ lifts to a curve $\tilde\alpha_0(s)=\tilde{H}(s,0)$ in $X$ via $f$. Then there exists $\epsilon>0$ such that the restriction of $H$ to $[0,1]\times [0,\epsilon]$ lifts to a map $\tilde{H}:[0,1]\times [0,\epsilon]\to X$ via $f$, whose restriction to $[0,1]\times\{0\}$ is the curve $\tilde\alpha_0$.
This lemma yields the claim made by do Carmo.
Best Answer
Take a sequence of points $(x_i)$ in the manifold whose distance from $p$ tends to infinity, and connect each of them to $p$ by a minimizing geodesic $\gamma_i(s)$. Choose a convergent subsequence $\gamma'_{i_k}(0)$ at $p$. Then the limit of the sequence is the desired direction of a ray.