Let $M$ be a Riemannian manifold and $p$ be a point on $M$. Let $U$ be a normal neighborhood about $p$ (that is, the exponential map $\exp_p$ maps a neighborhood of the origin in $T_pM$ diffeomorphically onto $U$). Fix a vector $v_p\in T_pM$.
For each $q\in U$, let $v_q$ be the vector in $T_qM$ obtained by parallel transporting $v_p$ along the radial geodesic joining $p$ to $q$. So we get a map $X:U\to TU$ which takes $q$ to $v_q$. So $X$ is a vector field on $U$.
Question. Is $X$ necessarily smooth?
What I thought is that by definition of $X$, we have $\nabla_{\partial/\partial r}X=0$ at all point of $U\setminus\{p\}$, where $\partial/\partial r$ is the radial vector field (which is defined at all points of $U$ except $p$). So $X$ is a solution of a system of partial differential equations. I do not know if this guarantees that $X$ is smooth on $U\setminus\{p\}$.
Best Answer
Yes.
There's kind of a trick to it. Let $(x^i)$ be normal coordinates centered at $p$, so the radial geodesics are given by $\gamma(t) = (tx^1,\dots, tx^n)$. The differential equation satisfied by $v = v^k \partial_k$ along such a geodesic is $$ \dot v^k(t) = - \Gamma_{ij}^k(tx^1,\dots,tx^n)x^i v^j(t),\tag{$*$} $$ with initial condition $v^k(0) = a^k$ (some arbitrary constants). (I'm using the summation convention here.)
The trick is to replace the $x^i$'s by new dependent variables $w^i$, and write this as a system of ODEs for the $2n$ functions $(v^i,\dots,v^n,w^1,\dots,w^n)$: \begin{align*} \dot v^k(t) &= - \Gamma_{ij}^k(tw^1(t),\dots,tw^n(t))w^i(t) v^j(t),\\ \dot w^k(t) &= 0, \end{align*} with initial conditions \begin{align*} v^k(0) &= a^k,\\ w^k(0) &= x^k. \end{align*} Because solutions to smooth ODEs depend smoothly on initial conditions as well as time, the solutions to this system can be written as smooth functions $v^k(t,a,x)$ and $w^k(t,a,x)$. It follows immediately from the form of the equation that $w^k$ is constant, $w^k \equiv x^k$, and therefore the vector field you're interested in is $$ v(x) = v^k(1,a,x) \partial_k, $$ which depends smoothly on $x$.