This question has been asked and answered on MathOverflow. I have replicated the accepted answer by J. GE below.
To understand the realationship between $Q$ and $P$, it suffices to study how they act on an orthornormal(o.n.) basis. Here is the detailed argument:
Let $J_i(t)$ be the Jacobi field along $c$, with $J_i(0)=0$ and $J_i'(0)=e_i$, where $e_1, \cdots, e_{n-1}, X$ is an o.n basis of $T_p(M)$. Then one can show that $J_i(t)=Q_t(e_i)$,
since
$$
Q_t(e_i)=dexp_{tX} (te_i).
$$
Now extend $e_i$ to $E_i(t)$ along $c$ via parallel transport $P$. One can write the $J_i$ (i.e. $Q_t$) in terms of the o.n. basis $E_i(t)$:
$$
Q_t(e_i)=J_i(t)=\sum_j a_{ij}E_j(t)=\sum_j a_{ij}P_t(e_j).
$$
Let $A=(a_{ij})$. Using Jacobi equation, one can show that
$$
A''(t)+R(t)A(t)=0,
$$
where $R(t)=(R_{ij}(t))$ is the curvature term defined by $R_{ij}(t)=\langle R(\dot{c}, E_i) \dot{c}, E_i \rangle$.
This is essentially Jacobi equation. So roughly speaking, $Q$ and $P$ satisfy the Jacobi equation.
remark: It seems my definition of $Q$ differs by a rescaling fact $t$. So if we use $N$ to denote your map, then $Q_t(e_i)= N(t e_i)$.
Choosing a left-invariant metric does indeed give you many one-parameter subgroups of isometries from which you can pick out Killing fields. However, these Killing fields will be right-invariant vector fields on $G$ (i.e. $X(g) = dR_g(X(e))$) and in general, these are not left-invariant. You can find a one-parameter subgroup of isometries by defining $\phi_s(g) = L_{exp(sv)}(g)$ for $s$ sufficiently small and $g \in G$ for some fixed $v \in \frak{g}.$ The vector field associated to this flow, $X$, can be found by computing
\begin{align} X(g) = X (\phi_0(g)) &= \frac{d}{ds}\mid_{s=0} exp(sv)\cdot g \\
&= \frac{d}{ds}\mid_{s=0} R_g (exp(sv))\\
&= dR_g( X(e)) \end{align}
So, left-invariant vectors are not necessarily Killing fields, and as you stated, one-parameter subgroups are not geodesics. The best way to see this is to take your favorite Lie group, $G$, and give it some left-invariant metric, $\langle \cdot, \cdot \rangle$ and study the one-parameter subgroups and left-invariant vector fields. Note, a left-invariant metric is a name for Riemannian metric you described above (i.e. choose an inner product $\langle \cdot, \cdot \rangle_{e_G}$ and use you left translations $L_g: G \rightarrow G$ that send $h\in G \mapsto gh \in G$ to define that inner product at each other tangent space of your manifold). I should also remark that even though all inner-product spaces are isometric, left-invariant metrics do not produce isometric Riemannian manifolds. The choice of initial inner product can dramatically alter the geometry.
My favorite example of a Lie group is the 3-dimensional $SL(2,\mathbf{R})$. This has a Lie algebra $\mathfrak{sl}(2,\mathbf{R})$ which has a basis $B =\left \{e,f,h \right \}$ with bracket relations:
\begin{equation} [h,e]=2e, \quad [h,f]=-2f, \quad [e,f]=h\end{equation}
Let's just choose an inner product on $\frak{sl}(2,\mathbf{R})$ so that $B$ is an orthonormal basis. The left-invariant vector field $X_e$ corresponding to $e$ is not a Killing field for this metric.
Claim: $X_e$ is not Killing.
Proof: A vector field $X$ is Killing iff $X$ sastifies the Killing equation
\begin{equation} \langle \nabla_Y X ,Z \rangle + \langle \nabla_Z X, Y \rangle =0 \quad \quad \text{for all} \quad Y,Z \in \mathfrak{X}(M) \end{equation}
(see: page 82 of Riemannian Geometry by do Carmo)
Well, if this is true, then that equation holds if $Y,Z$ are also left-invariant vector fields on $SL(2,\mathbf{R})$ (note, not every smooth vector field is left-invariant). Let's take $Y = X_h$ and $Z=X_f$ i.e. the left-invariant vector fields corresponding to the Lie algebra elements $h$ and $f$. One can compute that
\begin{equation} \langle \nabla_{X_h} X_e ,X_f \rangle + \langle \nabla_{X_f} X_e, X_h \rangle = \langle h, h \rangle_0 = 1 \end{equation}
where $\langle \cdot, \cdot \rangle_0$ is the inner product on $\frak{sl}(2,\mathbf{R}).$ $\Box$
Now, let's study one-parameter subgroups. Let $v \in \frak{sl}(2,\mathbf{R})$ and consider the one-parameter subgroup associated to $e$ i.e. $f_e(t) = exp(t e)$ for sufficiently small $t$.
Claim: $f_e(t)$ is not geodesic
Proof: Suppose by way of contradiction that this is a geodesic. Then, $\nabla_{X_e} X_e\equiv 0$ along the points on that curve. However, using the fact that we have a left-invariant metric, we can compute for all $v \in \frak{sl}(2,\mathbf{R})$,
\begin{equation} \langle X_v , \nabla_{X_e} X_e \rangle = \langle e, [v, e] \rangle_0 =0 \end{equation}
where $X_v$ is associated left-invariant vector field to $v$. This is clearly false for $v=h$. In general, that inner product vanishing for all $v$ is equivalent to the adjoint map of $ad_e: v \mapsto [e,v] $ is so that $ad^t_e(e) = 0$ where $(\cdot)^t$ denotes the transpose with respect to $\langle \cdot, \cdot \rangle_0$. If you write out the matrix associated to $ad_e$ written with respect to the orthonormal basis $B$, you can see this explicitly. $\Box$
Unfortunately, invariant Lie group geometry is not as simple as one might wish. However, there is a special class of Lie groups that do admit left-invariant metrics with geodesics corresponding to one-parameter subgroups and all left-invariant vector fields are Killing fields. These are Lie groups that admit a bi-invariant metric i.e. the metric can be equivalently defined using left-translations or right-translations. Any Abelian Lie group certainly have bi-invariant metrics. If one knows about the Killing form of a Lie algebra, you can see that compact semi-simple Lie groups also admit bi-invariant metrics. It turns out these are the only Lie groups that admit such metrics (Curvatures of left invariant metrics on lie groups, John Milnor, 1976--very readable!). You might look at exercise 3, Chapter 3 of do Carmo's book to see why bi-invariant metrics have these properties.
In light of these facts/computations, you might think of left-invariant metrics that don't have these properties as somehow having left translations and right translations that in some way do not mesh well geometrically.
Best Answer
There are two separate notions of exponential map: (1) a riemannian-geometric notion, for which your first item holds; and (2) a Lie group-theoretic notion, for which your second and third items hold.
In certain special situations these two coincide, for example a biinvariant metric on a compact Lie group.
A good reference for this is an appendix in Buser and Karcher:
Buser, Peter; Karcher, Hermann Gromov's almost flat manifolds. Astérisque, 81. Société Mathématique de France, Paris, 1981. 148 pp.