I thought this question was interesting; here's what some internet research has turned up:
1. When $M$ is simply connected, the convexity radius of $M$ is $\infty$ iff $M$ has no focal points.
On the one hand, if $M$ has no focal points then the argument linked to by Woodface in the comments above shows that all geodesic balls in $M$ are convex.
On the other hand, suppose $M$ has focal points. In particular there exists a geodesic $\gamma$ intersecting a geodesic $\sigma$ orthogonally at a point $p$ of $M$, such that $\sigma$ has a focal point at some $q$ along $\gamma$. The Morse Index Theorem tells us that for points $q'$ along $\gamma$ past $q$, $\gamma$ does not minimize the distance from $q'$ to $\sigma$.
That is, take some such point $q'$; let $d(p, q') = r$. The condition that $\sigma$ have a focal point between $p$ and $q'$ tells us that there are points $p_1, p_2$ close to $p$ on $\sigma$ such that $d(p_1, q') < r$ and $d(p_2, q') < r$. If $s < r$ is the larger of these two distances, we see that the geodesic ball of radius $s$ at $q'$ can't be convex.
More generally this is an argument that if $p \in M$ where $M$ is a complete Riemannian manifold, then the convexity radius $r_{\text{conv}}(p)$ at $p$ is bounded above by the focal radius $r_f(p)$ at $p$, where $r_f(p)$ is the minimum distance of a point $q$ that is a focal point of some geodesic through $p$.
2. There exist examples of manifolds with focal points but without conjugate points.
The universal covers of such manifolds will be "simple" by your definition (simply connected and without conjugate points), but, since they will also have focal points, they will have finite convexity radii. The main class of examples seem to be due to Gulliver in this paper.
Thus, the answer to your question is no: "simple" manifolds need not have infinite convexity radius.
3. This very recent paper (by Dibble) claims that even for compact manifolds, the convexity radius can be made arbitrarily small relative to the injectivity radius.
Thus Berger's conjecture that "the bound $r_{\text{conv}}(M) \geq \tfrac{1}{2} \text{inj}(M)$ should not be too difficult to prove" is quite false (since apparently the bound itself is false).
The paper (if I understand it correctly) also claims the positive result that the convexity radius is the minimum of (a) the focal radius and (b) one-quarter the length of the shortest non-trivial closed geodesic in $M$. This is a very nice result analagous to the classical result that the injectivity radius is the minimum of the conjugate radius and one-half the length of the shortest non-trivial closed geodesic. (This is due to Klingenberg. It's given, for instance, as Proposition 2.2 in Chapter 13 of do Carmo's book, and it's also cited in the above linked paper.)
Best Answer
Consider a surface of revolution such as $$ M = \{x \in \mathbb{R}^3 : x_1^2 + x_2^2 = \frac{1}{1+x_3^2} \} $$ with Riemannian metric inherited from the ambient space. For points with large $|x_3|$, the injectivity radius becomes arbitrarily small.