It's because magnetic field has zero divergence combined with the symmetry of the problem.
At each point on our Ampere's loop we define three orthogonal unit vectors: $\hat{t}$ which is tangential to the loop, $\hat{r}$ which points radially outward from the center of the loop, and $\hat{z}$ which is parallel to the wire. Using these direction, we write the $B$ field as
$$\vec{B} = B_t \hat{t} + B_r \hat{r} + B_z \hat{z}.$$
Note that as we go around the loop the direction element $d\vec{L}$ is perpendicular to $\hat{r}$ and $\hat{z}$, but parallel to $\hat{t}$:
$$
\vec{r}\cdot d\vec{L} = 0 \qquad
\vec{z}\cdot d\vec{L} = 0 \qquad
\vec{t}\cdot d\vec{L} = L.
$$
Also, due to the rotational and translational symmetry of the problem, the values of $B_t$, $B_r$, and $B_z$ are constant around any loop of radius $R$ centered on the axis of the wire.
Writing out the Ampere's law integral with the loop $C$ we get
$$
\begin{align}
\mu_0 I &= \int_C \vec{B}\cdot d\vec{L} \\
&=
B_t \int_C \hat{t} \cdot d\vec{L} +
B_r \int_C \hat{r} \cdot d\vec{L} +
B_z \int_C \hat{z} \cdot d\vec{L} \\
&=
B_t 2\pi R.
\end{align}
$$
Where $R$ is the radius of the loop.
This unambiguously solves for $B_t$.
However, as you pointed out, we have not shown that $B_r=0$ and $B_z=0$.
Prove $B_r=0$
Consider a surface $S$ bounded by a contour $C$.
Let $\hat{n}$ denote the outward pointing normal vector at each point on $C$.
There is an integral theorem which says
$$\int_C (\vec{V}\cdot \hat{n}) dL = \int_S (\nabla\cdot \vec{V})dA . \qquad (\text{divergence theorem})$$
We apply this theorem to our physics problem.
Let the Ampere's loop by the contour $C$ and let the flat disk whose boundary is $C$ be the surface $S$.
In this case, the outward pointing normal vector $\hat{n}$ is just the radial vector $\hat{r}$.
Therefore, the divergence theorem gives
$$
\begin{align}
\int_C (\vec{B} \cdot \hat{r})\,dL &= \int_S \underbrace{ (\nabla \cdot \vec{B})}_{0\text{ by Maxwell's equations}} \,dA \\
B_r \int_C dL &= 0 \\
B_r &= 0 .
\end{align}
$$
Since we already said that $B_r$ is constant on the loop, this proves that $B_r = 0$ everywhere (our loop has arbitrary radius so we've now shown that at any radius from the wire $B_r$ is identically zero).
Prove $B_z = 0$
We now construct a new loop, as shown in the attached diagram.
There is not current enclosed by this loop, so we have
$$
B_r \int_C \vec{r}\cdot d\vec{L} +
B_t \int_C \vec{t}\cdot d\vec{L} +
B_z \int_C \vec{z}\cdot d\vec{L}
= 0 . \qquad (*)$$
The loop lies entirely in the $r-z$ plane.
Therefore, the length element $d\vec{L}$ has no component in the $\hat{t}$ direction.
Combining this with the fact that we already showed $B_r=0$, we simplify $(*)$ to
$$B_z \int_C \vec{z} \cdot d\vec{L} = 0. $$
The side segments (blue) of our loop have no component in the $\hat{z}$ direction.
We can take the dotted segment to be arbitrarily far away where the $B$ field must go to zero.
Therefore, the only remaining segment contributing to the integral is the red segment.
The red segment is entirely in the $\hat{z}$ direction: $d\vec{L} = \hat{z}dL$.
This leaves us with
$$
\begin{align}
B_z \int_{\text{red segment}} (\hat{z} \cdot \hat{z}) dL &= 0 \\
B_z \times (\text{length of red segment}) &= 0 \\
B_z &= 0 .
\end{align}
$$
When the loop is wholly within the region of the uniform magnetic field, then there is no induced current even when the loop is moving. As you pointed out, in this case the induced currents on opposite sides of the loop are opposite and equal so they cancel out.
It is only when one side of the loop leaves the region of the magnetic field, or more generally enters a region in which the field is different, that the induced currents are no longer equal. In this case there is no induced current in the part of the loop which is no longer within the magnetic field. More generally, the side of the loop on which the magnetic field is stronger will determine the direction of the resultant current.
If the loop is moving out of the region of magnetic field with a uniform speed, would the induced current be steady? If not, why? Because of the loop's geometry?
Good follow-up question.
Induced emf (and therefore also induced current) is equal to rate of change of flux linkage through the loop. If there is a sudden change in flux linkage, the current will change suddenly.
If the loop is a rectangle with one side parallel to the edge of the magnetic field, then there is a sudden change in flux linkage as that edge crosses the boundary, and again when the trailing edge crosses. In between there is a constant decrease in flux linkage, therefore a constant current. The current-time graph is rectangular.
For both a circular loop and a rectangular loop with a corner crossing the boundary first, the current will increase and decrease continuously from zero to a maximum, because the flux linkage is not changing suddenly. For the circle the induced current does not change uniformly, because of the non-linear edges; the current-time graph is semi-eliptical. For the rectangular corner=first loop the current-time graph is triangular or trapezoidal. For both circular and rectangular corner-first loops, the the maximum current occurs when the widest part of the loop crosses the boundary, because that is when the flux changes most rapidly.
Best Answer
The "Right hand rule" is a convention both for cross-products and for x-y-z coordinate systems. In pre-war Germany it was common to use a LEFT-HANDED coordinate system where if x increased "to the right" and y increased "up the page" then z INCREASED "into the page." However, nowadays this is very uncommon and we use right-handed coordinate systems where if x increases "to the right" and y increases "up the page" then z increases "out of the page."
Clearly (?) there is no one intrinsically "correct" convention.
But, as stated above, nowadays we almost exclusively use the right-handed coordinate convention where $$ \hat x \times \hat y = \hat z\;. $$
Given the above convention you can "mathematically" compute cross products.
For example, if $$ \vec V = |V|\hat x $$ and $$ \vec B = |B|\hat y $$ then $$ F = q|V||B|\hat z $$ because, by convention: $$ \hat x \times \hat y = \hat z\;. $$
Update (per the comment):
The example above makes use of the right-hand rule for $\hat x \times \hat y$. But, in general, you will also find these two other rules useful: $$ \hat z \times \hat x = \hat y $$ $$ \hat y \times \hat z = \hat x $$