The last expression should be a dot product:
$$\frac{dK}{dt} = m\vec v \cdot \frac{d\vec v}{dt}$$
So when a force is applied perpendicularly to the direction of motion, $\frac{d\vec v}{dt}$ is a vector that is perpendicular to $\vec v$, so their dot product is zero, giving you zero change in the kinetic energy as expected.
Your question is interesting because it shows the dangers of working with infinitesimals without a careful control of their meaning.
Basically, your question is applicable to any motion, when the trajectory in phase space reaches an extremum along one of the directions. So, for simplicity let's discuss the simple one dimensional harmonic oscillator, which, in your example exactly corresponds to the $x$- component of the uniform circular motion.
The equation of motion is
$$
\ddot x(t) = - \omega^2x(t),
$$
valid for any time $t$.
When $x=0$, acceleration is zero and the speed along $x$ direction is maximum.
So,how it happens that there is a decrease of $\dot x$?
A naïf application of infinitesimals is misleading. In the present case
$$
\dot x(t+dt) \simeq \dot x(t) + \ddot x(t) dt
$$
would imply $\dot x(t+dt) = \dot x(t) $. But this is just the first order result. First order approximations are the leading term of a local analysis of the behavior of a regular function provided they do not vanish. When, like in the case of an extremum of velocity at time $t_0$ (threfore $\ddot x(t_0)=0$), the first order variation is zero, it is necessary to look for the next non-zero term in an expansion in powers of $dt$. In the present case:
$$
\dot x(t_0+dt) = \dot x(t_0) + \ddot x(t_0) dt + \frac12 \dddot x(t_0) dt^2 = \dot x(t_0) - \frac12 \omega^2 \dot x(t_0) dt^2,
$$
where use has been done of the equation of motion (by taking a time derivative of both sides) in order to express the third derivative of $x$ with respect to time as a function of $\dot x$.
Therefore, one can see that, at the dominant non vanishing order in $dt$, $\dot x$ is correctly varying.
Best Answer
The perpendicular component of a force will never change the speed. This is true.
What is happening in your case is at the instant that the force along the y-axis is applied, it will, for that short, differentially small time, only change the direction of velocity slightly upwards, but have no effect on the speed. However, if the force keeps acting upwards -- and now the object has a velocity component in the upwards y-direction -- the force is no longer perpendicular (there is a tangential component) and thereby the speed will change.
The force's direction would also need to change in such a way that it always remains perpendicular to the object's velocity. An example of this is uniform circular motion.