The initial premise of your question is not, in general, correct.
Consider a 1000 kg car driving at a constant speed of 20 m/s around a flat circular racetrack with a radius of 500 meters. Note the constant speed: the car will take the same time to drive around the track, 157.1 seconds, hour after hour. The only change in the car's velocity is in the direction of the velocity, not its size. The only acceleration is the centripetal acceleration, and the only horizontal force needed is the centripetal force $$a_{cp}=\frac {v^2}{r}=0.8 \frac{m}{sec^2}$$ $$F_{cp}=m \times a_{cp}=800 \ newtons$$
The force exerted by the car's engine serves only to balance the various drag forces.
This force is supplied by the friction between the tires and the pavement; pour out some oil on the track to see what happens when the required centripetal force is not present!
The only acceleration is directed exactly towards the center of the circular track; there is no tangential acceleration. The car, at some moment, is travelling North at 20 m/s, and one half-circuit later, it is travelling South at the same speed. Clearly, it has accelerated.
Assume now that the driver presses on the brake pedal in a manner that she knows will bring the car to a stop in 40 seconds. There is now a tangential acceleration, at $-0.5 \frac{m}{sec^2}$, in addition to the centripetal acceleration above, and a tangential force of 500 newtons directed towards the back of the car. The total force needed from the tires is now the resultant of these two forces: 943.4 newtons directed around 32 degrees aft of inward. Touching the brakes could throw you into a skid! Of course, as the braking changes the car's speed, the centripetal force will decrease...
Something that is left out of (or insufficiently emphasized in) a lot of textbook treatments of centripetal acceleration/force is how physicist use this fact.
In introductory treatments, uniform circular motion plays a very similar role to equilibrium.
You are expected to read a problem, notice that some object (say a ladder with a fireman on it) is not accelerating and then proceeded to use take advantage of the equations of static equilibrium $\sum_i F_i = 0$ and $\sum_i \tau_i = 0$ to work the problem.
Equivalently you are expected to read a problem notice that some object is following a circular path and say "Oh! I know something about the sum of the radial forces $\sum_i F_i \cdot \hat{\mathbf{r}} = -m \frac{v^2}{r}$. Then you look at the forces in action in the problem and figure out the implication of that constraint.
If the professor is swinging a bucket of water over his head and the water does not fall out and soak the instructor then you know that near the top the water was in circular motion and the net force generated by gravity and the reaction between the water and bucket (which is ultimately supported by the tension in the professors arm) works out to centripetal force. At the slowest speed for which the water doesn't fall out that is the smallest possible value of centripetal force consistent with a circular path. And since the effect of gravity on the water is non-negotialble the only way to reduce the centripetal force is to reduce the reaction force. Therefore, when the reaction force is zero, the bucket has the slowest possible speed that keeps the professor dry. QED.
You've been trying to work the problem forward: I know the force and from that I deduce the action. That works, and is done in a complete treatment of planetary motion, for instance, but the math is complicated.
What the book is expecting you to do is work the problem backwards: the [object] is following a circular path so I know the sum of the radial forces on it, so I can constrain one of the forces...
Best Answer
It is Newton's first law:
If by some magic, the attraction of the sun stopped, the earth would leave off on a tangent.
It all started with the formation of the solar system:
The initial forces were probably thermodynamic exchanges of scatterings . The slow domination of the collective gravitational field condensing to a sun and planets again depends on laws : conservation of angular momentum in particular.
So as implied in the comments, the real question is "why Newton's first law".
And the answer is that laws and postulates in physics are the extra axioms imposed so that the mathematical theory fits the data. It explains the tangential velocity, but the law itself was chosen so that the kinematics would be fitted and new set ups could be explained and predicted. That is what physics is about, understanding mathematically the way nature works.