The important idea here is that an "ideal" string does not stretch; its length stays constant. Therefore, the distance between the two objects attached to the string, as measured along the length of the string, is fixed. In some sense, the string is acting like the rigid object you mention since the "along the string length" remains the same.
So, in your case, if the hanging mass moves downward by, say, 3 cm, then the cart must also move horizontally by 3 cm. If they didn't move the same amount, then the string would be stretching or it would become loose. Now, since the displacements of the two objects (in their respective directions) must be the same in any given time interval, the velocities must be the same (just divide by a short time interval and take it to zero). Likewise, the accelerations must be the same as well since the velocities are the same.
I'm being a bit sloppy here with directionality and vectors, but I hope my point gets across.
In your original question, you argue that intuitively it doesn't make sense if you examine Newton's second law. It seems you are implicitly assuming that the net forces on the two objects are the same in order to say that the accelerations could be different. This isn't the case (unless they happen to have the same mass). In fact, their accelerations must be the same for the reason stated above.
When the (inertial) mass is zero, then the acceleration can be non-zero for zero force.
This is similar, conceptually, to what has been discussed recently regarding an ideal conductor.
Consider Ohm's Law:
$$V = IR$$
Now, what if $R = 0$ as is the case with an ideal conductor?
Clearly, the voltage must be zero for any current. The current through the conductor, then, is determined by constraints external to the ideal wire, i.e., by whatever the ideal wire is connected to.
Consider Newton's 2nd Law:
$$F = ma$$
Now, what if $m= 0$ as is the case with the massless rope?
Clearly, the force must be zero for any acceleration. The acceleration, then, is determined by constraints external to the massless rope, e.g., the attached masses.
Yes, the massless rope is ideal and, thus, not physical but, there can be effectively massless ropes just as there can be effectively ideal conductors. Which is to say that, to the precision one is working to, the rope has zero mass and zero force acting on it but non-zero acceleration.
Best Answer
It is best to draw free body diagrams for the two masses.
$F$ is the applied force and $T$ the tension in the massless and inextensible rope joining the two masses.
There is no friction and both masses have the same acceleration $a$.
Applying Newton's second law for each of the masses:
$T = m_1\;a$ and $F-T= m_2\; a \Rightarrow F = (m_1+m_2)\;a$ so $F>T$
You can think of it as the force $F$ is accelerating both masses whereas the force $T$ only has to accelerate mass $m_2$.