I think we are supposed to assume that the buoyancy force of the balloon is equal to the weight force of the balloon, ladder, and climber. If this is the case, the system is in equilibrium with its environment, with no net forces to the environment.
You could look at it as a center of mass problem. We assume the ladder has negligible mass. When the climber reaches the balloon, they will be together at the center of mass of the system. The climber and balloon start apart by the length of ladder $l$, while the center of mass is $\frac{l\times M}{m + M}$ above the climber, and $\frac{l\times m}{m + M}$ below the balloon. The ratio of the balloon distance to travel compared to the person distance to travel is $\frac{m}{M}$. Therefore the velocity of the balloon at any point in time is $-v_{person} \times \frac{m_{person}}{M_{balloon}}$.
Center of mass problems can be reformulated as conservation of momentum problems. At $t_0$, the balloon and person are both at rest, with the person hanging off the bottom of the ladder - net momentum 0. At $t_1$, the climber has momentum $m_{person}\times v_{person}$, so the momentum of the balloon must balance this: $M_{balloon} \times \frac{-m_{person}\times v_{person}}{M_{balloon}}$. Same result.
The teacher is wrong.
[Edit: I added a slightly easier/better argument at the bottom]
A hot air balloon can fly due to its buoyancy (i.e. hot air is lighter than cold air), also called the Archimedes force. If the balloon is floating in mid-air, and thus not ascending or descending, this means that the gravity on the balloon is exactly matched by the the Archimedes force on it in the opposite direction: $F_A = F_g$.
When the man starts climbing the ladder of the balloon, the force due to gravity on the balloon-man system will become $mg + 3mg = \frac{4}{3}F_g$. Now the man's contribution to the Archimedes force $F_A$ is less easy to calculate (see https://en.wikipedia.org/wiki/Buoyancy#Forces_and_equilibrium). But since the man is not floating in the air, his buoyancy is considerably less, meaning that $F_{A,man} < F_{g,man} = mg$.
So we had $F_{A,balloon} = F_{g,balloon} = 3mg$. And for the man $F_{A,man} < F_{g,man} = mg$.
This means that:
$F_{A,balloon+man} = F_{A,man} + F_{A,balloon} < F_{g,man} + F_{g,balloon} = 4mg = F_{g,man+balloon}$.
Or in words: the balloon-man system will accelerate slowly down to the ground, since its buoyancy is less that the gravity on it.
The reason why in reality it might work that you climb on the ladder of a balloon without the balloon descending, is because in that case the pilot of the hot-air balloon will light the fire in it, heating the air in the balloon, which will increase its buoyancy and thus compensate for your extra weight and lack of buoyancy.
Edit:
You can also see it like this: As long as the man is standing on the ground, the man-balloon system is in equilibrium, meaning no resulting external nett force, and thus indeed the COM should not move.
The force of gravity on the man is balanced by the normal force (i.e. the upward force exerted by the ground supporting him) and the man's Archimedes force (although this is actually negligible for a person in air). The gravity on the balloon is counteracted by the Archimedes force on it. So
$F_{n,man} + F_{A,man} = F_{g,man}$
and
$F_{A,balloon} = F_{g,balloon}$
Thus the total net force on the man-balloon system is 0:
$F_{net} = F_{n,man} + F_{A,man} - F_{g,man} + F_{A,balloon} - F_{g,balloon} = 0$
But as soon as the man starts climbing the ladder of the balloon, the man is no longer supported by the ground, and thus the normal force $F_{n,man}$ drops out. You can see that the total external nett force on the man-balloon system will no longer be zero, and thus the COM does not have to remain at the same place.
Best Answer
It is easy to see from this website, problem/solution 23, http://books.google.com.au/books?id=8NtJLfGf94QC&pg=PA431&lpg=PA431&dq=monkey+climbing+ladder+on+pulley&source=bl&ots=2tWFRhdKME&sig=9ocddC7lk53A1_XdrVAwPm7MBw0&hl=en&sa=X&ei=Jv9SUevUEoTziAfSs4HIBg&ved=0CDAQ6AEwAA#v=onepage&q=monkey%20climbing%20ladder%20on%20pulley&f=false, that as the man moves up a distance $l'$, the centre of gravity moves up by a distance $l = \frac{ml'}{2M}$. Where $m$ is the mass of the man, and $M$ is the mass of the counter-weight.
This means that if the man is moving at a velocity of $V$ meters per second, then the centre of gravity will move up at a rate of $\frac{mV}{2M}$ meters per second as well.