When you look at the dynamics in the rotating reference frame, there are 4 forces acting on the particle: the two gravitational pulls from the massive bodies, the centrifugal push away from the center of rotation (located between the massive objects) and the Coriolis force.
The first three forces depend on the position of the particle, and can be derived from a potential (that also depends on the position), whose level curves are shown in the picture presented with the question. This potential has local maxima at L4 and L5.
The Coriolis force depends on the velocity of the particle: it is perpendicular to it, contained in the plane of motion and proportional to the speed. It curves the motion of the particle to the right (if the massive bodies and the reference system rotate counterclockwise, which is what you see in our Solar System if you stand on the North pole of the Earth).
If a particle placed at L4 tries to leave the point with a mild speed, the Coriolis force curves its trajectory. The trajectory is too curly to get anywhere. See the animation at http://demonstrations.wolfram.com/OrbitsAroundTheLagrangePointL4/.
Of course this doesn't prove that the particle will stay near L4 forever. I don't know a proof. I've seen some computations that show that the dynamical equation linearised at L4 is stable if the mass ratio of the massive objects is sufficiently large, but this also is not enough to prove stability in the non-linearised problem.
I would be convinced that the equilibrium is stable if I were shown that there exists a conserved quantity (depending on the position and speed) that has a strict local extremum at that point of phase space (position=L4, speed=0).
The "energy" (potential discussed above + kinetic energy measured in our non-inertial reference system) is conserved, because the Coriolis force is perpendicular to the trajectory, so it doesn't perform work (in fact, in Lagrangian mechanics it is derived from a potential that depends on the position and speed of the particle). But this quantity doesn't have an extremum at our equilibrium point, because the potential has a local maximum at L4 and the kinetic term is minimum when the speed is 0.
So I can't prove that the equilibrium is stable.
In the three-body problem, the Lagrange points are those points in space where two bodies with large mass (Earth and Sun), through the interaction of the respective gravitational force, allow a third body with a much lower mass to maintain a stable position relative to them.
In a planetary system it implies that a small object, such as a satellite or an asteroid, which shares the same orbit of a planet and positioned in a Lagrange point, will keep constant the distances between the major celestial bodies, the star and the planet with which shares the orbit.
For this to happen, the resultant of the gravitational accelerations imparted by the celestial bodies to the object must be exactly the centripetal acceleration necessary to keep the object in orbit at that particular distance from the largest celestial body, with the same angular velocity as the planet.
Out of curiosity:
you can also verify using Lagrangian mechanics these five Lagrangian points.
Let be $(P-O)=\rho \vec{e}_\rho$ the distance between our third point and the mass centre of the system; $(T-O)=x_E \vec{e}_x$ the distance between the Earth and the CM and $(S-O)=x_S \vec{e}_x$ the distance between the Sun and the CM (very near to the centre of the Sun).
You will find that the system potential energy is $$U_{tot}=U_{grav}+U_{centr}=-G\frac{M_E\cdot m}{||P-T||}-G\frac{M_S\cdot m}{||P-S||}-\frac{1}{2}m\Omega^2\rho^2$$
If you derivate this energy potential respect to $\rho$ and $\theta$ you will find these points. And also, if you make the Hessian matrix you will find that 3 of them are points of instability $(L_1, L_2, L_3)$ and the other 2 are points of stability $(L_4, L_5)$.
I do not pretend that you understand perfectly these calculations but only the principal idea that is behind.
I hope that the first part of the answer could be helpful for you and maybe also the second one.
Best Answer
Yes the free body moves outward, but there are two critical things you have to know to interpret this statement correctly.
First, this is the effective potential, taking into account gravity and centrifugal force. It has this form because we went into the non-inertial frame co-rotating with the two masses. Mathematically, the potential is $$ \Phi_\mathrm{eff}(\vec{r}) = -G \left(\underbrace{\frac{M_1}{\lvert \vec{r}-\vec{r}_1 \rvert}}_\text{potential from mass 1} + \underbrace{\frac{M_2}{\lvert \vec{r}-\vec{r}_2 \rvert}}_\text{potential from mass 2} + \underbrace{\frac{M_1+M_2}{2\lvert \vec{r}_1-\vec{r}_2 \rvert^3} \lvert \vec{r} \rvert^2}_\text{centrifugal component}\right), $$ and it only decreases far away because of that last term.
Physically, this is because placing an object "at rest" in this frame corresponds to having it move with the same angular frequency as $M_1$ and $M_2$ about the center of mass. If you initialize an object $5\ \mathrm{AU}$ on a tangential path having the same angular velocity as the Earth, it will be moving too fast for a circular orbit at that distance, and so it will move away from the Sun.
This does not mean the object will go away forever, and that brings us to the second point, explained in Chay's response: Not all effective forces have been accounted for; in particular, the Coriolis force does not arise from $\Phi_\mathrm{eff}$. The Coriolis force depends on velocity, so it has no scalar potential depending solely on position, and so it is not included in the analysis so far. Once your test object starts moving in your rotating frame, it will experience a perpendicular deflection that will eventually force it to turn around.