[Math] History of Lagrange Multipliers

Question

How did Lagrange discover Lagrange multipliers? Also, was it related to his work on the calculus of variations? And how did he originally understand/implement the technique?

Answer 1

Here is some information from the book A History of Analysis written by Hans Niels Jahnke

Excerpt from section 5.2 The analytical mechanics, 5.2.1 The principle of virtual velocities:

Lagrange presented his new conception in a memoir (Lagrange 1764), written to answer a question advanced in 1762 by the Academy of Paris, concerning the libration of the moon. More than fifteen years later (Lagrange 1780), he outlined a more elaborate and general version of the same program. He finally realised this program in the Méchanique analitique (Lagrange 1788).

Lagrange's idea has two clear sources. The first is the Bernoullian principle of virtual velocities for the equilibrium of any system of bodies animated by central forces. ... The second source is d'Alembert's principle for dynamics. ... Lagrange showed ((Lagrange $1780$, $218$-$220$) and (Lagrange $1788$, $216$-$227$)) that, if the forces are conservative, then $(5.7)$ gives rise to the equation

$$\sum_{i=1}^{\nu}\left[d\frac{\delta T}{\delta d\varphi_i}-\frac{\delta T}{\delta \varphi_i}+\frac{\delta U}{\delta \varphi_i}\right]\delta \varphi_i = 0\qquad(5.8)$$

where $T$ and $U$ are what we would now call the kinetic and the potential energy of the system. This is the original version of the so-called Lagrangian equation of motion. Its deduction from $(5.7)$ depends on the choice of the variable $\varphi_i$. We must ensure that the variations for these variables are mutually independent.

Thus, we have to chose these variables in a suitable way, subject to the equations of conditions of the system. Since this could be difficult, Lagrange presented a way of making this elimination easier in the Méchanique analitique. This device is now known as the method of Lagrange multipliers (Lagrange $1788$, $44$-$58$ and $227$-$232$).

The basic idea is to express the internal constraints of the system by adding to the first term of $(5.8)$ a suitable term for any equation of condition. If this equation is $\Psi=0$, this term has to be of the form $\psi\delta\Psi$, where $\psi$ is an indeterminate multiplier to be determined by analytical procedures.

The Lagrangian equation of motion and the method of Lagrange multipliers are important contributions to the theoretical mechanics. However, they appear in both the $1780$ memoir on the libration of the moon and in the Méchanique analitique as tools to make possible an application of the quasi-algebraic method of undetermined coefficients to the general equation expressing the principle of virtual velocities, in order to deduce from it the equations of motion for any system of bodies.

Added 2014-05-17: Information added according to the comment of NotNotLogical

He was asking for some more information about what insight drove Lagrange to make the discovery. Here are additional aspects which I found in the paper On the Genesis of the Lagrange Multipliers from P. Bussotti (see also the comment from Steve Kass to this question).

The following text is essentially from P. Bussottis paper: On the Genesis of the Lagrange Multipliers

1.) The Lagrange Multiplier were introduced in the framework of statics

He introduced this mathematical approach in the framework of statics in order to determine the general equations of equilibrium for problems with constraints. At the beginning of his Mécanique Analytique (Ref $1$), Lagrange tackles statics and poses three principles as the foundations for the subject: (i) the principle of the lever (ii) the principle of the composition of forces and (iii) the principle of virtual velocities. It was this third principle virtual velocities where the Lagrange Multiplier occurred the first time.

Lagrange writes in (Ref $1$, pages $17$-$18$):

By virtual velocity, it has to be meant the one that a body in an equilibrium condition would receive if the equilibrium was interrupted; namely, the velocity that the body would really assume in the first instant of movement; the principle consists in this: the forces are in equilibrium if they are in the inverse proportion to their virtual velocities. . . .

So, if we consider a material point to which only two forces $P$ and $Q$ are applied and if $\frac{dp}{dt}$ and $\frac{dq}{dt}$ ($p$ and $q$ being the directions, respectively, of $P$ and $Q$) are the virtual velocities that the body receives, respectively, by the forces $P$ and $Q$, then the body is in equilibrium if $$P\frac{dp}{dt}+Q\frac{dq}{dt}=0$$ This equation represents the principle of the virtual velocities. The quantity $\frac{dp}{dt}$ is the moment of the force $P$.

At the beginning of Section $2$ (Ref. $1$, paragraph $1$), when the real mathematical treatment begins, Lagrange replaces $\frac{dp}{dt}$ with $dp$. He justifies the substitution in this way (Ref. $1$, page $24$):

Lagrange (Ref. $1$, page $24$):

In order to express this principle by means of formulas, we suppose that the forces $P, Q, R$ are directed along given lines and that they are in equilibrium; let $p, q, r$, etc. denote the lines that are directions of the forces $P, Q, R$.

We indicate the variations or the differences (now called differentials) of these lines with $dp, dq, dr$, etc., and suppose that they are caused by an arbitrary infinitesimal change in the position of the different bodies or points of the system. It is clear that these differences will express the spaces that are covered in the same instant by the action of the forces $P, Q, R$ along their respective directions . ...

The differences $dp, dq, dr$ will be proportional to the virtual velocities of the forces $P, Q, R$. In order to simplify the treatment, it will be possible to consider these differences instead of the velocities.

So, the general equation of the equilibrium of a point on which three forces are applied will be

$$Pdp+Qdq+Rdr=0$$

2.) Lagrange's motivation for introducing the multiplier method:

At the end of Section $2$ (Ref. 1, paragraph 3, page 27), he points out that it is important to reduce Mechanics to purely analytical operations and to free it from intuitive geometrical considerations.

For this reason, in Section $4$, he introduces the method of the multipliers (Méthode des multipliers is the title of paragraph $1$ of Section $4$).

In this way, Lagrange is able to solve the problem in a more general and in a simpler way. Particularly, he studies in detail the equilibrium under constraints. He examines, in a first instance, the equilibrium of a material point or of a system of material points that are submitted to constraints. If the constraining equations are $L=0, M=0$, . . . and if $L, M$ are functions of several variables, it follows necessarily that $dL=0, dM=0$, etc. Consequently, if $\lambda,\mu$ are arbitrary real numbers [Lagrange calls them quantités indéterminées or coefficientes indéterminées (Ref. $1$, page $70$)], we have $\lambda dL=0, \mu dM=0$, etc. He explains:

Lagrange (Ref. $1$, page $70$):

We will consider the sum of all the moments of the forces that must be in equilibrium, we will add the different differential functions that must be zero, on the basis of the conditions of the problem, after having multiplied every function by an indeterminate coefficient; we will make the whole equal to zero and so we will have a differential equation that we will treat as an ordinary equation de maximis et minimis.

The general form of this differential equation is

$$P dp+Q dq+\cdots+\lambda dL+\mu dM\cdots = 0$$

Such equation is the general equation of equilibrium. Considering each variable of an orthogonal reference frame, we will have a particular equation of equilibrium. For example, in the direction $x$, we will have

$$P \frac{dp}{dx}+Q \frac{dq}{dx}+\cdots+\lambda \frac{dL}{dx}+\mu \frac{dM}{dx}\cdots = 0$$

From a physical point of view, the reaction of the constraint is equated to an agent force; indeed, $\lambda$ and $\mu$ are treated as forces and the expressions $\lambda dL$ and $\mu dM$ represent the virtual works that are realized by these forces (Lagrange continues to call them moments).

The advantage of this method is that it allows the treatment of the problems with constraint in the same way as free problems, adding the virtual works realized by the reaction of the constraints to the virtual works realized by the agent forces.

A first extremely important consequence of such an approach is that* the analytical and numerical treatment of the constraints is postponed to the writing of the equilibrium condition*.

From a mathematical point of view, Lagrange points out that the difficulty consists in determining the values of $\lambda , \mu$, etc. Actually, this difficulty can be overcome easily (Ref. $1$, paragraph $4$, Section $4$, page $71$), since in fact the use of the constraints allows having a system where the number of the equations and the number of the variables is the same.

An example can be useful to interpret and clarify the Lagrange reasoning: if we have two agent forces $P$ and $Q$ and a constraint represented by the equation $L=0$, we will have (imagining it to be in the ordinary space) the three unknown quantities $x, y, z$ and the unknown quantity $\lambda$ , and we will have four equations

\begin{align*} P \frac{dp}{dx}+Q \frac{dq}{dx}+\lambda \frac{dL}{dx}=0\\ P \frac{dp}{dy}+Q \frac{dq}{dy}+\lambda \frac{dL}{dy}=0\\ P \frac{dp}{dz}+Q \frac{dq}{dz}+\lambda \frac{dL}{dz}=0\\ L=0 \end{align*}

If the multiplier is interpreted as a new coordinate, the last equation can be replaced by another one that must be valid in the case of equilibrium: \begin{align*} P \frac{dp}{d\lambda}+Q \frac{dq}{d\lambda}+ \lambda \frac{dL}{d\lambda}=0 \end{align*}

In this way, the solution of the problem of the equilibrium of a material point or of a system of material points appears to be, from a mathematical point of view, analogous to the problem of determining the maximum or the minimum of a function under constraints.