Why do we get the same differential equations from both principles? Surely there is a fundamental connection between them? When written out, the two seem to have nothing in common.
$$\sum _i ( \mathbf F _i – \dot{\mathbf p}_i) \cdot \delta \mathbf r _i = 0$$
$$S[q(t)] = \int ^{t_2} _{t_1} \mathcal L (q,\dot{q},t)dt$$
After playing with d'Alembert's principle we find that we can rewrite the whole thing as $$\sum _i \left[ \frac{d}{dt} \left( \frac{\partial T}{\partial \dot{q}_i} \right) – \frac{\partial T}{\partial q_i}-Q_i\right] \delta q_i$$
This can further be rewritten under certain conditions so we get the exact form of the E-L equation.
It seems to me that both ways of arriving at the result are fundamentally different.
A function must obey the E-L equations in order to minimize the action over a path, but when we look at the virtual work, it appears that they come from the fact that (quoting Goldstein) "particles in the system will be in equilibrium under a force equal to the actual force plus a 'reversed effective force'."
I think I understand the principle of stationary action, I can see how it leads to the E-L equations, but d'Alembert's Principle seems so arbitrary, I can't see any motivation for it.
Best Answer
The principle of Least (Stationary) Action (aka Hamilton's Principle) is derived from Newton's axioms plus D'Alembert's principle of virtual displacements.
Because D'Alembert's principle allows to account for the (reactions of the) bonds between the components of a system in a transparent way, the Lagrangian and Hamiltonian formulations are possible.
Note1: Newton's axioms, as given, cannot derive neither the Lagrangian form nor the Hamiltonian as they would need the reactions of the bonds to be added literally inside the formalism, thus resulting in different dimensionality and equations for the same problem where the (reactions of the) constraints would appear as extra unknowns.
Note2: D'Alembert's principle is more general than the Lagrangian or Hamiltonian formalisms, as it can account also for non-holonomic bonds (in a slight generalisation).
UPDATE1:
When the forces are conservative, meaning derived from a potential $V(q_i)$ i.e $Q_i = -\frac{\partial V}{\partial q_i}$, and the potential is not depending on velocities $\dot{q_j}$ i.e $\partial V / \partial \dot{q_j} = 0$ (or the potential $V(q_i, \dot{q_i})$ can depend on velocities in a specific way i.e $Q_i = \frac{d}{dt} \left( \frac{\partial V}{\partial \dot{q_i}} \right) - \frac{\partial V}{\partial q_i}$, refered to as generalised potetial, like in the case of Electromagnetism), then the equations of motion become:
$$\frac{d}{dt} \left( \frac{\partial T}{\partial \dot{q_i}} \right) - \frac{\partial T}{\partial q_i}-Q_i = \frac{d}{dt} \left( \frac{\partial L}{\partial \dot{q_i}} \right) - \frac{\partial L}{\partial q_i}$$
where $L=T-V$ is the Lagrangian.
(ref: Theoretical Mechanics, Vol II, J. Hatzidimitriou, in Greek)
UPDATE2:
One can infact formulate D'Alembert's principle as an "action principle" but this "action" is in general very different from the known Hamiltonian/Lagrangian action.
Variational principles of classical mechanics
Variational Principles Cheat Sheet
THE GENERALIZED D' ALEMBERT-LAGRANGE EQUATION
1.2 Prehistory of the Lagrangian Approach
GENERALIZED LAGRANGE–D’ALEMBERT PRINCIPLE
For a further generalisation of d'Alembert-Lagrange-Gauss principle to non-linear (non-ideal) constraints see the work of Udwadia Firdaus (for example New General Principle of Mechanics and Its Application to General Nonideal Nonholonomic Systems)