Hamilton's principle states that the path taken by the system between times $t_1$ and $t_2$ and the coordinates $q_1$ and $q_2$ is the one for which the variation of the action functional is zero:
$$
\delta S= \delta \int_{t_{1}}^{t_{2}} L\left(q_{j}, \dot{q}_{j}, t\right) d t=0, \tag{1}
$$
If we assume that the limits of the integral are fixed, we can pass the $\delta$ operator inside the integral by Leibniz's rule, obtaining the classical result
$$\delta \int_{t_{1}}^{t_{2}} L\left(q_{j}, \dot{q}_{j}, t\right) dt =\int_{t_{1}}^{t_{2}} \delta L\left(q_{j}, \dot{q}_{j}, t\right) dt, \tag{2}$$
$$
\int_{t_{1}}^{t_{2}}\left(\frac{\partial L}{\partial q_{j}}-\frac{d}{d t} \frac{\partial L}{\partial \dot{q}_{j}}\right) \delta q_{j} d t=0.
$$
However, according to a footnote on p. 208 in Marion and Thornton's Classical Mechanics book,
It is not necessary that the limits of integration be considered fixed. If they are allowed to vary, the problem increases to finding not only $q(t)$ but also $t_{1}$ and $t_{2}$ such that $S$ is an extremum.
What form would then take Hamilton's principle in this case where the limits of integration are not fixed? That is, how would the $\delta$ operator act on an integral of this kind?
Note. I have found in another source that the variation of the action would take this form, but not a justification for it:
$$\delta S=\int_{t_{1}}^{t_{2}}\left(\frac{\partial L}{\partial q_j}-\frac{d}{d t}\left(\frac{\partial L}{\partial \dot{q_j}}\right)\right) \delta q_j d t+\left.(p_j \delta q_j-H \delta t)\right|_{t_{1}} ^{t_{2}}. \tag{3}$$
Best Answer
In the derivation of the Euler-Lagrange equation you typically use integration by parts, which gives something like
\begin{align*} \delta S &= \sum_k \frac{\partial L}{\partial \dot{q}_k}\delta q_k \Bigg|_{t_1}^{t_2} + \int_{t_1}^{t_2} \sum_k \left(\frac{\partial L}{\partial q_k} - \frac{\mathrm{d}}{\mathrm{d}t}\frac{\partial L}{\partial \dot{q}_k} \right) \delta q_k \, \mathrm{d}t \end{align*} Assuming the variation vanishes at $t_1, t_2$ i.e. $\delta q_k(t_1) = 0 = \delta q_k(t_2)$ and assuming you can vary them independently you get the usual EL equations. But if you don't fix the endpoints the first sum will be nonzero and you have more conditions that you need to fulfill. If they aren't fixed you will need to solve the EL equations and $$ \frac{\partial L}{\partial \dot{q}_k}\Bigg|_{t_1, t_2} = 0 $$ to ensure that $\delta S = 0$. Note that in the Hamilton formalism this will be the generalised momentum that appears in your equation. If $\partial_t L \neq 0$ then the energy $H$ won't be conserved so it'll contribute to the variation at the endpoints aswell.