variational principle - When can we add a total time derivative of $f(q, dot{q}, t)$ to a Lagrangian?

The other day, I was listening to this lecture on the Lagrangian for a charged particle in an electromagnetic field, and at one point in the video, the lecturer mentions that we can add any total time derivative of a function $f(q, t)$ to the Lagrangian without altering its equations of motion.

This is nothing new to me, and I understand it fully, but shortly afterwards (approximately two minutes after the linked starting point), he goes on to say that you can, in fact, add a total time derivative of a function $f(q, \dot{q}, t)$, given certain conditions. This definitely surprised me, and I would love to know more about it, but the lecturer quickly moves on, so my question is as follows: under what conditions can one add the total time derivative of a function which depends on the particle's generalized velocities in addition to its generalized coordinates and time without affecting the particle's equations of motion?

Answer

I) In general, it is true that if we plug a local Lagrangian

$$\tag{1} L\quad \longrightarrow \quad \tilde{L}~=~L+\frac{df}{dt}$$

modified with a total derivative term into the Euler-Lagrange expression

$$\tag{2} \sum_{n} \left(-\frac{d}{dt}\right)^n \frac{\partial \tilde{L}}{\partial q^{(n)}}~=~\sum_{n} \left(-\frac{d}{dt}\right)^n \frac{\partial L}{\partial q^{(n)}}, $$

it would lead to identically the same Euler-Lagrange expression without any restrictions on $L$ and $f$.

II) The caveat is that the Euler-Lagrange expression (2) is only$^1$ physically legitimate, if it has a physical interpretation as a variational/functional derivative of an action principle. However, existence of a variational/functional derivative is a non-trivial issue, which relies on well-posed boundary conditions for the variational problem. In plain English: Boundary conditions are needed in order to justify integration by parts. See also e.g. my related Phys.SE answers here & here.

III) A Lagrangian $L(q,\dot{q},\ldots, q^{(N)},t)$ of order $N$ leads to equation of motion of order $\leq 2N$. Typically we require the Lagrangian $L(q,\dot{q},t)$ to be of first order $N=1$. See e.g. this and this Phys.SE posts.

IV) Concretely, let us assume that we are given a first-order Lagrangian $L(q,\dot{q},t)$. If one redefines the Lagrangian with a total derivative

$$\tag{3} \tilde{L}(q, \dot{q}, \ddot{q}, t)~=~L(q, \dot{q}, t)+\frac{d}{dt}f(q, \dot{q}, t), $$

where $f(q, \dot{q}, t)$ depends on velocity $\dot{q}$, then the new Lagrangian $\tilde{L}(q, \dot{q}, \ddot{q}, t)$ may also depend on acceleration $\ddot{q}$, i.e. be of higher order.

V) With a higher-order $\tilde{L}(q, \dot{q}, \ddot{q}, t)$, we might have to impose additional boundary conditions in order to derive Euler-Lagrange equations from the principle of a stationary action by use of repeated integrations by parts.

VI) It seems that Prof. V. Balakrishnan in the video has the issues IV and V in mind when he spoke of 'putting further conditions' on the system. Finally, OP may also find this Phys.SE post interesting.

--

$^1$ Here we ignore derivations of Lagrange equations directly from Newton's laws, i.e. without the use of the principle of a stationary action, such as e.g. this Phys.SE post, because they usually don't involve redefinitions (3).