newtonian mechanics - Why does a system try to minimize potential energy?

In mechanics problems, especially one-dimensional ones, we talk about how a particle goes in a direction to minimize potential energy. This is easy to see when we use cartesian coordinates: For example, $-\frac{dU}{dx}=F$ (or in the multidimensional case, the gradient), so the force will go in the direction of minimizing the potential energy.

However, it becomes less clear in other cases. For example, I read a problem that involved a ball attached to a pivot, so it could only rotate. It was then claimed that the ball would rotate towards minimal potential energy, however $-\frac{dU}{d\theta} \neq F$! I think in this case it might be equal to torque, which would make their reasoning correct, but it seems like regardless of the degrees of freedom of the problem, it is always assumed that the forces act in a way such that the potential energy is minimized. Could someone give a good explanation for why this is?

Edit: I should note that I typed this in google and found this page. where it states that minimizing potential energy and increasing heat increases entropy. For one, this isn't really an explanation because it doesn't state why it increases entropy. Also, if possible, I would like an explanation that doesn't involve entropy. But if it is impossible to make a rigorous argument that doesn't involve entropy then using entropy is fine.

As a side note, how does this relate to Hamilton's Principle?

Answer

This is a physical rather than a mathematical justification - ignore my answer if that isn't what you wanted!

All systems have some thermal motion so they explore the phase space in their immediate vicinity. If there is a nearby point with a free energy lower by some amount $\Delta G$ then the relative probability of finding the system at that point will be $\exp(-\Delta G/RT)$. So if the energy is minimised by moving to that point, i.e.$\Delta G < 0$, we just have to wait and we'll find the system has moved there. The only place in phase space the system won't move is when the free energy is at a (local) minimum. That's why a system always (locally) minimises its free energy if you wait long enough.