symmetry - Noether's current expression in Peskin and Schroeder

In the second chapter of Peskin and Schroeder, An Introduction to Quantum Field Theory, it is said that the action is invariant if the Lagrangian density changes by a four-divergence. But if we calculate any change in Lagrangian density we observe that under the conditions of equation of motion being satisfied, it only changes by a four-divergence term.

If ${\cal L}(x)$ changes to ${\cal L}(x) + \alpha \partial_\mu J^{\mu} (x)$ then action is invariant. But isn't this only in the case of extremization of action to obtain Euler-Lagrange equations.

Comparing this to $\delta {\cal L}$

$$\alpha \delta {\cal L} = \frac{\partial {\cal L}}{\partial \phi} (\alpha \delta \phi) + \frac{\partial {\cal L}}{\partial \partial_{\mu}\phi} \partial_{\mu}(\alpha \delta \phi)$$

$$= \alpha \partial_\mu \left(\frac{\partial {\cal L}}{\partial \partial_{\mu}\phi} \delta \phi \right) + \alpha \left[ \frac{\partial {\cal L}}{\partial \phi} - \partial_\mu \left(\frac{\partial {\cal L}}{\partial \partial_{\mu}\phi} \right) \right] \delta \phi.$$

Getting the second term to zero assuming application of equations of motion. Doesn't this imply that the noether's current itself is zero, rather than its derivative? That is:

$$J^{\mu} (x) = \frac{\partial {\cal L}}{\partial \partial_{\mu}\phi} \delta \phi .$$

I add that my doubt is why changing ${\cal L}$ by a four divergence term lead to invariance of action globally when that idea itself was derived while extremizing the action which I assume is a local extremization and not a global one.