The energy-momentum tensor of a ferromagnet derived according to the standard prescription of Noether's theorem has a major flaw: the term originating from the spin Berry phase is gauge dependent. As a consequence, some physical quantities computed from the tensor show unphysical behavior. For example, the presence of a spin-polarized current does not affect the energy of the domain wall in the commonly accepted gauge, which implies, incorrectly, the absence of the adiabatic spin torque. In other gauges, the spin torque shows unphysical glitches occurring when the plane of magnetization crosses the Dirac string associated with a magnetic monopole in spin space. We derive a gauge-invariant energy-momentum tensor that is free from these artifacts but requires the addition of an extra spatial dimension, with the ferromagnet living on its boundary. It can be obtained most directly from the Wess-Zumino action for spins, which relies on the same extra dimension.