We resolve subtleties in calculating the post-Minksowskian dynamics of binary systems, as a spin expansion, from massive scattering amplitudes of fixed finite spin. In particular, the apparently ambiguous spin Casimir terms can be fully determined from the gradient of the spin-diagonal part of the amplitudes with respect to = − ( +1) , using an interpolation between massive amplitudes with different spin representations. From two-loop amplitudes of spin-0 and spin-1 particles minimally coupled to gravity, we extract the spin Casimir terms in the conservative scattering angle between a spinless and a spinning black hole at $$ \mathcal{O} $$ ( ), finding agreement with known results in the literature. This completes an earlier study [Phys. Rev. Lett. 130 (2023), 021601] that calculated the non-Casimir terms from amplitudes. We also illustrate our methods using a model of spinning bodies in electrodynamics, finding agreement between scattering amplitude predictions and classical predictions in a electromagnetic background up to $$ \mathcal{O} $$ ( ). For both gravity and electrodynamics, the finite part of the amplitude coincides with the two-body radial action in the aligned spin limit, generalizing the amplitude-action relation beyond the spinless case. Surprisingly, the two-loop amplitude displays a hidden spin-shift symmetry in the probe limit, which was previously observed at one loop. We conjecture that the symmetry holds to all orders in the coupling constant and is a consequence of integrability of Kerr orbits in the probe limit at the first few orders in spin.