We systematically calculate the radiative corrections of order $\alpha /\pi$ to elastic muon-proton scattering at low momentum transfers. These include vacuum polarization, photon-loop form factors of the muon and the proton, two-photon exchange corrections and soft photon radiation. In particular, we discuss these corrections for the kinematics of the upcoming AMBER experiment with a 100 GeV muon beam. It is found that for the ratio to the Born cross section, only the minor terms from the photon-loop form factors of the proton and two-photon exchange depend on the proton structure as predetermined by the strong interactions. The inclusion of soft photon radiation (below 20 MeV), which as such is mandatory to cancel the infrared divergences arising from virtual photon loops, points to a prominent role of bremsstrahlung among the radiative corrections of order $\alpha /\pi$. Therefore, the calculation of the process ${\mu }^{\mp }p\to {\mu }^{\mp }p\gamma$ must be extended beyond the soft photon approximation and tailored to the specific experimental conditions for a proper analysis of the upcoming AMBER data.