I investigate a class of models with scalar and pseudoscalar solutions to the $g-2$ anomaly for both the muon and the electron over the mass range of perturbativity ($m_{\varphi }\lesssim 50$ GeV), with Yukawa couplings proportional to the lepton’s mass. In particular, I investigate the constraints from BaBar, beam dump experiments, $Z$ decay measured quantities, LEP mono-$\gamma$ searches, $ee⟶\tau \tau \left(\gamma \right)$ searches, and solar and horizontal branch (HB) star bounds. For a pseudoscalar, I find that no region in the parameter space can simultaneously provide a solution for both the electron and the muon anomalies while maintaining the required form of the couplings, and therefore, the pseudoscalar solution is disfavored. On the other hand, I find for the scalar case that there is an open window above ~30 MeV in the allowed region, but with significant tension with experiment for the region $m_{\varphi }\gtrsim 10\text{GeV}$. In addition, there is a smaller window between ~350 KeV and 1 MeV that is not ruled out by cosmological observations. Part of the first open window is expected to be covered by the proposed NA64 experiment. Similar analysis can be readily applied to other proposed solutions to the anomaly, such as solutions with $Z^{\prime }$ or with the dark photon.