We use the functional renormalization group equation for the effective average action to study the fixed point structure of gravity-fermion systems on a curved background spacetime. We approximate the effective average action by the Einstein-Hilbert action supplemented by a fermion kinetic term and a coupling of the fermion bilinears to the spacetime curvature. The latter interaction is singled out based on a “smart truncation building principle”. The resulting renormalization group flow possesses two families of interacting renormalization group fixed points extending to any number of fermions. The first family exhibits an upper bound on the number of fermions for which the fixed points could provide a phenomenologically interesting high-energy completion via the asymptotic safety mechanism. The second family comes without such a bound. The inclusion of the non-minimal gravity-matter interaction is crucial for discriminating the two families. Our work also clarifies the origin of the strong regulator-dependence of the fixed point structure reported in earlier literature and we comment on the relation of our findings to studies of the same system based on a vertex expansion of the effective average action around a flat background spacetime.