We relate the counting of refined BPS numbers on compact elliptically fibred Calabi-Yau threefolds $\textit{X}$ to Wilson loop expectations values in the gauge theories that emerge in various rigid local limits of the 5d supergravity theory defined by M-theory compactification on $\textit{X}$. In these local limits $\textit{X}$ the volumes of curves in certain classes go to infinity, the corresponding very massive M2-brane states can be treated as Wilson loop particles and the refined topological string partition function on $\textit{X}$ becomes a sum of terms proportional to associated refined Wilson loop expectation values. The resulting ansatz for the complete refined topological partition function on $\textit{X}$ is written in terms of the proportionality coefficients which depend only on the $\textit{ϵ}$ deformations and the Wilson loop expectations values which satisfy holomorphic anomaly equations. Since the ansatz is quite restrictive and can be further constrained by the one-form symmetries and E-string type limits for large base curves, we can efficiently evaluate the refined BPS numbers on $\textit{X}$, which we do explicitly for local gauge groups up to rank three and $\textit{h}$ ($\textit{X}$) = 5. These refined BPS numbers pass an impressive number of consistency checks imposed by the direct counting of these numbers using the moduli space of one dimensional stable sheaves on $\textit{X}$ and give us numerical predictions for the complex structure dependency of the refined BPS numbers.