We consider the construction of genus zero correlators of SU(N ) k  WZW models involving two Kac-Moody primaries in the fundamental and two in the anti-fundamental representation from modular averaging of the contribution of the vacuum conformal block. We perform the averaging by two prescriptions — averaging over the stabiliser group associated with the correlator and averaging over the entire modular group. For the first method, in cases where we find the orbit of the vacuum conformal block to be finite, modular averaging reproduces the exact result for the correlators. In other cases, we perform the modular averaging numerically, the results are in agreement with the exact answers. Construction of correlators from averaging over whole of the modular group is more involved. Here, we find some examples where modular averaging does not reproduce the correlator. We find a close relationship between the modular averaging sums of the theories related by level-rank duality. We establish a one to one correspondence between elements of the orbits of the vacuum conformal blocks of dual theories. The contributions of paired terms to their respective correlators are simply related. One consequence of this is that the ratio between the OPE coefficients associated with dual correlators can be obtained analytically without performing the sums involved in the modular averagings. The pairing of terms in the modular averaging sums for dual theories suggests an interesting connection between level-rank duality and semi-classical holographic computations of the correlators in the theories.