The present study unfolds the relative importance of $N/Z$, shell effects, and thermal fluctuations in shaping the temperature dependence of the isovector giant dipole resonance (IVGDR) width. To this end, we measured the γ-ray spectra from excited ${}^{62,68}\mathrm{Zn}$ populated through 4He ${+}^{58,64}$Ni reactions at beam energies of 28 MeV and 40 MeV, and compared the corresponding IVGDR widths with those for nuclei close to the doubly magic 56Ni and moderately away from it. The large-area modular BaF 2 detector array was used to detect high-energy γ rays ( $E_{\gamma }>4$ MeV). The Bayesian inference approach was combined with the statistical model analysis to extract the IVGDR parameters from the measured spectra. Calculation of IVGDR widths was performed using various theoretical approaches, specifically the thermal shape fluctuation model (TSFM) with microscopic energy density functional inputs. TSFM calculation was also performed using the free energy surfaces from the deformed liquid drop model. For 68Zn, the width is observed to increase closely following the TSFM prediction. In contrast, suppression of the width is noticed for 62Zn at low temperatures, similar to other nearby nuclei with N and/or Z closer to 28. Moreover, 68Zn indicates an early onset of the saturation in the IVGDR width, mimicking structural changes appropriately comprehended by the microscopic calculations. These findings suggest that the relative influence of microscopic effects and thermal broadening in the IVGDR width is strongly governed by the proximity of the decaying nucleus to magicity.