We study the creation and evolution of cosmological perturbations in renormalizable quadratic gravity with a Weyl term. We adopt a prescription that implies the stability of the vacuum at the price of introducing a massive spin-two ghost state, leading to the loss of unitarity. The theory may still be predictive regardless the interpretation of non-unitary processes provided that their rate is negligible compared to the Universe expansion rate. This implies that the ghost is effectively stable. In such a setup, there are two scalar degrees of freedom excited during inflation. The first one is the usual curvature perturbation whose power spectrum appears to coincide with that of single-field inflation. The second one is a scalar component of the ghost encoded in the shift vector of the metric in the uniform inflaton gauge. The amplitudes of primordial tensor and vector perturbations are strongly suppressed. After inflation the ghost field starts to oscillate and its energy density shortly becomes dominant in the Universe. For all ghost masses allowed by laboratory constraints ghosts should have ``overclosed'' the Universe at temperatures higher than that of primordial nucleosynthesis. Thus, the model with the light Weyl ghost is ruled out.