Single image super-resolution (SISR) is an ill-posed problem with an\nindeterminate number of valid solutions. Solving this problem with neural\nnetworks would require access to extensive experience, either presented as a\nlarge training set over natural images or a condensed representation from\nanother pre-trained network. Perceptual loss functions, which belong to the\nlatter category, have achieved breakthrough success in SISR and several other\ncomputer vision tasks. While perceptual loss plays a central role in the\ngeneration of photo-realistic images, it also produces undesired pattern\nartifacts in the super-resolved outputs. In this paper, we show that the root\ncause of these pattern artifacts can be traced back to a mismatch between the\npre-training objective of perceptual loss and the super-resolution objective.\nTo address this issue, we propose to augment the existing perceptual loss\nformulation with a novel content loss function that uses the latent features of\na discriminator network to filter the unwanted artifacts across several levels\nof adversarial similarity. Further, our modification has a stabilizing effect\non non-convex optimization in adversarial training. The proposed approach\noffers notable gains in perceptual quality based on an extensive human\nevaluation study and a competent reconstruction fidelity when tested on\nobjective evaluation metrics.\n