Medical Image Analysis is currently experiencing a paradigm shift due to Deep\nLearning. This technology has recently attracted so much interest of the\nMedical Imaging community that it led to a specialized conference in `Medical\nImaging with Deep Learning' in the year 2018. This article surveys the recent\ndevelopments in this direction, and provides a critical review of the related\nmajor aspects. We organize the reviewed literature according to the underlying\nPattern Recognition tasks, and further sub-categorize it following a taxonomy\nbased on human anatomy. This article does not assume prior knowledge of Deep\nLearning and makes a significant contribution in explaining the core Deep\nLearning concepts to the non-experts in the Medical community. Unique to this\nstudy is the Computer Vision/Machine Learning perspective taken on the advances\nof Deep Learning in Medical Imaging. This enables us to single out `lack of\nappropriately annotated large-scale datasets' as the core challenge (among\nother challenges) in this research direction. We draw on the insights from the\nsister research fields of Computer Vision, Pattern Recognition and Machine\nLearning etc.; where the techniques of dealing with such challenges have\nalready matured, to provide promising directions for the Medical Imaging\ncommunity to fully harness Deep Learning in the future.\n