In self-supervised learning, a model is trained to solve a pretext task,\nusing a data set whose annotations are created by a machine. The objective is\nto transfer the trained weights to perform a downstream task in the target\ndomain. We critically examine the most notable pretext tasks to extract\nfeatures from image data and further go on to conduct experiments on resource\nconstrained networks, which aid faster experimentation and deployment. We study\nthe performance of various self-supervised techniques keeping all other\nparameters uniform. We study the patterns that emerge by varying model type,\nsize and amount of pre-training done for the backbone as well as establish a\nstandard to compare against for future research. We also conduct comprehensive\nstudies to understand the quality of representations learned by different\narchitectures.\n