We tackle the problem of discovering novel classes in an image collection\ngiven labelled examples of other classes. This setting is similar to\nsemi-supervised learning, but significantly harder because there are no\nlabelled examples for the new classes. The challenge, then, is to leverage the\ninformation contained in the labelled images in order to learn a\ngeneral-purpose clustering model and use the latter to identify the new classes\nin the unlabelled data. In this work we address this problem by combining three\nideas: (1) we suggest that the common approach of bootstrapping an image\nrepresentation using the labeled data only introduces an unwanted bias, and\nthat this can be avoided by using self-supervised learning to train the\nrepresentation from scratch on the union of labelled and unlabelled data; (2)\nwe use rank statistics to transfer the model's knowledge of the labelled\nclasses to the problem of clustering the unlabelled images; and, (3) we train\nthe data representation by optimizing a joint objective function on the\nlabelled and unlabelled subsets of the data, improving both the supervised\nclassification of the labelled data, and the clustering of the unlabelled data.\nWe evaluate our approach on standard classification benchmarks and outperform\ncurrent methods for novel category discovery by a significant margin.\n