Self-supervised contrastive representation learning has proved incredibly\nsuccessful in the vision and natural language domains, enabling\nstate-of-the-art performance with orders of magnitude less labeled data.\nHowever, such methods are domain-specific and little has been done to leverage\nthis technique on real-world tabular datasets. We propose SCARF, a simple,\nwidely-applicable technique for contrastive learning, where views are formed by\ncorrupting a random subset of features. When applied to pre-train deep neural\nnetworks on the 69 real-world, tabular classification datasets from the\nOpenML-CC18 benchmark, SCARF not only improves classification accuracy in the\nfully-supervised setting but does so also in the presence of label noise and in\nthe semi-supervised setting where only a fraction of the available training\ndata is labeled. We show that SCARF complements existing strategies and\noutperforms alternatives like autoencoders. We conduct comprehensive ablations,\ndetailing the importance of a range of factors.\n