Unsupervised and self-supervised learning approaches have become a crucial\ntool to learn representations for downstream prediction tasks. While these\napproaches are widely used in practice and achieve impressive empirical gains,\ntheir theoretical understanding largely lags behind. Towards bridging this gap,\nwe present a unifying perspective where several such approaches can be viewed\nas imposing a regularization on the representation via a learnable function\nusing unlabeled data. We propose a discriminative theoretical framework for\nanalyzing the sample complexity of these approaches, which generalizes the\nframework of (Balcan and Blum, 2010) to allow learnable regularization\nfunctions. Our sample complexity bounds show that, with carefully chosen\nhypothesis classes to exploit the structure in the data, these learnable\nregularization functions can prune the hypothesis space, and help reduce the\namount of labeled data needed. We then provide two concrete examples of\nfunctional regularization, one using auto-encoders and the other using masked\nself-supervision, and apply our framework to quantify the reduction in the\nsample complexity bound of labeled data. We also provide complementary\nempirical results to support our analysis.\n