Finding valuable training data points for deep neural networks has been a\ncore research challenge with many applications. In recent years, various\ntechniques for calculating the "value" of individual training datapoints have\nbeen proposed for explaining trained models. However, the value of a training\ndatapoint also depends on other selected training datapoints - a notion that is\nnot explicitly captured by existing methods. In this paper, we study the\nproblem of selecting high-value subsets of training data. The key idea is to\ndesign a learnable framework for online subset selection, which can be learned\nusing mini-batches of training data, thus making our method scalable. This\nresults in a parameterized convex subset selection problem that is amenable to\na differentiable convex programming paradigm, thus allowing us to learn the\nparameters of the selection model in end-to-end training. Using this framework,\nwe design an online alternating minimization-based algorithm for jointly\nlearning the parameters of the selection model and ML model. Extensive\nevaluation on a synthetic dataset, and three standard datasets, show that our\nalgorithm finds consistently higher value subsets of training data, compared to\nthe recent state-of-the-art methods, sometimes ~20% higher value than existing\nmethods. The subsets are also useful in finding mislabelled training data. Our\nalgorithm takes running time comparable to the existing valuation functions.\n