Deep neural networks have shown impressive performance in supervised\nlearning, enabled by their ability to fit well to the provided training data.\nHowever, their performance is largely dependent on the quality of the training\ndata and often degrades in the presence of noise. We propose a principled\napproach for tackling label noise with the aim of assigning importance weights\nto individual instances and class labels. Our method works by formulating a\nclass of constrained optimization problems that yield simple closed form\nupdates for these importance weights. The proposed optimization problems are\nsolved per mini-batch which obviates the need of storing and updating the\nweights over the full dataset. Our optimization framework also provides a\ntheoretical perspective on existing label smoothing heuristics for addressing\nlabel noise (such as label bootstrapping). We evaluate our method on several\nbenchmark datasets and observe considerable performance gains in the presence\nof label noise.\n