Prior works have found it beneficial to combine provably noise-robust loss\nfunctions e.g., mean absolute error (MAE) with standard categorical loss\nfunction e.g. cross entropy (CE) to improve their learnability. Here, we\npropose to use Jensen-Shannon divergence as a noise-robust loss function and\nshow that it interestingly interpolate between CE and MAE with a controllable\nmixing parameter. Furthermore, we make a crucial observation that CE exhibit\nlower consistency around noisy data points. Based on this observation, we adopt\na generalized version of the Jensen-Shannon divergence for multiple\ndistributions to encourage consistency around data points. Using this loss\nfunction, we show state-of-the-art results on both synthetic (CIFAR), and\nreal-world (e.g., WebVision) noise with varying noise rates.\n