Multi-agent reinforcement learning has been successfully applied to\nfully-cooperative and fully-competitive environments, but little is currently\nknown about mixed cooperative/competitive environments. In this paper, we focus\non a particular class of multi-agent mixed cooperative/competitive stochastic\ngames called Markov Potential Games (MPGs), which include cooperative games as\na special case. Recent results have shown that independent policy gradient\nconverges in MPGs but it was not known whether Independent Natural Policy\nGradient converges in MPGs as well. We prove that Independent Natural Policy\nGradient always converges in the last iterate using constant learning rates.\nThe proof deviates from the existing approaches and the main challenge lies in\nthe fact that Markov Potential Games do not have unique optimal values (as\nsingle-agent settings exhibit) so different initializations can lead to\ndifferent limit point values. We complement our theoretical results with\nexperiments that indicate that Natural Policy Gradient outperforms Policy\nGradient in routing games and congestion games.\n