Climate Change Policy Exploration using Reinforcement Learning

Climate Change is an incredibly complicated problem that humanity faces. When many variables interact with each other, it can be difficult for humans to grasp the causes and effects of the very large-scale problem of climate change. The climate is a dynamical system, where small changes can have considerable and unpredictable repercussions in the long term. Understanding how to nudge this system in the right ways could help us find creative solutions to climate change. In this research, we combine Deep Reinforcement Learning and a World-Earth system model to find, and explain, creative strategies to a sustainable future. This is an extension of the work from Strnad et al. [1], where we extend on the method and analysis, by taking multiple directions. We use four different Reinforcement Learning agents varying in complexity to probe the environment in different ways and to find various strategies. The environment is a low-complexity World Earth system model where the goal is to reach a future where all the energy for the economy is produced by renewables by enacting different policies. We use a reward function based on planetary boundaries that we modify to force the agents to find a wider range of strategies. To favour applicability, we slightly modify the environment, by injecting noise and making it fully observable, to understand the impacts of these factors on the learning of the agents. We discover that our Reinforcement Learning agents learn to reach a carbon-free future, but for some initial conditions, strong feedback loops in the system prevent the agents from being able to control the environment. We find that in this simplistic model, the growth of the economy is a significant feature for the agents when deciding which policies to enact. While this approach shows some promise, there is much more to be done for this framework to be applicable to climate related policy. We hope this novel research will inspire more work on this topic.