Robust Deep Reinforcement Learning for Volt-VAR Optimization in Active Distribution System Under Uncertainty

The deep reinforcement learning (DRL) based Volt-VAR optimization (VVO) methods have been widely studied for active distribution networks (ADNs). Most approaches neglect power injection uncertainties due to increased levels and complexities of distributed energy resources (DERs) and stochastic loads, such as electric vehicles. This article proposes a robust deep reinforcement learning (RDRL) framework for VVO via a robust deep deterministic policy gradient (DDPG) algorithm that accounts and characterizes the uncertainty. The algorithm effectively manages hybrid action spaces, generated by discrete controls such as capacitor switching, tap changes and continuous controls from smart inverters and synchronous condensers. The algorithm is designed to handle uncertainties by quantifying uncertainty sets with conformal prediction and modeling uncertainties as adversarial attacks. An innovative training approach is discussed using high fidelity models to produce simulated training sets that are rich in events that are crucial for the proper training of the proposed algorithm. Numerical results are presented on three IEEE test cases demonstrating the efficiency and zero-violation of the proposed robust DDPG with soft safety constraints against uncertainties and compared to benchmark algorithms.