Generalization in Reinforcement Learning for Radio Access Networks

Modern radio access networks (RANs) operate in highly dynamic and heterogeneous environments, where hand-tuned, rule-based radio resource management (RRM) algorithms frequently underperform. While reinforcement learning (RL) can surpass these heuristics in constrained scenarios, the unpredictable nature of radio channels and the diversity of cell deployments introduce substantial generalization challenges. Data-driven policies often overfit to their training distributions, leading to degraded performance when applied to unseen conditions. To address this, we propose a generalization-focused RL framework for RAN control that: (i) robustly reconstructs dynamically varying states from partial and noisy observations, while encoding static and semi-static information—such as radio nodes, cell attributes, and their topology—through graph representations; (ii) applies extensive domain randomization to broaden the training distribution; and (iii) distributes data generation across multiple actors while centralizing training in a cloud-compatible architecture aligned with O-RAN principles. Although training generalizable policies increases computational and data-management complexity, our distributed design mitigates these costs by scaling data collection and training across heterogeneous network conditions. Applied to downlink link adaptation (LA) across five <inline-formula> <tex-math notation="LaTeX">$5^{\text {th}}$ </tex-math></inline-formula> Generation (5G) benchmarks, the resulting generalized RL policy achieves approximately 10% higher average cell throughput and spectral efficiency than the state-of-the-art LA baseline in full-buffer multiple input multiple output (MIMO) and massive MIMO (mMIMO) scenarios, and approximately 20% under high-mobility conditions. Furthermore, it matches the performance of specialized RL policies in full-buffer traffic while delivering up to <inline-formula> <tex-math notation="LaTeX">$4\times $ </tex-math></inline-formula> and <inline-formula> <tex-math notation="LaTeX">$2\times $ </tex-math></inline-formula> throughput gains in enhanced mobile broadband (eMBB) and mixed-traffic benchmarks, respectively. In larger deployments with nine cells, graph attention network (GAT) models achieve an additional 30% throughput gain over multi-layer perceptron (MLP) baselines, underscoring the effectiveness of attention-based architectures in larger network settings. These performance gains, combined with the scalable and distributed learning architecture, provide a practical foundation for AI-native <inline-formula> <tex-math notation="LaTeX">$6^{\text {th}}$ </tex-math></inline-formula> Generation (6G) RANs, enabling the deployment of a single, generalizable RL agent that operates network-wide and consistently outperforms traditional heuristics.