A Balanced Approach of Rapid Genetic Exploration and Surrogate Exploitation for Hyperparameter Optimization

This paper introduces a novel approach to hyperparameter optimization (HPO), proposing a methodology that balances exploration and exploitation to enhance optimization performance. While evolutionary algorithms (EAs) have shown potential in HPO, they often struggle with effective exploitation. To address this limitation, we propose an improved hyperparameter optimization (HPO) framework that integrates a linear surrogate model into the genetic algorithm (GA). The GA’ss flexible structure allows for seamless integration of multiple optimization strategies, and the surrogate model significantly boosts its exploitation capabilities. Specifically, we achieved an average performance improvement of 1.89% (max 6.55%, min −3.45%) over the existing state-of-the-art HPO strategy.