Quantum Local Search (QLS) is a promising approach that employs small-scale quantum computers to tackle large combinatorial optimization problems through local search on quantum hardware. However, the random selection of the sub-problem to solve in QLS may not be efficient. In this study, we propose a reinforcement learning (RL) based approach for training an agent to improve sub-problem selection in QLS beyond random selection. Our results demonstrate that the RL agent effectively enhances the average approximation ratio of QLS on fully-connected random Ising problems, indicating the potential of combining RL techniques with Noisy Intermediate-scale Quantum (NISQ) algorithms. This research opens a promising direction for integrating RL into quantum computing to enhance the performance of optimization tasks.