Numerical time integration is fundamental to simulating initial and boundary value problems across science, engineering, and even finance. Traditionally, time integration schemes require adaptive time-stepping to ensure computational speed and sufficient accuracy. Either directly or indirectly, these schemes require a certain degree of smoothness of the underlying dynamics. However, a significant number of problems in nature are nonsmooth. In this case, conventional schemes struggle to accurately integrate the solution or often fail to do so in a timely manner. In this work, we use an alternative approach based on Reinforcement Learning (RL) to select the optimal time step for any time integrator method, balancing computational speed and accuracy. Our RL approach learns to optimally adapt its time step through training, even in the presence of nonsmoothness. This capability makes it a prominent method for integrating a large spectrum of challenging dynamical systems. To highlight its potential, we consider three model problems of increasing complexity. We begin with a nonlinear control method (sliding mode control), then proceed to an electrical circuit with a diode, and finally address a frictional instability problem that models a seismic fault with Coulomb friction. These examples collectively demonstrate the robustness of our strategy in handling nonsmooth, set-valued dynamics across diverse spatiotemporal scales–a notoriously challenging scenario for conventional numerical time integrators. Our results indicate that the RL-based adaptive integrator can learn an optimal time-integration strategy and achieve considerable speed-ups in relatively simple case scenarios, as well as a tenfold speed-up in the most challenging cases with spatiotemporal complex dynamics. These promising findings suggest that our approach can provide a novel and efficient alternative for time integration in various dynamical systems.