Applying deep reinforcement learning to active flow control in turbulent conditions

Machine learning has recently become a promising technique in fluid mechanics, especially for active flow control (AFC) applications. A recent work [J. Fluid Mech. (2019), vol. 865, pp. 281-302] has demonstrated the feasibility and efficiency of deep reinforcement learning (DRL) in performing AFC for a circular cylinder at a low Reynolds number, i.e., $Re = 100$. As a follow-up study, we investigate the same DRL-based AFC problem at an intermediate Reynolds number ($Re = 1000$), where the flow's strong nonlinearity poses great challenges to the control. The DRL agent interacts with the flow via receiving information from velocity probes and determines the strength of actuation realized by a pair of synthetic jets. A remarkable drag reduction of around $30\%$ is achieved. By analysing turbulent quantities, it is shown that the drag reduction is obtained via elongating the recirculation bubble and reducing turbulent fluctuations in the wake. This study constitutes, to our knowledge, the first successful application of DRL to AFC in turbulent conditions, and it is, therefore, a key milestone in progressing towards the control of fully turbulent, chaotic, multimodal, and strongly nonlinear flow configurations.