Soft Actor-Critic Reinforcement Learning for Turbulence Modelling in Dense Gas Flows
Researchers
Research Areas
Compressible Flows, Computational Fluid Dynamics, Turbulence Modelling, Machine Learning
Project Brief
The proposed PhD thesis is focused on the development of novel turbulence closures for the Reynolds-Averaged Navier-Stokes (RANS) equations applied to the description of dense gas flows in geometries of industrial interest (ORC turbines). The complex thermodynamic description of dense gases motivates the use of machine learning techniques to derive relevant models. The first stage of the work will adapt to RANS modelling a supervised learning methodology previously developed and applied by the research team for Large Eddy Simulation. This development will take advantage of reference High-Fidelity LES results produced with the solver AVBP developed at CERFACS. All RANS simulations needed to build models relevant for dense gas flows will be performed using the open-source SU2 solver. In a second stage, an even more innovative approach will be tackled through the exploration of non-supervised learning. While most (if not all) methods currently proposed to improve turbulence modelling in Fluid Dynamics are intrinsically supervised (they require a form of high-fidelity results to be tuned), alternative methods exist which do not require supervision, among which Reinforcement Learning (RL). RL relies on an agent which dynamically interacts with its environment (here the turbulent solver) making use of rewards gathered through its evolution and possibly following a policy in order to eventually produce an accurate representation of local turbulence statistics. RL has led to impressive progress for applications such as self-driving cars or chess but still needs to be properly transposed to the CFD domain, where its lack of supervision is especially attractive since bypassing the need high-fidelity data. This third project is focused on Double Agent or Soft Actor-Critic RL which combines two independent predictions of Q and an artificial neural network dedicated to the policy. The models developed will be assessed on various configurations including one of interest for ENOGIA, a French ORC turbine manufacturer, partner of the project.