Hiring Institution
École Centrale de Lyon (EC Lyon)

PhD-Awarding Institutions
École Centrale de Lyon (EC Lyon)
Queensland University of Technology (QUT)

Download the full Position Description in PDF

Option 1

Q-based reinforcement learning for turbulence modelling in dense gas flows

The proposed PhD thesis is focused on the development of novel turbulence closures for the Reynolds-Averaged Navier-Stokes (RANS) equations applied to the description of dense gas flows in geometries of industrial interest (ORC turbines). The complex thermodynamic description of dense gases motivates the use of machine learning techniques to derive relevant models. The first stage of the work will adapt to RANS modelling a supervised learning methodology previously developed and applied by the research team for Large Eddy Simulation. This development will take advantage of reference High-Fidelity LES results produced with the solver AVBP developed at CERFACS. All RANS simulations needed to build models relevant for dense gas flows will be performed using the open-source SU2 solver. In a second stage, an even more innovative approach will be tackled through the exploration of non-supervised learning. While most (if not all) methods currently proposed to improve turbulence modelling in Fluid Dynamics are intrinsically supervised (they require a form of high-fidelity results to be tuned), alternative methods exist which do not require supervision, among which Reinforcement Learning (RL). RL relies on an agent which dynamically interacts with its environment (here the turbulent solver) making use of rewards gathered through its evolution and possibly following a policy in order to eventually produce an accurate representation of local turbulence statistics. RL has led to impressive progress for applications such as self-driving cars or chess but still needs to be properly transposed to the CFD domain, where its lack of supervision is especially attractive since bypassing the need high-fidelity data. This first project is focused on Q-based RL where the agent learns to predict a future value from a known state and acts consequently. The models developed will be assessed on various configurations including one of interest for ENOGIA, a French ORC turbine manufacturer, partner of the project.

Option 2

Policy-based reinforcement learning for turbulence modelling in dense gas flows

The proposed PhD thesis is focused on the development of novel turbulence closures for the Reynolds-Averaged Navier-Stokes (RANS) equations applied to the description of dense gas flows in geometries of industrial interest (ORC turbines). The complex thermodynamic description of dense gases motivates the use of machine learning techniques to derive relevant models. The first stage of the work will adapt to RANS modelling a supervised learning methodology previously developed and applied by the research team for Large Eddy Simulation. This development will take advantage of reference High-Fidelity LES results produced with the solver AVBP developed at CERFACS. All RANS simulations needed to build models relevant for dense gas flows will be performed using the open-source SU2 solver. In a second stage, an even more innovative approach will be tackled through the exploration of non-supervised learning. While most (if not all) methods currently proposed to improve turbulence modelling in Fluid Dynamics are intrinsically supervised (they require a form of high-fidelity results to be tuned), alternative methods exist which do not require supervision, among which Reinforcement Learning (RL). RL relies on an agent which dynamically interacts with its environment (here the turbulent solver) making use of rewards gathered through its evolution and possibly following a policy in order to eventually produce an accurate representation of local turbulence statistics. RL has led to impressive progress for applications such as self-driving cars or chess but still needs to be properly transposed to the CFD domain, where its lack of supervision is especially attractive since bypassing the need high-fidelity data. This second project is focused on policy-based RL where the agent learns from a policy which is not directly correlated with the value of Q. The models developed will be assessed on various configurations including one of interest for ENOGIA, a French ORC turbine manufacturer, partner of the project.

Option 3

Soft Actor-Critic reinforcement learning for turbulence modelling in dense gas flows

The proposed PhD thesis is focused on the development of novel turbulence closures for the Reynolds-Averaged Navier-Stokes (RANS) equations applied to the description of dense gas flows in geometries of industrial interest (ORC turbines). The complex thermodynamic description of dense gases motivates the use of machine learning techniques to derive relevant models. The first stage of the work will adapt to RANS modelling a supervised learning methodology previously developed and applied by the research team for Large Eddy Simulation. This development will take advantage of reference High-Fidelity LES results produced with the solver AVBP developed at CERFACS. All RANS simulations needed to build models relevant for dense gas flows will be performed using the open-source SU2 solver. In a second stage, an even more innovative approach will be tackled through the exploration of non-supervised learning. While most (if not all) methods currently proposed to improve turbulence modelling in Fluid Dynamics are intrinsically supervised (they require a form of high-fidelity results to be tuned), alternative methods exist which do not require supervision, among which Reinforcement Learning (RL). RL relies on an agent which dynamically interacts with its environment (here the turbulent solver) making use of rewards gathered through its evolution and possibly following a policy in order to eventually produce an accurate representation of local turbulence statistics. RL has led to impressive progress for applications such as self-driving cars or chess but still needs to be properly transposed to the CFD domain, where its lack of supervision is especially attractive since bypassing the need high-fidelity data. This third project is focused on Double Agent or Soft Actor-Critic RL which combines two independent predictions of Q and an artificial neural network dedicated to the policy. The models developed will be assessed on various configurations including one of interest for ENOGIA, a French ORC turbine manufacturer, partner of the project.

Christophe Corre
Alexis Giauque
Emilie Sauret

Compressible Flows, Computational Fluid Dynamics, Turbulence Modelling, Machine Learning