Deep Reinforcement Learning for Fluid Dynamics and Control

Links to papers in video: @3:58 Machine learning for fluid mechanics Brunton, Noack, Koumoutsakos, Ann. Rev. Fluid Mech 52:477--508, 2020 www.annualreviews.org/doi/pdf/10.1146/annurev-flui… @5:04 Efficient collective swimming by harnessing vortices through deep reinforcement learning Verma, Novati, Koumoutsakos, Proc. Nat. Acad. Sci. 115(23):5849--5854, 2018 www.pnas.org/content/115/23/5849 @6:57 Automating turbulence modelling by multi-agent reinforcement learning Novati, Lascombes de Laroussilhe, Koumoutsakos, Nat. Mach. Int. 3:87--96, 2021 www.nature.com/articles/s42256-020-00272-0 @8:47 A review of Deep Reinforcement Learning for fluid mechanics, Garnier, Viquerat, Rabault, Larcher, Kuhnle, Hachem, 2019 arxiv.org/abs/1908.04127 @9:57 Artificial neural networks trained through deep reinforcement learning discover control strategies for active flow control Rabault, Kuchta, Jensen, Reglade, Cerardi, J. Fluid Mech. 865, 2019 doi.org/10.1017/jfm.2019.62 @10:56 Reinforcement learning for bluff body active flow control in experiments and simulations Fan, Yang, Wang, Triantafyllou, Karniadakis, Proc. Nat. Acad. Sci. 117(42), 2020 doi.org/10.1073/pnas.2004939117 @11:50 Fluid directed rigid body control using deep reinforcement learning Ma, Tian, Pan, Ren, Manocha, SIGGRAPH 2018 gamma.cs.unc.edu/DRL_FluidRigid/ @13:26 Autonomous helicopter flight via Reinforcement Learning Ng, Kim, Jordan, Sastry, NeurIPS 2004 papers.nips.cc/paper/2003/file/b427426b8acd2c2e538… @13:26 An Application of Reinforcement Learning to Aerobatic Helicopter Flight Abbeel, Coates, Quigly, Ng, NeurIPS 2007 proceedings.neurips.cc/paper/2006/file/98c39996bf1… @13:26 Autonomous helicopter aerobatics through apprenticeship learning Abeel, Coates, Ng, Int J Rob Res 2010 journals.sagepub.com/doi/abs/10.1177/0278364910371… @14:02 Learning to fly like a bird Tedrake, Jackowski, Cory, Roberts, Hoburg, Int. Symp. Rob. Res. 2009 groups.csail.mit.edu/robotics-center/public_papers… @14:58 Control of a Quadrotor with Reinforcement Learning Hwangbo, Sa, Siegwart, Hutter, IEEE Rob Aut 2(4) 2017 arxiv.org/abs/1707.05110 @15:22 Learning to soar in turbulent environments Reddy, Celani, Sejnowski, Vergassola Proc. Nat. Acad. Sci. 113(33, 2016 www.pnas.org/content/113/33/E4877 @16:31 Learning to Fly: Computational Controller Design for Hybrid UAVs with Reinforcement Learning Xu, Du, Foshey, Li, Zhu, Schulz, Matusik, SIGGRAPH 2019 people.csail.mit.edu/jiex/papers/LearningToFly/ind…

you're so incredibly apt at explaining such mind-blowing phenomenon while tying them into interesting and novel areas of study! Thank YOU!

It is so pleasant to see someone doing their work so passionately! You are an outstanding professor, Dr Brunton!

We need more professors and lecturers like you Dr. Brunton. You made academic publications more interesting to study !

This channel is pure gold.

I really enjoy the enthusiasm you show when delivering the topics! Perfect and outstanding...

the best teacher I havve ever seen on youtube . greetings and regards from INDIA !!!!

Thank you very much for sharing you great papers and knowledge!

Thank you professor, it's great to watch your videos!

Hey Steve, thanks for the great video and the paper highlight ;)

I'm thinking about Jim Lovell and Apollo 13. After the explosion, he had to relearn flying the craft in a new configuration. He said it would go left when he wanted to go right. But they did it. This kind of work could save lives when we suddenly find ourselves in a new place we didn't count on.

Dear prof, I really admire it.. Your videos make to do research on AI (DRL) in CFD Thank you professor.

It makes me want to join Washington University

The legend of control engineers 🙏 thanx

Thank you, fantastic video!

Great video! One thing I'm curious about - my understanding is that reinforcement learning is difficult in practice b/c it's hard to build a simulated environment which matches reality. But here, it seems that issue has been managed, since these drones are flown in the real circumstance. So, are our turbulence simulations really that good or are there other clever tricks here? In general, do you see the accuracy of the simulation as the primary limiting factor in reinforcement learning?

Thank you for making this video

Using the thermals to climb reminds me of the finite horizon, energy optimal trajectory video you just posted.

This makes me really want to get into UW's CS program. Fingers crossed!

a great video! Thank you