RL Course by David Silver - Lecture 1: Introduction to Reinforcement Learning

The complete set of 10 lectures is brilliant. David's an excellent teacher. Highly recommended!

Just finished lecture 10 and I've come back to write a review for anyone starting. Excellent course. Well paced, enough examples to provide a good intuition, and taught by someone who's leading the field in applying RL to games. Thank you David and Karolina for sharing these online.

what a wonderful time to be alive!! thank god we have the opportunity to study a full module from one of the best unis in the world. taught by one of the leaders of its field

Just finished lecture 1 and can already tell this is going to be one of the absolute best courses 👌

I love that David is one of the foremost minds in Reinforcement Learning, but he can explain it in ways that even a novice can understand.

0:01 Outline Admin 1:10 About Reinforcement Learning 6:13 The Reinforcement Learning problem 22:00 Inside an RL angent 57:00 Problems within Reinforcement Learning

1:10 Admin 6:13 About Reinforcement Learning 6:22 Sits in the intersection of many fields of science: solving decision making problem in these fields. 9:10 Branches of machine learning. 9:37 Characteristics of RL: no correct answer, delayed feedback, sequence matters, agent influences environment. 12:30 Example of RL 21:57 The Reinforcement Learning Problem 22:57 Reward 27:53 Sequential Decision Making. Action 29:36 Agent & Environment. Observation 33:52 History & State: stream of actions, observations & rewards. 37:13 Environment state 40:35 Agent State 42:00 Information State (Markov State). Contains all useful information from history. 51:13 Fully observable environment 52:26 Partially observable environment 57:04 Inside an RL Agent 58:42 Policy 59:51 Value Function: prediction of the expected future reward. 1:06:29 Model: transition model, reward model. 1:08:02 Maze example to explain these 3 key components. 1:10:53 Taxonomy of RL agents based on these 3 key components: policy-based, value-based, actor critic (which combines both policy & values function), model-free, model-based 1:15:52 Problems within Reinforcement Learning. 1:16:14 Learning vs. Planning. partial known environment vs. fully known environment. 1:20:38 Exploration vs. Exploitation. 1:24:25 Prediction vs. Control. 1:26:42 Course Overview

Excellent moment around 24:10 when David makes it crystal clear that there needs to be a metric to train by (better/worse) and that it's possible - and necessary - to try to come up with a scalar metric that roughly approximates success or failure in a field. When you train something to optimize for a metric, important to be clear up-front what that metric is.

Wow, this is incredible. I'm currently going through Udacity and this lecture series blows their material from GT out of the water. Excellent examples, great explanation of theory, just wow. This actually helped me understand RL. THANK YOU!!!!!

I am a simple man. I see a great course, I press like

Mr. Silver covers in 90 minutes what most books do not in 99 pages. Cheers and thanks!

how lucky we are to have access to this kind of knowledge only with a button ! Thank you all in DeepMind public this course

1:10 Admin 6:13 About Reinforcement Learning 21:57 The Reinforcement Learning Problem 57:04 Inside an RL Agent 1:15:52 Problems within Reinforcement Learning

Thank you so much for repeating the questions each time.

That was brilliant. Really helping me to get my head around the subject. Thanks David

This is one of the clearest and most illuminating introductions I've watched on RL and its practical applications. Really looking forward to the following instalments.

@1:07:00. Instead of defining P_{ss'}^a and R_s^a, it's better to define p(s',r|s,a), which gives the joint probability of the new state and reward. The latter is the approach followed by the 2nd edition of Sutton&Barto's book.

Just completed Coursera's Reinforcement Learning Specialization and this is a nice addition to reinforce the concept I am learning.

The lecturer is great!

David is awesome at explaining a complex topic!. Great lecture. The examples really helped in understanding the concepts..