David Silver's lecture series on reinforcement learning, given at UCL and published on YouTube, covering core RL concepts such as Markov decision processes, dynamic programming, temporal-difference learning and policy methods. Commenters note it covers material from the Sutton and Barto textbook.

Commenters call it excellent and great, praise Silver as an amazing teacher whose visual explanations make the Bellman equation click, and often pair it with Sutton and Barto.

What commenters said (10)

1. Strongly recommendsswimmingbrain2026in “Hamilton-Jacobi-Bellman Equation: Reinforcement Learning and Diffusion Models”

   Sutton & Barto is the standard recommendation and it's good, but if you want to build intuition for the Bellman equation specifically, I'd start with David Silver's UCL lectures on YouTube before touching any textbook. He builds up from scratch and the visual explanations stick. Once that clicks, the Sutton & Barto chapters on dynamic programming and TD learning will make a lot more sense. I'm going through RL coursework right now and that was roughly my path. The jump from discrete Bellman to the continuous HJB in this post is genuinely hard though, don't feel bad if it takes a while to land.

2. Strongly recommendsayhanfuat2023in “Deepmind – Reinforcement Learning Lecture Series (2021)”

   This is a more up to date version of David Silver’s course and judging from the few lectures I watched, it is very good but if you haven’t, I recommend having a look at Silver’s lectures, too. He is an amazing teacher and it is a joy to watch him explain something. https://www.deepmind.com/learning-resources/introduction-to-...

3. Strongly recommendsmmq2023in “Deepmind – Reinforcement Learning Lecture Series (2021)”

   David Silver's reinforcement learning lecture series is excellent as well:

   https://www.youtube.com/watch?v=2pWv7GOvuf0&list=PLzuuYNsE1E...

4. Strongly recommendsshmageggy2020in “Dopamine and Temporal Difference Learning”

   Others mentioned the neural side, but I think that's ancillary to the main idea of the paper which is about RL. David Silver's lectures on YouTube are excellent: http://www0.cs.ucl.ac.uk/staff/d.silver/web/Teaching.html

5. Recommendsmalshe2024in “Reinforcement Learning: An Introduction (2018)”

   Thanks for sharing this. David Silver's YouTube lectures on RL are great too

6. Recommendsdwrodri2022in “Introduction to Probability for Data Science”

   This looks like a fantastic resource. Thanks for sharing!

   I really enjoy the Bayesian side of ML, but it's definitely not the most accessible. Erik Bernhardsson cites latent dirichlet allocation as a big inspiration behind the music recommendation system he originally designed for Spotify, which is apparently still in use today[1]. I still struggle with grokking latent factor models, but it can be so rewarding to build your own and watch it work (even with only moderate success!).

   Kevin Murphy has been working on a new edition of MLaPP that is now two volumes, with the last volume on advanced topics slated for release next year. However, both the old edition and the drafts for the new edition are available on his website here[2].

   The University of Tübingen has a course on probabilistic ml which probably has one of the most thorough walkthroughs of a latent factor model I've found on the Internet. You can find the full playlist of lectures for free here on YouTube[3].

   In terms of other resources for deep study on fastinating topics which require some command over stats and probability:

   - David Silverman's lectures on reinforcement learning are fantastic [4]

   - The Machine Learning Summer School lectures are often quite good, with exceptionally talented researchers / practictioners being invited to provide multi-hour lectures on their domain of expertise with the intended audience being a bunch of graduate students with intermediate backgrounds in general ML topics. [5]

   1: https://www.slideshare.net/erikbern/music-recommendations-ml... 2: https://probml.github.io/pml-book/ 3: https://www.youtube.com/playlist?list=PL05umP7R6ij1tHaOFY96m... 4: https://www.youtube.com/playlist?list=PLqYmG7hTraZDM-OYHWgPe... 5: http://mlss.cc

7. Recommendscolmvp2019in “Reinforcement Learning: An Introduction (2018) [pdf]”

   I also recommend interested people to watch David Silver's RL lectures at UCL on YouTube. He covers material from the book.

   https://www.youtube.com/watch?v=2pWv7GOvuf0&list=PL7-jPKtc4r...

8. Recommendsalkhatib2019in “Dopamine framework: Fast prototyping of reinforcement learning algorithms (2018)”

   I came across this a few days ago: For Reinforcement Learning specifically, the standard text is Reinforcement Learning: An Introduction[1]. Dave's UCL Course on RL[2] is great too (playlist of all lectures)[3].

   Source: Julian Schrittwieser works on Deepmind at Google http://www.furidamu.org/

   [1]http://incompleteideas.net/book/the-book-2nd.html

   [2]http://www0.cs.ucl.ac.uk/staff/d.silver/web/Teaching.html

   [3]https://www.youtube.com/watch?v=2pWv7GOvuf0&list=PL7-jPKtc4r...

9. Recommendsclickok2018in “Show HN: Improved Seam Carving Using Forward Energy”

   I would not dive into GANs right off the bat. Ng's course is a good start, maybe also buy or borrow a copy of Russel and Norvig's "Artificial Intelligence: A Modern Approach"[0]. If you decide to look into deep learning, I've heard good thing about Bengio's book (see also Hugo Larochelle's lectures on YouTube) and if you want to learn more about reinforcement learning, "Reinforcement Learning: An Introduction" by Sutton and Barto is frankly amazing (and there's a great course on YouTube by David Silver).

   From reading your code, you're probably familiar with most of the stuff you need to learn ML, so I thought I would take a stab at explaining how you get from "no machine learning experience at all" to "surely GANs will solve all our problems". Broadly speaking, a lot of machine learning is just deciding what your goal is (carve an image, play a game, classify objects, group similar data points, etc.) definining an objective function (e.g., number of sharp edge removed, player score, a classification loss function[1], distance over points in a cluster) and then maximizing or minimizing that objective function.

   In classification, you want a function `f(.)` such that given a data point `x`, `f(x) ≈ y`, where `y` is the correct label for that data point. Say you have a bunch of these data points in a matrix `X`, and a vector of correct labels `y`–– how do you come up with a function that correctly classifies the data in `X`, and how can you do it so that data you might collect in the future would also be correctly classified?

   You could do this with linear least squares (come up with weights `w` to minimize `|f(x) - y|`, with f(x) = x^T w) but unless your data is cleanly separable, that probably won't work so well. There are a lot of possible techniques people have tried, but lately neural networks have become popular because they are powerful approximators, relatively easy to deploy, and pretty versatile. But the training process[2] still boils down to tweaking weights until `|f(x) - y|` is as low as you can make it.

   General adversarial networks are kinda a tweak on this idea-- you start with a bunch of images, and you train a network to generate images that are "similar" to them; call the initial images the "real" images and the generated ones "fakes". You want your network to generate fakes that are similar to the real images. How do you define similar? Well, you have a second network whose objective is to distinguish between real and fake images. You train both at the same time, using feedback from the second network to improve the first network, and using the images generated by the first network as training examples for the second.

   Initially, the first network will generate splotchy garbage that the discriminator can easily tell apart from the real images. Over time, the first network improves, creating increasingly realistic fakes, while the second network gets better at determining whether an image is real or generated. After much training, you have two networks: one that is good at generating images that are similar to the seed images, and another network that is good at identifying if an image is real or fake.

   ––––––––

   0. It even has a website and example code http://aima.cs.berkeley.edu/

   1. https://en.wikipedia.org/wiki/Loss_functions_for_classificat...

   2. See here for a nice explanation of the backprop algorithm, http://neuralnetworksanddeeplearning.com/chap2.html or if you want the OG (original Geoff) paper on the topic, it's in the proceedings for Parallel Distributed Processing 1986 -- "Learning representations by back-propagating errors", Rumelhart, Hinton, and Williams.

10. Mentionspmulv2018in “Curated Resources to get your ML journey started”

    Here are some others that are also commonly recommended on HN:

    Andrew Ng's Machine Learning MOOC: https://www.coursera.org/learn/machine-learning

    Andrew Ng's Deep Learning Courses: https://www.deeplearning.ai/

    David Silver's Reinforcement Learning Youtube Series: http://www0.cs.ucl.ac.uk/staff/d.silver/web/Teaching.html

    Fast.ai Courses on ML and Deep Learning: https://www.fast.ai/

Related courses

• Machine Learning (Andrew Ng)Coursera · 240 recommenders
• Deep Learning Specialization (Andrew Ng)Coursera · 31 recommenders
• Deep Learning Course (Hugo Larochelle)YouTube · 2 recommenders
• Probabilistic Machine Learning (University of Tübingen)YouTube · 1 recommender
• Machine Learning Summer School lecturesMLSS · 1 recommender
• Practical Deep Learning for Codersfast.ai · 112 recommenders
• CS231n: Convolutional Neural Networks for Visual RecognitionStanford University · 39 recommenders
• Neural Networks for Machine Learning (Geoffrey Hinton)Coursera · 25 recommenders