A university lecture series on deep learning by Nando de Freitas (Oxford), delivered as recorded video lectures on YouTube covering neural networks and their training, including backpropagation.

Commenters call it a great lecture series for learning deep learning, noting it is a bit math heavy but has a really nice derivation of backprop.

What commenters said (4)

1. Strongly recommendsdeepnotderp2018in “Ask HN: How to Seriously Start with Machine Learning and AI”

   For deep learning: Surprised no one's mentioned Nando de Freitas's YouTube lectures, great series! Hugo Larochelle's YouTube course is great as well!

2. Recommendsdeepnotderp2017in “A Curated List of AI and Machine Learning Resources from Around the Web”

   Don't forget Nando's Deep Learning lecture series! Admittedly it's a little bit math heavy, but don't let that scare you off!

   Also Hugo Larochelle's course is great for beginners!

3. Recommendsjg86102016in “Neural Networks Are Impressively Good at Compression”

   It's good to see people write up their experiments, it's useful for the rest of us to test how we understand neural nets.

   I think there are a few mistake in your maths though. You can learn a 1-1 discrete mapping through a single node where you are using a one-hot vector. You just assign a weight to each of the input nodes, and then use a delta function on the other side. If I understood correctly, this is what you are doing.

   Also, if you use a tanh in your input layer, but keep a linear output layer (as you start off with), you are still doing a linear approximation because you have a rank H (where H is the hidden layer) matrix that is trying to linearly approximate your input data. This is done optimally using PCA.

   I'd second the advice to look into the coursera courses, or the nando de freitas oxford course on youtube (that actually has a really nice derivation of backprop).

4. Recommendsbcaine2016in “Introduction to Statistical Learning, with Applications in R”

   Two courses I've found are very good for Neural Networks & Deep Learning are Karpathy's CS231n from Stanford[0] and Nando de Freitas's Deep Learning class from Oxford[1][2].

   Been through parts of both and they seem really good. That said, I'd bet that both of these classes would be a lot more meaningful after either Hastie's class (listed here) or Andrew Ng's course.

   [0] http://cs231n.github.io/

   [1] https://www.youtube.com/playlist?list=PLE6Wd9FR--EfW8dtjAuPo...

   [2] http://www.cs.ox.ac.uk/teaching/courses/2014-2015/ml/

Related courses

• Machine Learning (Andrew Ng)Coursera · 240 recommenders
• Deep Learning Course (Hugo Larochelle)YouTube · 2 recommenders
• CS231n: Convolutional Neural Networks for Visual RecognitionStanford University · 39 recommenders
• Practical Deep Learning for Codersfast.ai · 112 recommenders
• Deep Learning Specialization (Andrew Ng)Coursera · 31 recommenders
• Neural Networks for Machine Learning (Geoffrey Hinton)Coursera · 25 recommenders
• CS229 Machine LearningStanford University · 20 recommenders
• Learning From DataCaltech · 20 recommenders