#56 overall
CS224N: Natural Language Processing with Deep Learning
Stanford University
9 recommenders · 8 threads · recommended in 6 years, 2016–2023 · score 9.1
Stanford's graduate course on natural language processing using deep learning, taught by Christopher Manning. It covers word vectors, neural architectures such as attention and transformers, and their use across NLP tasks, with lecture videos and course materials available online.
Commenters point to it as the go-to deep-learning-focused NLP course, with one calling it a good high-level grounding and another praising its historical perspective and lectures on word vectors.
Machine Learning & AIadvancedfreevideo lectures#nlp#deep learning#word embeddings#transformers#manning
Recommendations by year
20062026
Highly recommend the Stanford course notes on AI. I've found these useful:
* CS224n natural language processing
* CS330 Meta Learning
* CS224U Natural language understanding
* CS234 Reinforcement Learning
* CS221 Artificial intelligence
* CS229 theoretical machine learning
The course notes can be found at CS(x).Stanford.edu such as
cs229.stanford.edu
Of course the main problem isn't that there's not enough good material. The problem is that there's too much! So a course that teaches how to pick the most important courses for oneself is sorely needed
It covers a lot of the fundamentals in some detail (attention and transformers, decoding, transfer learning) that are underneath current cutting edge NLP; this is still a very good foundation likely to be good for several more years.
What might be missing is in-context learning, prompt engineering, novel forms of attention, RLHF, and LoRA (though it covers adaptors), but this is still changing rapidly and the details may be irrelevant in another year. If you have a look at a recent course like Stanford CS224N 2023 there's a lot of overlap.
It's difficult because there are hundreds of really good courses and I don't keep track. It's almost a full time job to just keep track of this stuff given how much there is. Search for one of these "awesome Deep Learning lectures" Github repos and you will find them.
The Stanford courses (224, 231, and the Transformer one) are a safe bet. In general, I would recommend university courses where the lectures are online. There are a lot of them. Fast.ai is supposed to be good as well from what I've heard, but I don't have experience with that myself.
I have also started learning NLP this year. Here is what have followed. If you are completely new to deeplearning I would recommend course from https://www.fast.ai. Since it will teach you the concepts in a more practical way and also the next courses will more theory than practical.
For theory on NLP with deeplearning you can follow Stanford course by Christopher Manning.
http://web.stanford.edu/class/cs224n/index.html#coursework
it will give you a good understanding of how deeplearning is used in a certain area of NLP. But remember deeplearning is one of the techniques for solving NLP problems if you are more interested in Understanding NLP then I would recommend the following book.
https://web.stanford.edu/~jurafsky/slp3/ed3book.pdf
Although, Stanford course should give you a great high-level understanding of how GPT models work but if you really need to go under the hood then I would suggest after finishing this course you should learn Unsupervised Learning in DeepLearning.
I'd recommend following a course to get an idea of what's involved. The Stanford course[1,2] goes deep into the neural network side of things; although lots of state of the art systems use these, I'd suggest that starting by thinking about language and what makes it difficult to process (with the help of some (computational) linguistics) is also helpful. Knowledge of e.g. morphology may help you make decisions down the line with e.g. stemming for word embeddings. The Cambridge course[3] gives an introduction in this area (disclosure: I TA this).
Aside from that, yes. Lots, but not all, of NLP these days involves ML in some form. Most things I've seen are done in Python, what with all the NLP-specific and general ML libraries available.
In terms of textbooks, Speech & Language Processing by Jurafsky & Martin is great. Might not be the best way of diving in, but it's a good resource to go deeper into things. The draft third edition, while only partially complete, is online for free [4].
[1] http://web.stanford.edu/class/cs224n/
[2] https://www.youtube.com/playlist?list=PL3FW7Lu3i5Jsnh1rnUwq_...
[3] https://www.cl.cam.ac.uk/teaching/1819/NLP/
[4] https://web.stanford.edu/~jurafsky/slp3/
Check out the Chris Manning book and Stanford course videos.
For anyone interested in more explanation, Stanford CS224N course has 2 great lectures on them.
Recommend Dan Jurafsky and Chris Manning @ Stanford online course:
https://www.youtube.com/watch?v=nfoudtpBV68
Yes, using language models is how it's done.
Theoretically, it's not that hard, in practice, it's really hard.
There is an online language modelling course from Stanford that you should check out if you want to take a stab.
For a historical perspective check out stanford's nlp course: https://youtu.be/IxQtK2SjWWM?t=1267
Deep learning only started beating tradition methods in 2016!