#4 overallClassic
Practical Deep Learning for Coders
fast.ai
112 recommenders · 101 threads · recommended in 10 years, 2017–2026 · score 127.2
A free, top-down, hands-on deep learning course by Jeremy Howard and Rachel Thomas at fast.ai that teaches practitioners to build working models with PyTorch and Jupyter notebooks, with relatively little emphasis on theory. A companion Part 2 covers more advanced topics such as diffusion models.
Commenters repeatedly call it the best practical entry point into deep learning for programmers, praising its focus on building working models quickly without requiring a heavy math background.
Machine Learning & AIbeginnerfreevideo lectures#deep learning#pytorch#practical#howard#transfer learning
Recommendations by year
20062026
Definitely do the fast.ai course. Totally worth it.
But also use ISLR, Goodfellow, Bishop, etc.
Start with Andrew Ng's ML, then do the first part of Aurelien Geron book, then do Ng's DL specialization, then do fast.ai. Then learn PyTorch. A great book would be Sebastian Raschka's book. Also d2l.ai. A fast-paced, but really good course would be the Neuromatch DL tutorials.
Then move forward based on your interests.
Yann LeCun has THE best MOOC on DL on YouTube.
For the Math, I majored in Physics, so stuff came naturally. I suggest Imperial London's MOOC on Mathematics for ML specialization, Robert Ghrist's Calculus course, VMLS book for Linear Algebra. For stats, haven’t found a good one yet.
What you read, how much- these all depend on what you want to do. Where do you want to see yourself, and so on.
If you just want to brag about DL and put it on your resume so that you can get a job writing SQL queries and make PowerBI presentation as a "Data Scientist", then the bars are low.
If you want to do some DL, then that is another league altogether.
You need to be able to quickly read papers, understand ideas, use those for your own projects or papers.
Makes sense?
Fast.AI is the best course out there if you had like to get in to applied side of deep learning. However, I wish people would be more careful about their claims. It's one thing to mechanically apply same technique to same problems. It's quite another thing to compose known techniques in a new way to solve new problems. And still quite different to invent brand new techniques to solve previously unsolvable problems. The first category is "practitioner", second is "engineer" and the third is "scientist".
To transition from practitioner to engineer, one will need to know low level details including directly relevant math and popular prior art. One of the distinguishing difference between a practitioner and an engineer is that later needs to be able do systematic debugging to identify and fix the problem. I often ask folks during interviews how they would go about when their ML model to do X doesn't work as expected. This usually reveals the difference between practitioner and engineer with high accuracy.
To transition from engineer to scientist, one needs to know much broader body of prior art and beyond directly related math. A typical interview that would differentiate between engineer and researcher would be to discuss some known unsolved problem and see if the person can elaborate details of prior art, opinions of top researchers, wins achieved so far and directions for future efforts.
This is again not to say that practitioners can't do debugging OR that engineers can't make ground breaking innovations. Faraday, Edison, Wright Brothers - all either started out as practitioners or engineers and successfully attacked some of the hardest unsolved problems of their time, never obtaining formal academic credentials. On the other end far many more academics have broken new grounds then non-academics. At the end of the day its all probabilities. Claiming that PhD is no longer required to do "AI" doesn't reveal all these subtlety and marginalizes tools and foundation that PhD equips oneself on this journey.
Even though you specifically say you are willing to go full blown PhD and are interested in digging deep on algorithms etc. I strongly recommend working through "Practical Deep Learning for Coders" course at fast.ai It's free :)
It gives you an excellent feel for what is possible and they are very focused on solving interesting and practical problems right away. They explicitly try to take the "requires a math PhD" out of deep learning. Once you're through with the course you have a very solid practical overview and understanding and can solve tons of real world problems (it's almost a startup idea generator tbh.) and once you're at that stage it becomes tons easier to dive deep into specific algorithms and optimizations.
tl;dr: Take the course (they also walk you through setting up a AWS GPU server so no fancy hardware required) and you'll be able to solve real world problems with state of the art algorithms.
You've got a lot of comments like "if you're not willing to read a paper with BASIC CALCULUS ... then I'm not sure machine learning is the right place for you" to this, but I second you on this. I do have a PhD in CS and not afraid of calculus in papers. I just don't think reading PhD papers is the best path to getting practical results in DL/ML. "Not right place for you" is the sign of the "exclusiveness" problem.
I'd recommend checking the following course by Jeremy Howard and Rachel Thomas:
http://course.fast.ai/
It is designed to make DL accessible and achieves its goal very well. I'm not sure if the average Joe Hacker will be able to write top-pocker-players-beating program after this course, but it definitely gives enough insight and practice to get started in DL and be able to design and code solutions in a large number of problem classess.
i was quite put off by it. i feel like the teaching technique is pretty poor and the focus in on all the wrong things. mainly the tech gets in the way for learning. i don't want to figure out how to learn numpy when i'm trying to learn how to understand deep learning, that in itself is hard enough. i quite after a week (i did the stanford course first and this was going to be my second).
i would recommend the coursera course by andrew ng. i had an amazing time. the code stays out of your way and he walks you through the algorithms and explains the theory very well.
i just started the fast.ai by jeremy howard, and literally have been blown away but the course. it is AMAZING! by lesson 3 i'm able to build cnn models and score on top 20% in kaggle competitions. not bad for a complete novice. HIGHLY RECOMMENDED.
once im done with the fast.ai course i may look back around to google's deep learning course. i think it may be easier for more experienced users to digest its info.
https://www.coursera.org/learn/machine-learning
http://course.fast.ai/
Edit: added fast.ai link
If you want to learn the fundamentals of ML I recommend a book, such as Deep Learning: Foundations and Concepts by Chris Bishop. If you insist on staying online, one option is https://course.fast.ai/
If you don't know ML I don't think you're going to learn much through ad hoc demos.
As someone who never bothered to get any certificates (beyond a University degree) even when I'd do online courses (of which the most course-like must've been fast.ai), are these ever actually useful in any manner?
I had the same question and decided to get into the basics at least. I highly recommend the fast.ai course.
Before talking to experts you could start out working through intro materials so you will have clueful and specific questions for the experts.
The old Python usenet group had an informal custom that the more knowledgable users weren't expected to spend much time on newbie questions. If you were a newbie, you'd mostly ask easy questions (say about the break statement) that intermediate level users could answer, so they would. After a while, you'd become intermediate level yourself, and then you'd get to answer newbie questions, while more advanced users would answer your now intermediate-level questions, and so on. Finally, the big time gurus were lurking about and they would mostly answer questions that really needed attention from experts.
I felt like I'd "arrived" when one day, Guido van Rossum himself answered one of my questions. It wasn't that great or advanced a question, but still.
So yeah, if you're a beginner, maybe you can just post your questions or watch some videos, or take a class.
As for beginner AI stuff, I liked the fast.ai videos when I started watching them a few years ago, though I didn't get through that many. They have since been redone, so if I wanted to get into the subject, I'd start with them again at the beginning. I don't think I'd try to engage experts until I'd gotten through all the videos and tried other ways to deal with any stumbling blocks.
Do you want to train models from scratch, or do you want to build cool things on top of AI models?
If the former, I suggest digging into things like the excellent Fast AI course: https://course.fast.ai/
If the latter, the (relatively new) keyword you are looking for is likely "AI Engineer" - https://www.latent.space/p/ai-engineer
There's an argument that deep knowledge of how to train models isn't actually that useful when working with generative AI (LLMs etc) - knowing how to train or fine-tune a new model is less useful that developing knowledge of the other weird things you have to figure out about prompting, evals and using these models to build production-quality apps.
1) depends, I find ML much more interesting than software engineering.
2) Not me
3) With difficulty, but through courses like fast.ai
If you’re not into maths and problem solving this is probably the wrong path. The main value add you bring is being able to transform a business problem into solvable maths.
Read Introduction to statistical learning and look at the fast.ai courses. I also recommend the paper attention is all you need. If you find all of those things interesting and not too mathsy then I’d say go for the switch. Else, maybe look at options in data engineering or as a software engineer in a ML team.
There is an NLP section in Jeremy Howard's "Practical Deep Learning for Coders" course (free): https://course.fast.ai/Lessons/lesson4.html
The whole course is fantastic. I recommend it frequently to folks who want to start with DL basics and ramp up quickly to more advanced material.
Executive level education is a tricky tightrope to walk in my opinion - often you’ll hear upon completion “I now know enough to be dangerous!”. Practically, and I mean this in the most endearing way, it translates to “I know enough to be confidently incorrect”.
AI is a wide field. The hot topic is generative AI.
If you really want to learn the depths, I’d start with brushing up on statistics 101 and 201. At least you’ll know how to run controlled experiments to see if these AI work.
For practical, hands-on learning it’s hard to go past either Jeremy Howard’s FastAi courses or Andrew Ng’s deep learning courses. The latter is via Coursera from memory, so you can put a little sticker on your LinkedIn if that’s a factor.
For executive education, I’d point back to my opening quote. Either do the hands-on learning that is rapid or sign up to a 2-year exec Masters program. MIT offer a fantastic, full-course load Exec MBA (that you get a regular MBA diploma at the end, it’s not watered down in accreditation or effort). In doing so, you can specialize heavily in AI/ML and you will be coding in those classes.
This a Jupyter notebook.
Well written and comprehensive, N.B. last commit 7 months ago.
Part of the fast.ai free and excellent Deep Learning Course by Jeremy Howard.[0]
This is a pragmatic course with the emphasis on getting things done and no more theory than needed.
Can’t recommend fast.au courses enough - their particular blend of free and pragmatic is unique.
And Jeremy Howard is one of the clearest teachers around. He is engaging, and easy to grok without dumbing down.
[0] https://course.fast.ai/
I’ve seen the fast.ai course recommended over and over here.
Can‘t recommend the fast.ai course by Jeremy Howard highly enough, it walks you through building your own deep learning stack from scratch. What I really appreciate about it is that it demystifies a lot of jargon to what really are quite simple ideas at their core (e.g. „rectified linear unit“ sounding scary even though its literally only a line with a floor).
The 2022 edition isn’t so much about LLMs as about image generation with stable diffusion, but the underlying techniques are still foundational enough to be generally useful. YMMV, but for me building things from scratch, even if results don‘t reach SOTA, is the single most effective way to learn what‘s really going on.
fast.ai course is the best: https://course.fast.ai/ Comes with its own framework and is deeply practical.
I don't know if this is the best resource for learning AI/ML. I'd sooner recommend Fast.AI, Caltech's open courses, etc.[1]
Also, my understanding is that TensorFlow is becoming less and less popular, so you might want to focus on PyTorch. Curious if others agree?
[1] See: https://phaseai.com/resources/free-resources-ai-ml-2024
[2] For example, scroll down to see the chart here: https://www.assemblyai.com/blog/pytorch-vs-tensorflow-in-202...
A lot of people enjoy Khan Academy: https://www.khanacademy.org/
If you do better with an honest-to-goodness course, I'd recommend Outlier, as they have for-credit courses in introductory stats, calculus 1, college algebra, and pre-calc, depending on your previous math level: https://www.outlier.org/
Many people, however, have found more success starting with a course like fast.ai, which does not assume much mathematical maturity, before later diving into mathematics. But, I'd still recommend learning the math, even if you ultimately don't fall in love with the field of statistical learning. I've met many people who took a "higher" math course for the first time as adults and discovered that they loved the subject so much more than they did in high school.
I enjoyed the fast AI course: https://course.fast.ai/
and Karpathy's course: https://karpathy.ai/zero-to-hero.html
I haven't watched it yet, but the Practical Deep Learning for Coders course that's available on YouTube is often recommended
https://course.fast.ai/
You haven't specified in which way this is too advanced for you.
Still, I'll take a stab at recommending some other resources.
1: Practical Deep Learning by fast.ai - a (free) hands on course that's designed to get you to making something useful as quickly as possible.
2: Neural Networks Zero to Hero from Andrej Karpathy - A Series of Youtube Lectures that requires only basic python knowledge and takes you all the way up to a simplified implementation of the tech inside gpt models.
3: Neural Networks from Scratch by Sentdex. A book that's explicitly written to teach you the basics. Doesn't cover stuff like Transformers and other advanced concepts, but really takes its time with the basics.
Take a look at all three of them. They all have a different teaching style and I'd guess that at least one of them will gel with you.
start with this gem https://course.fast.ai/Lessons
I enjoyed Ng's course a lot. Fast.ai is also good, a little more demanding imho. Didn't try the dive...
Jeremy Howard's FastAI's course is another great one: https://course.fast.ai/
I'm enrolled in their latest course via University of Queensland; presently, they're teaching us by implementing one of the latest text-to-image papers in PyTorch. They cover the math as side lectures if you're interested in it and have the pre-requisite knowledge. But it's not necessary if what you're keen on is the programming of models.
If you're interested in Deep Learning specifically, the fast.ai "Practical Deep Learning for coders" course [1] is often recommended. It says "You don’t need any university math either — we’ll teach you the calculus and linear algebra you need during the course".
[1]: https://course.fast.ai/
Do you have Linear Algebra knowledge, and Stats 101 knowledge?
Then start with ISLR.
Then go and watch Andrew Ng Machine Learning course on Coursera (a new version was added in 2022 that uses Python).
Then read the sklearn book from its maintainers/core devs. It's from O'Reilly.
Then go do the Deep Learning Specialization from deeplearning.ai.
Then do fast.ai course.
If interested in Deep RL, watch David Silver lectures, then read Deep RL in Action by Zai, Brown. Then do the HF course on Deep RL.
This is how you get started. Choose your books based on your personality, needs, and contents covered.
And among MOOCs, I highly suggest the one by Canziani, LeCun from NYU. (I loved the 2020 version.)
The one taught by Fei Fei Li and Andrej Karpathy is nice.
These two MOOCs can substitute classic books based on quality.
I have never read cover to cover any of the famous books. I read a lot from them sticking to specific subjects.
Get to reading papers, finding implementations. Ng + ISLR will give you good grounds. Fast.ai + deeplearning.ai will give you capability to solve real problems. NYU + Tubingen + Stanford + UMich (Justin Johnson) courses will bring you to the edge.
You need a lot of practical experience that aren’t taught anywhere. So, get your hands dirty early. Learn to use frameworks, cloud platforms, etc.
Then start reading papers.
A crystal clear grasp on Math foundations is a must. Get it if you don't have already.
I’m a big fan of learning through practice vs learning all the theory up front, and for anyone else who feels the same, the Fast AI course and book are very good: https://fast.ai
The authors are working on a new course that’ll dive deep into the modern Stable Diffusion stuff too, which I’m looking forward to.
I’d posit we don’t understand AIML enough to know their foundations with much certainty. Take for example the discovery of emergent zero-shot properties in the latest LLMs. My recommendation to a beginner would be to grok gradient descent, matrix multiplication, and the universal approximation theorem, then get on to engineering like the rest of us. You can’t go wrong with Jeremy Howard’s FastAI course and his “Deep Learning for Coders.”
Fast.ai courses are great resources for leaning how modern ML/DL models work under the hood
https://course.fast.ai/
FastAI gets recommended a lot I think, if you can already code - focuses on hacking with frameworks instead of starting with the boring linear algebra stuff.
https://course.fast.ai/
I'm in the same boat kinda, but even more outdated on some parts. I had some AI specialization back in college, but that was before deep learning was even a thing, so we did stuff like self-organizing maps and evolutionary algorithms, but it wasn't really all that useful for much back then. Been following deep learning from the sidelines, but for work my AI work has been restricted to GOFAI until recently.
Some of the stuff I'm currently reading/watching or have recently
Practical Deep Learning, though it sounds like you may know this stuff already (https://course.fast.ai/)
Practical Deep learning part 2, more about diffusion models. Full course coming early next year (https://www.fast.ai/posts/part2-2022-preview.html)
Hugging Face course (https://huggingface.co/course/chapter1/1)
Diffusion models from hugging face https://huggingface.co/blog/annotated-diffusion https://huggingface.co/docs/diffusers/index
Andrej Karpathy's Neural Networks: Zero to Hero. He goes from the basics up to how GPT etc, so you can start wherever suits you (https://www.youtube.com/playlist?list=PLAqhIrjkxbuWI23v9cThs...)
3blue1brown's videos. I've found all his videos on neural networks and math worth watching, even for stuff that I already know, he sometimes has some new perspectives and nice animations.
brilliant.org. Nice math refresher and the courses there are almost like fun little games.
If you want to learn A.I. try https://www.fast.ai/ Its probably one of the best places to start. Highly recommended.
Deep Learning from the Foundations IS part 2. They are not talking about how part 1 starts. They are talking about how part 2 starts. This mirrors how their book is written, with the content typically in Deep Learning for Foundations being the second half of the book. I've taken the courses every year, it's always been like that.
You should consider taking a look at part 1 of the course if you want to understand the starting curriculum: https://course.fast.ai/
It’s a bit dated now, but the DeepLearning.ai GANs specialization covers topics through StyleGAN. If you have no experience with ML at all I would probably start with the first two courses of their Deep Learning specialization and then jump into the GANs specialization.
I would also highly recommend FastAI’s Deep Learning for Coders (and their new course that came out this year). You’ll start immediately with some cool applications (basic image recognition and NLP) and then drill down from there to learn how they work in detail.
It’s set up such that you can learn as much as you want (basics with no depth: first chapter; basic understanding of how a neural network is trained with SGD: first four chapters; understanding of decision trees, LSTMs, and CNNs: first half; detailed understanding of how to build everything from scratch: whole book).
FastAI. Specifically "Deep Learning for Coders" which was recently updated. https://course.fast.ai/
Do what the instructor recommends: watch each lesson once in its entirety and then re-watch it while playing along. But don't just type their commands verbatim. Try and do something slightly different.
+1 for fast.ai it is and was probably best course for noobs for years (i rember 2017/18 courses as big help in DL even that i got exp with machine learning before that [about 1 year of mostly classical ML] :P)
The first Fast.ai course back in around 2016 changed my life.
I was studying a masters in statistics and computer science that had 1 neural networks lecture and nobody knew anything about deep learning. Fast.ai and Jeremy’s teaching style helped me start playing with deep learning models really quickly and I changed my thesis topic to computer vision.
I ended up consulting on the topic and doing various startups leading to the startup I’m working on now which just finished YC (AiSupervision W22).
I doubt be here without fast.ai. I highly recommend and appreciate all the work that Jeremy and the rest of fast.ai do!
I second this - while both are great courses, I found I could only dedicate very short amounts of time recently to any kind of study, and going from the ground-up more thoroughly seemed like I was making no progress. The fast.ai top down approach worked a bit better for me for those reasons, otherwise it would have been interesting starting with the deep dive.
I highly recommend https://course.fast.ai/. It's much more top down: in the first lesson or two, you train a NN image classifier, rather than starting with first principles and linear algebra. I found this structure to be more motivating and effective.
Just build interesting stuff, you will learn all necessary tech in meantime. I suggest fast.ai course
If you have not at least skimmed through fast.ai, you should definitely do that, as the course itself addresses some of this and the people on their forums are among the most helpful I have ever seen!
Second, this could be more than enough! Especially if you are doing transfer learning.
Third, you can "inflate" the amount of images you have now with "Image Data Augmentation"
This is quite interesting, and I hope the course material will be made available for others (including myself) to check out.
This really resonates with me, as a deep learning researcher who started out by taking the fast.ai course. The fast.ai course highlighted the same ideas of getting your hands dirty and exploring in a top-down approach. It's great to see how this sort of approach translates to other fields.
For the purpose of this question, you might search for pre-trained models that predict what you want to predict, and use them.
https://modelzoo.co/
https://github.com/tensorflow/models
https://cv.gluon.ai/model_zoo/index.html
If there are no pre-trained models, you could take a look at fast.ai courses:
https://course18.fast.ai/ml.html (for Machine Learning (ML))
https://www.fast.ai (for others "DL" (Deep Learning))
This courses is targeted towards programmers, and they dive right into the code to use ML (train models and use them) with Jupyter notebooks. They also have a book whose notebooks are here: https://github.com/fastai/fastbook
If you have some maths background (high-school, or first year university), check out Stanford's CS229 with Andrew Ng:
https://www.youtube.com/playlist?list=PLoROMvodv4rMiGQp3WXSh...
I think it's better than the Coursera version because he uses the board (whiteboard in 2018, blackboard in 2008-ish version)
They’re much narrower in scope than a full curriculum (or even a course, for that matter), but Better Explained [0] has some very good overviews of math topics. They are a useful supplement I’ve come across repeatedly while searching topics I found challenging over the years.
On the topic of open source learning, I take every chance I can to heartily recommend fast.ai’s course [1]. It’s a good intro to Deep Learning that leaves you informed enough to build things, and equips you to ask follow-on questions and dive deeper when/where you need to.
[0]: https://betterexplained.com/
[1]: https://course.fast.ai/
If they released it, people would figure out a way to run it “equitably” within months, if not weeks.
The amount of cheap GPU access floating out there is nuts. You can spin up a GPU instance to do best-in-class ML stuff using Fast.Ai on services like Paperspace or Colab, right now, for free.
In addition to posting your contact info please also check out this course designed for people from other fields who want to learn machine learning.
https://www.fast.ai/
If you get serious about data science, you're going to end up reading/watching a lot of different resources. I'm a data scientist and I'd suggest starting with fast.ai and then following your interests elsewhere. The best book or course is the one you're going to finish.
Yes. I would recommend the fast.ai course linked above. It covers all the essentials of deep learning and some classical machine learning. You'll have enough breadth of knowledge to know which areas you'd like to explore in more depth and all the skills you need to build practical projects.
Jeremy should update his JavaScript course as well. He might be one if the few people to make it look less messy than it is everywhere else. The fast.ai course is wonderful. Definitely recharged my own interest in deep learning.
I took the fast.ai courses and highly recommend them for anyone who really wants to learn ML.
Are there any plans for courses on reinforcement learning?
Took this course two times, first when they used TensorFlow and afterwards based on PyTorch. Like how it is practical from early on and updated on new research. Recommend trying to build networks from scratch in combination with the course, so you don’t become to dependent on the fast.ai framework.
I have also started learning NLP this year. Here is what have followed. If you are completely new to deeplearning I would recommend course from https://www.fast.ai. Since it will teach you the concepts in a more practical way and also the next courses will more theory than practical.
For theory on NLP with deeplearning you can follow Stanford course by Christopher Manning.
http://web.stanford.edu/class/cs224n/index.html#coursework
it will give you a good understanding of how deeplearning is used in a certain area of NLP. But remember deeplearning is one of the techniques for solving NLP problems if you are more interested in Understanding NLP then I would recommend the following book.
https://web.stanford.edu/~jurafsky/slp3/ed3book.pdf
Although, Stanford course should give you a great high-level understanding of how GPT models work but if you really need to go under the hood then I would suggest after finishing this course you should learn Unsupervised Learning in DeepLearning.
Machine Learning using fast.ai courses i would say they are really good and fast way for programmers to get onboard without going through all those boring statistics classes.
For whatever it’s worth fast.ai is my favourite MOOC and the most impressive set of videos I’ve seen shared freely on the internet. Thankyou for everything you do!
For folks that want to start making sense of what's being said I recommend going through d2l.ai, fast.ai, and deeplearningbook.org.
Most of contemporary AI is really just a combination of probability theory and some upper division math classes in an executable form. None of it is magic and the more people that know the vocabulary the less likely people are to buy into the hype.
If you want a high level overview of all this then Melanie Mitchell has a good book as well: https://melaniemitchell.me/aibook/. She does a really good job of putting everything into the right context and dispelling the marketing hype about the coming singularity and human obsolescence. In one of the chapters she covers deep reinforcement learning and it's one of the best high level explanations I've come across yet.
I am currently going through fast.ai's Deep Learning course and will totally recommend it because of its top-down approach.
Has anyone done non-DL courses on their website? For e.g., any thoughts on Rachel's Computational Linear Algebra?
Machine Learning:
* https://www.youtube.com/watch?v=UzxYlbK2c7E: Andrew Ng's machine Learning course, the recommended entry point by most people
* https://mlcourse.ai/ : More kaggle focused, but also more modern and has interesting projects
Do both courses simultaneously, take good notes, write useful flashcards, and above all do all the exercises and projects
Deep Learning
* https://www.fast.ai/ - Very hands-on, begin with " Practical Deep Learning for Coders" and then "Advanced Deep Learning for coders"
* https://www.coursera.org/specializations/deep-learning : More bottom-up approach, helps to understand the theory better
Do those two courses in parallel (you can try 2 weeks of coursera followed by one of fastai in the beginning, and then just alternate between them), take notes, write good flashcards and above all do the exercises and projects.
After that you will be done with the beginning, your next step will depend on what area interested you the most, and getting way too many resources right now can be extremely confusing, so I would recommend doing a follow-up post after you worked through the above resources. Also as non-ML stuff I recommend Scott Young's Ultralearning and Azeria's self improvement posts (https://azeria-labs.com/the-importance-of-deep-work-the-30-h...)
I have used TensorFlow almost exclusively for my deep learning work in the last five years and using TensorFlow is much more common, largely I think, because Andrew Ng’s excellent five part course used it. On the other hand Jerry Howards’s wonderful free classes use mostly PyTorch so maybe my argument does not hold.
For research and for use in published papers, PyTorch is much more widely used.
fast.ai has some great intro courses on deep learning.
It does exist! There are lots of answers to what you’re asking for. But here’s the best answer:
https://course.fast.ai/
This course uses python and pytorch and will have you building something that can tell 30+ breeds of cats and dogs apart in the first lesson. While the old course from Andrew ng you mentioned starts with theory and by the end gets to doing something useful, the fastai course starts with best practices and moves into the underlying theory later.
Also Andrew Ng has a newer deeplaearning.ai course on coursera that is all python and doesn’t do Octave.
IMO, for reaching PhD eligibility level at ML, you just need 2-3 core math courses. (as long as each is rigorous)
Linear Algebra, Probability and Statistics. (and some optimization)
The fast.ai course is great, but having the math background really helps bring the whole field together. Different ideas and models with the math, seem to stand completely independent of each other.
As a software engineer who wants to learn ML, the fast.ai course is great. But, if you find yourself in a situations where you are scoping out a data science problem, the math background helps immensely in coming up with a solution.
I hope this post helps someone looking to start with ML.
When I started my journey, I thought I should do what this article mentions.
My problem is that "learn mathematics" can take years. To me, it would be very frustrating.
So I started down that path... and got completely overwhelmed.
When everything took off and I went from 2 mph to 60 mph was when I did the fast.ai course. The fast.ai course is strongly recommended by me. You'll start seeing results and "actionable" work very quickly. If you so desire, then you can keep on with the more fundamental stuff like mathematics. Also have to give a shout-out to their forums. The people are super nice.
I know it doesn’t sound at first like what you’re looking for, but I strongly recommend the Fast.ai course.
Despite advertising itself as for coders and not math heavy, I found it to be much better at explaining the math than, for example goodfellas’ book.
It’s also much better at talking about the inspirations of certain methods - such as Dropout - than other sources I’ve found.
Overall, if you want a deep understanding of neural network, fast.ai is - somewhat ironically given its branding - the best resource I’ve encountered by a long shot.
I've seen many people recommend fast.ai, and while I have not completed the course, the few lessons I did work through were very well made and I really learned a lot. Also, totally free. Check them out.
Really awesome course! For those looking to run locally these seemed to be the most straight forward setup steps https://course.fast.ai/start_aws.html#step-6-access-fastai-m... assuming you have conda installed.
IMO do top down ML stuff and learn the math as-needed. This is recommended by Rachel Thomas and Jeremey Howard of fast.ai as well. There are a lot of great resources for the specific kinds of math you'd need for ML. I personally am self-proclaimed terrible at math ( also had very been schooling ) and was able to ML with not much extra work after their courses. Good luck!
In the fast.ai course, there are two or three amazing videos where Jeremy Howard explains the basic concepts of deep learning (convolutions in this case) using Excel spreadsheets, highly recommended.
I agree with mkagenius, I've played around a lot with deep learning and like physics it suffers a lot from unnecessary jargon that hides relatively simple concepts, for example this random quote:
Now everybody could just use plain English - if you have a negative number, set it to zero, in all other cases, keep the number - I don't know why someone has to use 'rectified linear unit' to describe this simple operation.
fast.ai courses are EXCELLENT. I've read/watched many interviews with both creators and have been even more impressed by their drive to bring deep learning to everyone, not just the AI specialists.
I've gone through each iteration of the fast.ai courses and must say that it is worth it every time. The latest 2019 courses are again excellent. Even though I've covered the concepts before, I always learn a ton from every lesson.
Although it is recommended to dedicate ~10 hours per lesson, it is very possible to go through each video, then circle back to focus on one of the concepts/problems presented.
I recommend giving the first lesson a try. It is very rewarding feeling like you can actually do or build something at the end of each lesson.
This. I can tell anybody thinking about taking this course, do it! The way Jeremy Howard explains things is so down to earth and clear, you'll be surprised at how easy all this ML/DL nonsense is :)
But seriously, there's a lot to learn here, and if you are looking to make your entry into the hype field of the decade I would strongly suggest starting here first
(Alternatively, for the more math inclined, perhaps take Andrew Ng Coursera course first: https://www.coursera.org/learn/machine-learning)
If you're interested in a very practical, explain-it-clearly, no marketing type of resource for deep learning, I can highly recommend https://www.fast.ai/
I did the first version of the course, and I felt that at the end I legitimately knew enough to read current papers and see through any hype presented to me anywhere else.
I'm not sure how this will be received, but I'm learning a lot through following Jeremy Howard. He's a huge PyTorch fan and he's spent the last 3 years trying to figure out what people like you need. He launched a course called FastAI and a DL library by the same name. His aim is to help anybody do it that wants to, with or without code.
MOOC: http://course.fast.ai
I just found a resource a few months ago that I'd love to recommend, but haven't started yet. It's mentorship you pay for, but not up front. You sign a contract to pay a certain percentage after you're hired. I plan on going through this program if my current job leads don't pan out.
Mentorship: https://sharpestminds.com
I'm interested in comments about either program in general. Speaking as someone who also has an EE degree, went through a web development bootcamp, and was disappointed by both at the help in getting hired that was offered after the curriculum was finished, I am also interested in your findings and results.
I would suggest going through the Fast.AI course [1]. It's an excellent course to learn more about DL in general, and some of torch API. The downside is that the course material was produced when PyTorch was still at 0.3, so some of the API has changed since then.
[1] https://course.fast.ai/
I read some tutorials on back propagation for this, but if you want to get into ML in general, I highly recommend fast.ai and Andrew Ng's free Stanford ML course.
Essence of calculus[0] by 3blue1brown for the basics and the second course in the Coursera Mathematics for Machine Learning would let you get started. You would rarely need calculus more advanced than that covered in the above, and if need be you will be in a position to look it up quickly. If you can sustain your interest in ML over a long period of time and are in no hurry, I would recommend going through all the math mentioned. If you are a top-down learner, the fast.ai course on ML and deep learning for coders will get you started head-first. All the best!
[0] - https://www.youtube.com/playlist?list=PLZHQObOWTQDMsr9K-rj53...
Markov chains are cool and text generation is fun for experiments. I would definitely recommend looking at deep learning language models, for a more fruitful investment of time to improve further. (courses on http://www.fast.ai/ are a good starting place)
I've done Andrew Ng's Coursera specialization (deeplearning.ai) and course.fast.ai, and I would 100% recommend starting with fast.ai. (Seeing results more quickly is motivating. It's also free.) When you know that you enjoy the topic, feel free to learn more rigorous ML theory from other sources.
It's hard to recommend something if we don't really know your level of knowledge so far.
I wholeheartedly recommend the fast.ai [0] course. It provides a lot of instantly applicable code, coupled with very good explanations which you can try out on novel problems later. It's focused on "learning by doing", and not "learning by reading" which fits my style really well.
That said, it doesn't dissuade the watcher from reading later, it's just not recommended to start out with.
[0]: course.fast.ai
Sorry for the tone here (unusually snarky comment).
Arguably, my point was that this argument, biasing your analysis by hidden motives that cannot be disproven, is invalid.
But clearly, I was wrong because fast.ai's courses are indeed free so the comparison doesn't stand.
I'll also add that fast.ai has great ML courses that go deep into details, and focus on practical, applicable, techniques. My comment was specifically about the argument that Google is doing this just to increase Cloud revenues.
I found fast.ai the best resource for learning ML/DL out there. So, while I'm not associated with them in any way, I wouldn't hesitate to recommend it as a course for anyone interested in learning more about ML/DL. Not sure that would count as marketing if I were to do so...
If you have at least some coding experience and you are interested in the practical aspects of ML/DL (i.e., you want to learn the how-to, not the why or the whence), my recommendation is to start with the fast.ai courses by Jeremy Howard (co-author of this "Matrix Calculus" cheat sheet) and Rachel Thomas[a]:
* fast.ai ML course: http://forums.fast.ai/t/another-treat-early-access-to-intro-...
* fast.ai DL course: part 1: http://course.fast.ai/ part 2: http://course.fast.ai/part2.html
The fast.ai courses spend very little time on theory, and you can follow the videos at your own pace.
Books:
* The best books on ML (excluding DL), in my view, are "An Introduction to Statistical Learning" by James, Witten, Hastie and Tibshirani, and "The Elements of Statistical Learning" by Hastie, Tibshirani and Friedman. The Elements arguably belongs on every ML practitioner's bookshelf -- it's a fantastic reference manual.[b]
* The only book on DL that I'm aware of is "Deep Learning," by Goodfellow, Bengio and Courville. It's a good book, but I suggest holding off on reading it until you've had a chance to experiment with a range of deep learning models. Otherwise, you will get very little useful out of it.[c]
Good luck!
[a] Scroll down on this page for their bios: http://course.fast.ai/about.html
[b] Introduction to Statistical Learning: http://www-bcf.usc.edu/~gareth/ISL/ The Elements of Statistical Learning: https://web.stanford.edu/~hastie/ElemStatLearn/
[c] http://www.deeplearningbook.org/
Have you looked at fast.ai's ML course? It might also be worth a try. http://forums.fast.ai/t/another-treat-early-access-to-intro-...
I'd also recommend the Andrew Ng Coursera course, which I talked about here: https://harveynick.com/2018/04/25/some-notes-on-the-andrew-n...
I am a firm believer in candid feedback; I talked to multiple people taking fast.ai and all of them took it as disturbing the flow of lectures. I really like fast.ai, it's an excellent hands-on course, so please don't get offended.
Why not both ? Both are excellent courses. I took deeplearning.ai last year. This year I am planning to complete fast.ai.
Hey!
I'd suggest starting with this excellent course:
https://www.coursera.org/learn/machine-learning
And then dive into competitions on kaggle.com.
Then following up with the deeplearning.ai specialization here: https://www.coursera.org/specializations/deep-learning
and the http://www.fast.ai/ courses.
You'll be up to speed in no time :)
I'd highly recommend course.fast.ai - it's focussed on deep learning, but is designed for beginners to get to a production stage.
fast.ai courses are the best I've taken. You start with implementing things immediately then get into the details later. It makes it less like a grind to learn everything beforehand.
For those not familiar, Jeremy Howard (one of the two authors of this paper), is one of the founders of course.fast.ai, an excellent open course teaching machine learning geared toward software engineers and others with and without a strong academic background.
I really enjoy the course and encourage those who haven't yet looked into it to try it out, so long as one has an interest in machine learning.
I'd suggest fast.ai mooc before keras docs. I took Hintons course and tried to learn through tf/keras docs, but wasn't able to really get going until I found fast.ai. Some of the best "classes" I've ever watched and there's a ton of people helping in the forums.
I'm starting the fast.ai courses as a recent stat grad looking to expand my knowledge. Heard many good testimonials. Besides that, I think Andrew Ng's Coursera offerings - intro to ML and the newer NN specialization - are great first steps. Personally I have a hard time learning from videos, so I refer to my copy of Pattern Recognition and Machine Learning by Bishop.
If you want a linear algebra text, I enjoy Strang's Introduction to Linear Algebra
In October, I quit my job to live on savings and work on AI/ML. I am not interested in developing novel AI approaches; my objective is to learn the application of machine learning to solve narrow problems.
In light of that, I believe the technology to have progressed to where one can learn how to use existing libraries to solve specific problems. Lots of businesses have specific problems, and will pay me to solve them.
The Fast.AI course (part 1 v2) is the best way to get started, IMO. The fastai library wraps up a lot of boilerplate, and gives you a simple recipe w/ conceptual understandings to achieve state of the art results (top 20% on just about any kaggle competition) in just a couple months of intensive study.
For now, I'd recommend you start the two courses on: http://fast.ai/
They will guide you through machine learning and deep learning basics without being overwhelming with math. Good luck!
I would highly recommend starting with course.fast.ai. Really helped me.
I'll give another recommendation for http://www.fast.ai/
I went through the first phase of the course as an intro to AI/DL and thought it was really great from a high-level perspective. If you have a decent understanding of Python you'll have a working model running on AWS within the first few hours of the course which is very rewarding.
It does a better job than I expected explaining the underlying intuition of the math, but doesn't dive deep into the actual formulas. There are obviously tradeoffs to this approach and if you want to continue in the field you'll need to do something to fill in this background, but as far getting your hands dirty and understanding the basics I really liked the fast.ai approach.
It was interesting reading this, because those 5 points basically describe Google to a T. It's an aborted Ph.D thesis that took VC, hired lots of Ph.Ds with a culture "like a family", and then embarked on a hypergrowth trajectory.
The irony is that when Google was young, there was a different set of "destructive trends". They still included taking VC, but it was also common knowledge that:
1. You had to sell a physical product for money.
2. You had to build out all your infrastructure before you could bring a product to market.
3. Sales & marketing were more important than engineering.
4. (When they were hiring on their hypergrowth trajectory, 2001-2004:) The web is dead, and programming is a terrible career because it's all about to be outsourced to India.
Perhaps the meta-lesson is to do what other people are not doing. The point of markets is that they reward unexpected successes, where an entrepreneur serves a population with little competition, and that necessarily means thinking for yourself and ignoring the common wisdom. Fast.ai does an admirable job at that - their deep learning course is excellent - but taking their advice at face value is just as dumb as cargo-culting Google.
From my personal experience I would recommend Andrew NG's courses. (I am on the second one right now). I have done fast.ai's first course which is low on math and more practical (you directly jump to coding, and then try to make some sense of things.) CS231n is definitely a great option, however I chose coursera because I needed deadlines/reminders of Coursera to keep me motivated and finish something. If you are looking for more options, there are lot more courses listed here in Machine Learning category : https://github.com/prakhar1989/awesome-courses however I don't know if someone has reviewed them.
First, why do you want to be a Software Engineer? Higher pay? Do you like coding? Interested in puzzles?
Second, I would just start small. Automate the boring stuff will get you in the right mindset for being able to factor out repetitive tasks by thinking about them as algorithms. In my opinion, algorithms are the original essence of what makes computer science a science: it's what Church and Turing (two google-worthy figures in our fields history) first formalized on their pursuit to define what it means for something to be computable.
Once you can break a problem down into an algorithm, you can write a computer program to solve that problem. That is the power of writing code: if you can understand it algorithmically, you can delegate the task to a machine. This takes time and lots of practice!!! I would focus on trying to do this over and over until it's natural. Once you feel comfortable in Python, try learning another language and see how much easier/harder writing programs in that new language becomes.
Third, you shouldn't feel like Machine Learing is out of the question. http://course.fast.ai is a MOOC specifically designed to prove that ML is indeed something anyone can play around with. (I'm just beginning the course, and I'd recommend it after a year of coding under your belt and have a good comfort with using a unix-like terminal).
Good luck, and keep practicing!
Ng's deeplearning.ai is good for getting a solid background in how it all works. If you want to just jump into building things, I'd highly recommend http://course.fast.ai.
Excellent!
I'd suggest anyone interested in jumping into building computer vision models should check out the excellent free MOOC on fast.ai
I studied post-grad ML and I can say you really don't need to go this deep to use ML - check out http://fast.ai for a code-first practical crash course :)
My recommendation is the fast.ai course by Jeremy Howard. His explanations are amazing and the practical usefulness is immediate.
http://course.fast.ai/
You could try http://fast.ai They show all the steps necessary to train an image classifier (for cats vs. dogs, but it should be possible to adapt to other classes easily) and the explanations are excellent.
But if you are trying to do OCR on complete texts, maybe even handwritten, you're getting close to research-level problems very quickly.
1. Start with the fast.ai courses. It's an applied deep learning course using state of the art techniques. 2. For classical machine learning (regression, etc...), Andrew Ng's course on Coursera is widely considered "the basics" 3. As you progress, check out CS231 and CS224 from Stanford for state of the art image processing and natural language processing techniques. The lecture videos are on YouTube and the course assignments are available online. The third course I recommend is Geoffrey Hinton's neural networks course on Coursera (he is one of the most important researchers in the field). 4. If you're an application engineer, focus on using existing tooling to build cool projects. Keras and scikit-learn are great out of the box tools. 5. If you are more research oriented, you can start reading papers. In Silicon Valley, there's a meet up group that reads papers every Monday and tries to implement the algorithms in the paper. It takes a while to get to this level, but try not to get overwhelmed. Experts spend 7 years studying this stuff full time to get a PhD. 6. You really don't need much math to get started with ML. A high school understanding of calculus and some basic understanding of numerical optimization are the two main concepts you need to know. If you want to get into the research, there'll come a time when you will need more advanced math, but in my experience you can pick that up as you go along if you are curious.
Maybe you could start an AI study group online? The Silicon Valley study group was great, but I was just visiting.
Try http://www.fast.ai/ Probably this is the best course form non-mathematical side.
I really really agree with you. I also think that the fast.ai courses are amazing.
It is a question of motivation. If you're reasonably proficient at programming and want to hit the ground running on a specific application (especially in a domain that has well-established methods) fast.ai is probably what you're looking for.
If you're new to programming and mathematics, or want to work on the state of the art of these methods, you need to first truly understand more fundamental ideas. For those who fall into this camp, I am collecting resources (work in progress) and trying to organise them into a learning pathway: https://github.com/hnarayanan/deep-learning
You or anyone else who's interested is free to offer suggestions on learning material.
+1
fast.ai is probably the best MOOC I've ever followed. As its name says, it is "for coders" and 100% applied. It is perfect for getting started, quickly.
Please read this blog post, in particular regarding the comment on "Hacker News contributors regularly give such awful advice on machine learning".
http://www.fast.ai/2017/03/17/not-commoditized-no-phd/
and:
http://www.fast.ai/2017/03/23/focus-on-coding/
No, start with fast.ai's deep learning course. They have a top down approach so you start with connecting layers to solve deep learning problems and then dig down into the theory behind the thing.
If you want a bottom up approach, go look at Ng's coursera course.
http://course.fast.ai
I'd suggest lecture 14 of the fast.ai mooc on some advice on feature engineering for timeseries data to make it possible to model with something like regression (or a non-recurrent neural net - can work surprisingly well)
I don't know what level you are operating at regarding machine learning or AI, but the fast.ai MOOC is one of the best educational resources in that area I have ever seen in my life. If you have not tried it out yet, I very highly recommend it.
Can you use Excel?
The fast.ai course is very good.
http://course.fast.ai
Its hands down the best course to get your hands dirty with latest , state of art stuff, and then learn how it works. It has completely different approach to most courses. It is top down.
Do this first, you can immediately apply it to cool stuff like image classification , nlp etc.
The assignments have additional resources where you can get into more detailed math (but the course doesn't dumb things down, but gives more intuitive explanations) and dive even deeper.
Python Machine Learning is a great book to start the author made a great curriculum if you would like to follow it. https://sebastianraschka.com/faq/docs/ml-curriculum.html
This is mostly ML if you would like to dive into deep learning I think fast.ai is the best course for anyone with programming experience and you can also use the deeplearning.net Tutorial as a side reference. If you have a practical experience and would love to understand the theory behinds it then Deep Learning Book is the Bible.
Jeremy your fast.ai course was amazing. Any idea when/if there's going to be a part 2?
Just watched the first video, it's a very nice intro to the mathematics behind neural networks, personally I like Jeremy & Rachel Howards video course [1] a lot better because it is much more geared towards practical applications but if you're into the theory end of things or if you simply want to understand better what is going on behind the scenes this video is really quite good and well worth your time.
[1] http://course.fast.ai/
The fundamental techniques of machine learning will always be there. For that reason, Ng's ML course will always be the prime entry point for people new to ML. As someone who has taken both the Ng course and has roughly watched the videos of the fast ai course and done some work, I would recommend first doing the Ng course. Things will be much more calm if you then approach the fast ai DL course. The fast ai course is amazing and hats off to Jeremy and Rachel for the great work. But without any idea of machine learning, I would say you will need to put at least 20 hours of work each week to work through it, and that is if you are also a really proficient programmer.
I agree with the previous post that you should focus on Keras rather than Tensorflow. Understanding Tensorflow is a great skill to have because you get a more appreciative and deeper understanding of the models when you dig deeper. But for most application, especially for a fun side project, Keras should be perfect.
I recommend http://course.fast.ai/ to learn more about the applications of neural networks and how to apply neural networks quickly through python.
Highly recommend this course - http://course.fast.ai/ that utilizes Keras as the main programming tool
Completely agree. I love the pragmatic nature of the course and the general attitude (anyone can do DL). It's probably the one MOOC I'd recommend to every developer or CS student. It's really mind blowing how good you can perform compared to what was state of the art not too long ago. Not quite finished with #1 either but can't wait for part 2. The pitch/promise for part 2 is that it'll basically take you to the bleeding edge of current (2017) research...that sounds rather exciting.
I'm now in week 4 of the first round of this MOOC (here: http://course.fast.ai/ ) and it's incredibly intuitive, highly recommended. It's less about the mathematics, more about what has been proven to work, and based on these scores I've gotten quite high in the Kaggle competitions the classes go through (Cats & Dogs, State Farm, Titanic)
The Jupyter noteboks are sometimes still quite messy and deserve some cleanup, but it's all open anyway, should submit a PR...
Sailing: You don't need a boat. Most cities near the water will have some sort of sailing club where for $300 a year you can join and use their fleet of boats. My end goal is to do some sort of week long solo trip.
Rock Climbing: Lots of fun and a great work out.
Meditation: Just getting started with this, but have already seen improvements in concentration and a decrease in anxiety.
Introductory Deep Learning: http://course.fast.ai/ This is probably my favorite course I've ever taken online.
I had taken the Ng course, which I recommend to get a basic understanding of ideas, but, the fast.ai course enables you to compete in Kaggle competitions in fairly short time.
Just watch the course.fast.ai intro/overview video, and decide from there.
A lot of people are wondering why MOOCs as a whole industry failed. I learned a lot from MOOCs, and can attribute much of my professional success to MOOCs. I can think of these reasons-
1. There are so much great content available for free, it almost doesn't make sense to pay for stuff.
2. Lots of universities, including MIT, NYU release their content for free along with handouts, assignments, problem sets, etc. And many great professors from other unis also release a lot of courses for free.
3. Much of the focus was on certificates, and honestly employers don't value them a lot, and higher education institutes don't value them at all.
4. Quality diminished in MOOCs. The early MOOCs from 2010s are of greater quality than the stuff that are released recently.
5. To make it in life, you are far better off with a university degree rather than without. And, people can't solely rely on online education.
6. MOOCs are expensive. Udacity nanodegrees were $1,000. My entire Master’s degree in CS was ~$1500 (in India).
7. They started focusing more on online degrees which almost no one value. And they are obscenely expensive. And platforms focus more and more on them. Think of Sedgewick's Algorithms course on Coursera. I don't see any new course of similar quality there. Focus is entirely on the online "degrees".
8. Good people left to do something cooler to them. The people who created Coursera, Udacity, etc. now do something else. Most provably some AI startup. People not similar to them are running the show now.
9. Coursera, edX, Udacity all have horrible, slow UIs. After a 2-8 minutes lecture (played in 1.5x) makes you wait a slow full 2 seconds to load the next page. And this is breaking, IMO. I would take a 2.5 hours university class lecture over this any day.
10. Unis use their own platform and abandoned joint ventures (?) because, maybe, lack of control. MIT has MITx, and some truly great new courses. I don't think they upload in edX anymore.
11. As per their behaviour, they focused on company-wide training, upskilling, etc. But I think most corporates don't look to upskill employees in a holistic way. They just want some tool specific training for their employees, that are too the point, practical in a negative way, and solely focuses on the tool at hand. Not every company is Texas Instruments, Facebool AIR, DeepMind, etc.
12. Despite all these, MOOCs are still successful, now often riding on particular people. They have their own websites with their own pricings, and own UI design. Think Jeremy Howard with fast.ai.
I think these are causes for the decline of MOOCA hype wave.
- AIMA by Russel and Norvig is a classic but I would say is more of overview of the field and for most topic areas isn't quite deep enough imo.
- For deep learning specifically, a more applied text that is beautifully written and chock full of examples is Francois Chollet's Deep Learning with Python (there a new second edition out with up to date examples using modern versions of Tensorflow). The first 3 chapters I would give as required reading for anyone interested in understanding some deep learning fundamentals.
- Deep Learning - goodfellow and bengio - seems like it would be hard to get through without a reading group not exactly a APUE or K&R type reading experience but I haven't spent enough time with it.
If you haven't taken a Linear Algebra or Differential Equations class its useful stuff to know for ML/DL theory but not fully necessary to do applied work with modern high level libraries, but definitely having a strong understanding of basic matrix math is useful.
If you have interests in natural language processing theres a couple good books:
- Natural Language Processing with Python - Bird Klein, Loper, is a great intro to NLP concepts and working with NLTK which may be a bit dated to some but I would definitely recommend, and its online for free. Great examples.(https://www.nltk.org/book/)
- Speech and Language Processing - Dan Jurafsky and James H. Martin - is good, though I have only spent much time with the pre-print
And then theres a lot of papers that are good reads. Let me know if you have any questions or want a list of good papers.
If you just want to get off the ground and start playing with stuff and building things I'd recommend fast.ai's free online course - its pretty high level and a lot is abstracted away but its a great start and can enable you to build lots of cool things pretty rapidly. Andrew Ng's online course also is quite requitable and will probably give you a bit more background and fundamentals.
If I were to choose one book from the bunch it would be Chollet it gives you pretty much all the building blocks you need to be able to read some papers and try to implement things yourself and I find building things a much more satisfying way to learn than sitting down and writing proofs or just taking notes but thats just my preference.
This is nicely put. It reminds me a bit of David Perkins's [0] "whole ballgame" approach (i.e. the best way to learn how to play baseball is to play a few games or approximations thereof, e.g. tee-ball; you'll struggle mightily at first, but you'll ultimately benefit by having a sense of how the game as a whole works, and it provides valuable context and motivation for subsequently working on specific skills like throwing the ball, swinging the bat, etc. And the ultimate result is greater mastery than if you had started from the skills and worked your way up to playing the game piecewise.) Fast.ai's pedagogy is based in part on this principle, and (as someone inclined to go down rabbit holes) it's been a huge help to me as I go through the course and book. [1]
I'd add too, recently released personal knowledge base apps (Obsidian, Roam, et al) are very well suited to fractal learning. I partition my notes into content (summaries) and concepts (with some Zettelkasten-style ephemeral notes mixed in). When I come across new concepts, I create a stub; then I fill it in when necessary with the necessary degree of detail, adding new details, revising, and forcing myself to make my account of it coherent as I encounter different facets of the concept in different books, articles, etc.
[0]: http://www.pz.harvard.edu/resources/making-learning-whole-ho...
[1]: http://course.fast.ai/
You are confusing platforms with individual models. A platform, such as Pytorch or Tensorflow, does not care what shape is your input. You design and train a model for the whatever inputs you want. On the other hand, if someone trained a specific model architecture (e.g. YOLOv3 or Resnet-50) on some dataset of square images, then yes, this particular pre-trained model will expect the same input size and shape as what it was trained on. Does it make sense? If you take a beginner level course on deep learning (e.g. Stanford CS231n or FastAI course) you will immediately realize there's nothing (in any sense of the word) that prevents you from using any input shape to train your model.
If you don't want to learn the tools you use, you will need to find someone who will train a model on your images. If you're willing to pay for it you will find plenty of help. However, if you want to neither learn nor pay, then what exactly are you complaining about?
Unless I knew for certain that I couldn’t do solo work, or knew for certain that I couldn’t muster the discipline, I would say “work on your own” in the beginning.
If I am to take your harmonious financial arrangement at face value, which you reaffirm in another comment, understand that your situation is probably very enviable to many. Make the most of it.
You didn’t specify but I assume you expect this situation to hold for at least a year, maybe two or three years? That’s great. And you already dabbled in ML, and you “really like it”? Well, its the remote option, but this is what I would do :
1. I would brainstorm a project about applying ML to law, or my field of law[0]. This is not completely unheard of, as I’m sure you’re aware.
2. If I come up with some holy grail ideas, I would put them aside for later. I would then try to find something to start with that is very simple, maybe something that would have helped me day-to-day in my previous lawyer life. Maybe something much smaller that would be a necessary but not sufficient item for one of the holy grail ideas, or like I said some small but obvious value added day-to-day lawyer stuff.
3. After I decided on the project, I would apply the Amazon way of working backwards, a. Create the press release: Describe in simple terms what you’re going to build b. 20 most frequent questions: Answer in clear, simple terms (& iterate until it is very clear what you’re going to build) c. UX Document: How are my users going to interact with this d. Keep the above simple, don’t overdo it.
4. So you came up with your first project, now you have to skill up. Since you dabbled in ML maybe you are already familiar with the ML learning angles people recommend on this site (fast.ai, Coursera etc). I would research and make a decision which ones to start with first (maybe you’ve already completed some) and focus on completing them together with all the relevant exercises etc. In parallel, I would be on the lookout about what I need to learn for the project and continuously apply what I learn to the project.
5. Out of necessity, you are probably going to need to dabble in integrating frontend web and mobile stuff. Research and decide on learning path based on what concrete things you actually need in your project.
6. Once you feel you know enough to build the shittiest and dumbest version of your idea build the shittiest and dumbest version of your idea before continuing a new online course in your studying plan.
7. Put your code on the web when you have the shittiest and dumbest version of your idea working and keeping updating it. I would upload two separate things as projects a. the software/product that the end user will use that has everything in it. b. The ML component as a separate project. Don’t let the frontend stuff detract from the ML stuff. Imagine someone with better web and mobile skills wanted to use your ML-law work to make something more polished. They could use this.
8. Now you have a portfolio, don’t you? You could go on building the next item on the road to a holy grail, or you could continue iterating on this as you learn.
9. Meanwhile, you could start applying with your portfolio for remote work at AI tech companies targeting law or NLP areas (more general overlapping domains) , or law firms that are scared of tech who don’t want to be the nokia or blackberry of law in the future (this is probably the unlikelier of the two, since this tech would disrupt billable hours). In the former, you would be the “domain” guy for the tech, in the latter you could be the “tech” guy for the domain. Your skills and background will be unique.
10. Alternatively, or in parallel, start selling what you built. Cause looked at it another way, this is identical to have received seed funding for 1-3 years to start a company.
Sure when you start working with others you may soon feel you’ve missed out on some fundamentals, but this is what I would do in terms of opportunity cost. By the end, you will have a unique background and you will be working in a niche area suited to your background, your weaknesses will matter less. After this, if I still felt iffy and it still didn't matter if I got a job or not, then I would consider a online degree through a US institution, but if its a very good program.
Anyway you asked me what I would do, obviously might not be the best advise (disclaimer).
[0] assuming doing anything related to law or your field of law doesn’t instantly fill you with dread or actively demotivate you in whatever you’re doing. If I hated law, the general outline would still be the same. Coming up with a project idea that kept me motivated and that I thought would be of use to others might take a little more time to find. I wouldn’t be able to use my law background for applying for work for maximum effect, so there would be no binding narrative between my previous lawyer life and what I’m doing now, that might make it more difficult to sell myself to potential employers. But with a portfolio that would matter less.
This is another good result of applying transfer learning to NLP.
Transfer learning works great for vision problems (just reuse one of the big SoTA trained on ImageNet networks - I like resnet50). This was enabled by the amazingly shared structure of vision problems. There was nothing similar for NLP, besides pre-trained first layers like word2vec. If you want to learn more, check out the fast.ai DL course, it features transfer learning a lot.
But this model and ULMFiT (nlp.fast.ai) show that deeper nets can be pretrained for NLP, and achieve good results when transfered to other datasets and problems.
This enables not just the obvious use case of "I don't have N GPUs to train a deep net from scratch but I can now finetune a pre-trained model" but more subtle and interesting cases like fine-tuning on a very small dataset (compared to ImageNet or 100000 samples NLP data sets) and cheap training on demand. Training a new model for every user was way too expensive if training from scratch, but if fine-tuning a pre-trained net takes just a few minutes, why not ?
I started with Andrew Ng's ML courses, which take a bottom-up approach beginning with the math. After finishing the deeplearning.ai track, I started rounding out my skillset with some data science and R programming classes, so I could be more comfortable working with unfamiliar data.
The fast.ai courses are a more top-down approach to ML, and there are plenty of good reasons for taking this approach. You'll start getting practice with libraries like Tensorflow right away. However, if you have a fairly strong math background and linear algebra doesn't give you nightmares, I highly recommend the Andrew Ng courses. A deep (ha!) understanding what's going on "under the hood" in ML will help your debugging, inform your strategy, and make your code better in the long run.
With all due respect, this is a quite random reading list, and appears more a result of the 'if "deep learning" in post.title: post.upvote()' trend on Hacker News ...
I will pick on two under the "Classics" section simply as I know the author and have used their work, so am not in any way saying the work isn't useful (it certainly can be in the right spot!), but it isn't "classic" or what I'd recommend for early readers at all. "Uncertainty in Deep Learning" and "Dropout as a Bayesian Approximation" were both published within the last year and a half and is a PhD thesis + paper on interpretations of neural networks in a Bayesian fashion. "Classic" for a paper + thesis less than two years old is quite a stretch even for the fast moving field of deep learning. The same holds true for many of the other papers in "Classics" such as batch norm which is (a) recent and (b) certainly not the only of such techniques (see layer norm, recurrent batch norm, ...) and (c) has complications in implementation[1].
As the simplest example, why is the original dropout paper[7] not under classics? It's an elegant paper, fundamentally important for current neural networks, and is more classic than "Dropout as a Bayesian Approximation" or "Dropout Rademacher Complexity of Deep Neural Networks" which are both listed.
I'm also highly dubious of the noted neuroscience connection - most deep learning researchers use very little from neuroscience.
Again, this list may be helpful to the creator of the repo and tailored toward their specific research direction but it is not useful for readers from Hacker News or those aiming to get their start in deep learning. Why so many upvotes? Zero comments? Zero discussion?
If you want a book, check out the Deep Learning book[2]. If you want a course for RNNs, check out CS224d[3]. If you want a course for CNNs, check out CS231n[4]. If you want to get down and dirty in a practical software engineering way, check out Fast AI[5]. If you want summaries of select recent deep learning papers in GitHub format, check out Denny Britz's notes[8]. There are many other starting points but those are my default suggestions.
If you really want to start learning, this isn't the right list for you and I'd really like to suggest a more sane and potentially tailored path. Seriously. If you reply with what you want, I'll do my best to suggest a starting point.
Background: I'm a deep learning researcher who publishes papers and articles[6].
[1]: http://www.alexirpan.com/2017/04/26/perils-batch-norm.html
[2]: http://www.deeplearningbook.org/
[3]: http://cs224d.stanford.edu/
[4]: http://cs231n.github.io/
[5]: http://course.fast.ai/
[6]: http://smerity.com/articles/2016/google_nmt_arch.html
[7]: https://www.cs.toronto.edu/~hinton/absps/JMLRdropout.pdf
[8]: https://github.com/dennybritz/deeplearning-papernotes