#1 overallClassic
Machine Learning (Andrew Ng)
Coursera
240 recommenders · 226 threads · recommended in 18 years, 2009–2026 · score 364.2
Andrew Ng's introductory machine learning course on Coursera, covering supervised and unsupervised learning, linear and logistic regression, neural networks, and practical advice on applying ML. It originated as the Stanford 'ML Class' in 2011 and uses Octave for programming assignments.
Commenters call it a great introduction that avoids heavy math while still conveying real understanding of the topic, and many say it was their starting point.
Machine Learning & AIbeginnerfreevideo lectures#machine learning#neural networks#octave#andrew ng#introductory
Recommendations by year
20062026
If you want a great introduction, check out Andrew Ng's Coursera course:
https://www.coursera.org/learn/machine-learning
This is the same course that was originally called "ML Class" in 2011 when it was offered via Stanford (it, plus the other course called "AI Class" - helped to launch Coursera and Udacity, respectively). It uses Octave as it's development language to teach the concepts.
Octave has as it's "primitives" the concept of the Vector and Matrix; that is, it is a language designed around the concepts of linear algebra. So, you can easily set a variable to the values that make up a vector, set another to say the identity matrix, multiply them together, and it will return a matrix.
But the course is initially taught showing how to do all of this "by hand" (much like this tutorial) - using for-next loops and such; essentially rolling your own matrix calcs. Then, once you know that, you are introduced into how to use the built-in calcs provided by octave, etc.
It is really a great course, and I encourage you to try it. It opened my eyes and mind to a lot of concepts I had been trying to grasp for a while that just didn't click until I took that course in 2011.
Of course, once it did, I was like "that's it? why couldn't I see that?" - such is the nature of learning I guess!
His method of building up from the basics all the way to a neural network using Octave will be a good foundation for further learning or playing with other tools like TensorFlow; I ended up taking a couple of other later MOOCs offered by Udacity (focused mainly on self-driving vehicle concepts), and ML Class (and what I took of AI Class) really helped to prepare me.
It also showed me things I was deficient in (something I need to correct some day). But that's a different story.
Suffice to say, that course will "unveil the magic"; it was really an amazing course that I am glad I took. Oh - and to show you what you can do with what you learn in that course, a fellow student during that time ended up making this (and the course was not even finished when he did it):
https://blog.davidsingleton.org/nnrccar/
Throughout the course, the ALVINN autonomous vehicle was referenced quite often, and it inspired him to recreate it successfully in miniature. It also serves to show just how far computer technology has come - what used to take a van-load of equipment can now be played around with using a device that fits in your pocket!
Most fun: Pat Pattison, Songwriting, Coursera. Very good lectures, very good material, very well presented. Teaches a lot about writing song lyrics in just 6 weeks, breaks it nicely down to steps and recipes. I used to think that the best feature of MOOCs is the automatic grading and feedback from programming homework, but in this course, for the homework songwriting you gave and got feedback from 3-5 random people in the course, and it was not only useful but this feeling of togetherness with strangers was even better than getting instantaneous feedback from a bot for programming homework. Shows that teaching art scales to MOOCs as well.
Nicest: Andrew Ng, Machine Learning, Coursera. Interesting topic, well-planned material, very well avoids going into the mathy details, but still conveys a feeling of understanding of the topic, so accessible to a wide audience. (Martin Odersky, Functional Programming Principles in Scala, Coursera, was almost equally nice, but had some rough edges in the first run.)
Most interesting: Probabilistic Graphical Models, Daphne Koller, Coursera. Very interesting topic. I took the first run of the course and it had lots of rough edges. Needs a lot of work to apply the lectures to the homework. I haven't seen such a demanding course since I took quantum mechanics at university.
Best organized: Jennifer Widom, Databases, Stanford. This is not the flashiest of a topic, but oh boy was it well organized. Runs like a clockwork. Everything in the lectures is relevant, everything from the lectures is applied and tested in the homework, there is lots of homework (but still not enough to make you remember SQL,XPath,XQuery,XSLT for the rest of your life if you don't keep using them), weekly homework has a nice progression from simpler things to medium difficult things, and the web environment is well designed, and gives wonderful feedback and guides you to get your queries correct.
[disclosure: GT CS alum]
This is something I wanted to highlight from your post. I don't think this is surprising, nor do I think it is reasonable to state that a course (or a degree program) is poor quality because it didn't meet the standards of Ng's ML course. That is an exceedingly high bar.
Its something I noticed, because I am a recent grad, so while I was in my mid-level courses, and had recently taken Tech's intro CS course, I was able to watch (Harvard's) CS50 and other courses. But on the other hand, I've seen some very bad online courses. The successful and large online courses are successful and large specifically because they are head and shoulders better than the rest. And there are a lot of decent online courses, so to measure against what are some of the absolute best online courses is to measure against courses that have more resources, more planning, and more feedback than most.
(as an aside, they also have more incentive to be good, but that's a bit tangential to the point that they also have more opportunity to be good).
You type in "Machine Learning" on coursera and you get over 1000 results (not all of which are relevant, but assuming even 10% are), its little wonder that one or two are going to be better than the even the best courses that you'll take during a bachelors or masters, because Coursera offers more Machine Learning courses than most people will take in their Bachelors or Masters.
Combine that with these courses coming prefiltered (you've heard of the Stanford Course, but what about "Applied Text Mining in Python" from UMichigan, which for all I know might be great, but it doesn't come with the hundreds of recommendations that the Ng course does, so I don't know that it will be great) and you have a really great recipe for a bias against the in person courses.
It's been a while since I took a Coursera course but I LOVED it at the beginning. Between Machine Learning, the (numerical) optimisation courses and NAND-To-Tetris (even for the platform alone) it had so many great courses to pick from.
Ironically the very first coursera course was (IIRC) Andrew Ng's machine learning course, which was fantastic, and the deep learning specialization, which was also phenomenal. I can unironically say that Andrew Ng was the best instructor I ever had in grad school (and I didn't go to Stanford...).
What are the best online courses you’ve taken?
On Coursera, I did Andrew Ng’s machine learning course and Dan Boneh’s cryptography course and both were excellent. Time well spent IMHO.
The next thing I want to take is a WinDbg course. Udemy has one that looks pretty good. I should probably also find a modern assembly language course…
Thanks! I'll follow the link and see what happens. And thanks for recommending Andrew Ng's course too, hopefully it gives enough background to know how the users (AI scientists) want us to prepare the data.
IMO absolutely yes. I would start with the linked introduction and then ask myself if I enjoyed it.
for a deeper dive, check out the sth like Georgia Tech’s CS 8803 O21: GPU Hardware and Software.
To get into MLE/AI Data Engineering, I would start with a brief introductory ML course like Andrew Ng’s on Coursera
Andrew Ng's course is great for the learning NN's from scratch, but not understanding how NN's fit in the broader discipline of AI.
Just do Andrew NG's course.
It will show you the maths, you'll build simple neural nets (from maths!) that can read digits and scale it from there.
While doing it, you may struggle with some of the maths, just take a deep breath and invest sometime to fill the gaps you need and continue.
Learn about CNNs and everything else step-by-step. It's awesome.
Andrew Ng's courses tend to be widely recommended, including by someone I know who went from physics to ML.
This was pre-gpt-3.5 release. It's classical ML.
Somehow I've never found an explanation for MLE that's intuitive and suitable for someone who didn't take graduate-level statistics already. I'm 100% on board with the introduction (MSE and cross-entropy make total intuitive sense; you can see how they penalize 'wrongness' to an increasing degree) but in the next paragraph we jump right to:
and it's straight to the grad-level textbook stuff that breezily assumes familiarity with advanced mathematical notation.
One of the reasons I loved Andrew Ng's machine learning course so much is that it eased you into understanding the notation, terminology, and signposted things like "hey this is really important" vs. "hey this is just a weird notational quirk that mathematicians have, don't worry about it too much."
I learned a lot about cryptography in Stanford Cryptography I and II courses, and about machine learning in Andrew Ng course (but it was years ago so it's kind of useless now)
I then tried to learn compilers on something Standford has called LaGunita or whatever and it was closed while I was studying it (and it never worked properly). But I wouldn't finish that course anyway, compilers are hard.
This defeatist mentality is plain stupid. There are under 20yrs old working on AI today, they went from 0 to their current knowledge in most cases 2-3 years. So what to do? Level up.
I'm an old fart too, blah, blah, blah, since the days of 2400bps modem. For anyone that wants to level up, go take Andrew Ng's ML course, watch some online lectures on neural networks, deep learning, reinforcement learning. Never in the history of the world have we had so much learning resources at our finger tips, plenty of youtube videos, blogs, articles, free books, free courses, software libraries to get into it.
Stop with the self pity, get up and dance. Start learning, at some point pick up scikit learn for shallow learning and pytorch for DL. GPUs are super cheap. You can pick up a used 3060 RTX for < $250. Or you can just rent for even cheaper. If you are technical and find yourself agreeing with the author, please snap out of it. You are going to feel lost for quite a while, but I assure you, you will find your way if you keep at it.
AI gets me excited, computing has been too boring for far too long!
I'd probably just recommend the classic Andrew Ng courses on Coursera if you want to learn Machine Learning.
Check out Coursera, there's an ML intro course by Stanford that will walk you through all the relevant math https://www.coursera.org/specializations/machine-learning-in...
Learn Machine Learning first. Andrew Ng's course is great. Then learn Deep Learning through Andrew NG's course on Coursera.
Then go through fast.ai for practical projects.
Learn PyTorch well. There are many books around. I like the one from Manning and the one by Sebastian Raschka.
Then chart your own path from there.
Many people "learned Math" in college, but these were actually mostly mindless following of algorithms to solve problems by hand. If you want proper refreshers, go through ICL's Mathematics for Machine Learning Specialization on Coursera or 3blue1brown's series on Linear Algebra and Calculus.
People getting into AI today follow courses like:
LLM University - https://docs.cohere.com/docs/llmu
and never get to learn about linear regression, bias and variance, cost function and gradient descent, regularisation and optimisation - all the good things taught by Andrew Ng in the amazing course he run 12 years ago just before creating Coursera.
Is that a good thing?
Depends on your background and what you're most interested in learning.
- if you want to really understand the mathematical foundations, you should take Andrew Ng's course (the full Stanford course not the Coursera one)
- If you want a quick sampler of ML and to know the big ideas without investing too much time, take Andrew Ng's coursera course. I'm not sure what his deeplearning.ai course is like but it seems like that one is in python.
- if you want to learn by getting your hands dirty, either fast.ai's ML course or Dive into Deep Learning is good. Fast.ai's course will get you into the nitty gritty parts of training neural networks.
I enjoyed Ng's course a lot. Fast.ai is also good, a little more demanding imho. Didn't try the dive...
The beginning of Andrew Ng’s machine learning course on coursera does that too, it touches on the math a bit and explains how to imagine gradient descent in 3d space
Didn’t do the full course, but after the first few chapters I was able to write a very basic implementation in raw python (emphasizing here on “very basic”)
Do you have Linear Algebra knowledge, and Stats 101 knowledge?
Then start with ISLR.
Then go and watch Andrew Ng Machine Learning course on Coursera (a new version was added in 2022 that uses Python).
Then read the sklearn book from its maintainers/core devs. It's from O'Reilly.
Then go do the Deep Learning Specialization from deeplearning.ai.
Then do fast.ai course.
If interested in Deep RL, watch David Silver lectures, then read Deep RL in Action by Zai, Brown. Then do the HF course on Deep RL.
This is how you get started. Choose your books based on your personality, needs, and contents covered.
And among MOOCs, I highly suggest the one by Canziani, LeCun from NYU. (I loved the 2020 version.)
The one taught by Fei Fei Li and Andrej Karpathy is nice.
These two MOOCs can substitute classic books based on quality.
I have never read cover to cover any of the famous books. I read a lot from them sticking to specific subjects.
Get to reading papers, finding implementations. Ng + ISLR will give you good grounds. Fast.ai + deeplearning.ai will give you capability to solve real problems. NYU + Tubingen + Stanford + UMich (Justin Johnson) courses will bring you to the edge.
You need a lot of practical experience that aren’t taught anywhere. So, get your hands dirty early. Learn to use frameworks, cloud platforms, etc.
Then start reading papers.
A crystal clear grasp on Math foundations is a must. Get it if you don't have already.
ISL is a legit good book. Has the correct amount and balance or rigor and application.
The explanation, examples, projects, math- all are crisp.
As the name suggests, it is only an introduction (unlike CLRS). And it does serve as a great beginners' book giving you proper foundation for the things that you learn and apply in the future.
One thing people complain about is it being written in R, but no serious hacker should fear R, as it can be picked up in 30 minutes, and you can implement the ideas in Python.
As someone with industry experience in Deep Learning, I will recommend this book.
The ML course by Andrew Ng has no parallel, though. One must try and do that course. Not sure about the current iteration, but the classic one (w/ Octabe/MATLAB) was really great.
I personally consider Linear algebra to be foundational in AI/ML. Intro to Linear algebra, Gilbert Strang. And his free course on MIT OCW is fantastic too.
While having strong mathematical foundation is useful, I think developing intuition is even more important. For this, I recommend Andrew Ng's coursera courses first before you dive too deep.
Andrew Ng's online courses are great to get your feet wet. You use Octave/Matlab to implement the basics of many machine learning models from scratch, and build yourself up to using python to design several popular deep learning models including convolutional and transformers. It's not required, but a good idea to understand at least the basics of linear algebra and calculus.
Definitely do the fast.ai course. Totally worth it.
But also use ISLR, Goodfellow, Bishop, etc.
Start with Andrew Ng's ML, then do the first part of Aurelien Geron book, then do Ng's DL specialization, then do fast.ai. Then learn PyTorch. A great book would be Sebastian Raschka's book. Also d2l.ai. A fast-paced, but really good course would be the Neuromatch DL tutorials.
Then move forward based on your interests.
Yann LeCun has THE best MOOC on DL on YouTube.
For the Math, I majored in Physics, so stuff came naturally. I suggest Imperial London's MOOC on Mathematics for ML specialization, Robert Ghrist's Calculus course, VMLS book for Linear Algebra. For stats, haven’t found a good one yet.
What you read, how much- these all depend on what you want to do. Where do you want to see yourself, and so on.
If you just want to brag about DL and put it on your resume so that you can get a job writing SQL queries and make PowerBI presentation as a "Data Scientist", then the bars are low.
If you want to do some DL, then that is another league altogether.
You need to be able to quickly read papers, understand ideas, use those for your own projects or papers.
Makes sense?
Every time I read a slogan like "learn X in Y weeks" or "learn through games instead of lectures", I have to think of a Peter Norvig piece: Teach Yourself Programming in Ten Years.
Honestly instead of taking paid courses that promise to teach incredibly difficult subjects in weeks, just take one of the longer courses like Andrew Ng's that start with the fundamentals and are free. Bootcamp style education is awful.
https://norvig.com/21-days.html
I would suggest to start with reading The Foundations of Mathematics by Stewart and Tall: https://global.oup.com/ukhe/product/the-foundations-of-mathe...
This will give you an understanding of which topics make sense to you and which you can barely follow. From that, you will need to catch up but where from depends only on your level. I guess, you might have some Khan Academying to do to catch up with the school program.
After you are done with The Foundations of Mathematics, I think you can take Andrew Ng's courses and go from there. I also use https://www.routledge.com/A-Concise-Handbook-of-Mathematics-... as a general reference when I need to look up a topic a have a rather vague recollection of.
A parameter is a scalar value, most of which are in the attention matrices and feedforward matrices, you also hear these called “weights”. Any intro to DL course will cover these in detail. I recommend started with Andrew Ng’s Coursera class on Intro to Machine Learning, although there may be better ones out there now.
I am not an expert or industry practitioner of either recommendation engines or anomaly detection, just I meant simple and useful things to add to a website.
Suppose your website has posts and you want to flag posts when they have abnormally high likes because they might be great reading or complimenting your new release. You could collect a dataset of likes after a day, X, of each post. Then calculate mean and variance, fit a normal distribution[1]. Then calculate z such that P(X >= z) = 0.01 (1%). z represents the cut off point at which typically only 1% of posts are above. Then when a post is above z say 1000 likes then you see what all the fuss is about.
I am just talking about applying 16-18 school maths in a simple way, to point out unlikely events. Of course the distribution of likes may not look like a normal curve if you plot (number of posts with x likes against x) so a different distribution may make more sense. It may not be a perfect model but just a quick and dirty thing to try, :).
Personally I enjoyed completing the free Andrew Ng Machine Learning course[2] on Coursera which covers this and quickly training a simple recommendation engine for movies. It also covers multi-variate Gaussian distributions if you want to flag based on more than one criteria. For this course, the maths is relatively accessible and they go over what you may have forgotten so you can pick up maths as you go along.
Of course you can go far more complex if you like but I don't know much about that.
[1] Normal distribution https://en.wikipedia.org/wiki/Normal_distribution
[2] https://www.coursera.org/learn/machine-learning
If you actually want to understand and implement neural nets from scratch, look into 3Blue1Brown's videos as well as Andrew Ng's course.
https://www.3blue1brown.com/topics/neural-networks
https://www.coursera.org/learn/machine-learning
Yes, a simple approach would be to use a convolutional network for multiple object detection. You would need a training set that was transferable. In your case I think you would best use a training set generator and generate a bunch of webpages with known image locations and sizes. Coursera has a video and links to references on efficient multiple object detection, look at the Andrew Ng courses. I suspect this would be an easy application. You would probably want to use PyTorch or Tensorflow to specify the model and use a GPU or GPU instance if on the cloud to accelerate the training sufficiently to explore hyper-parameters intuitively. Selenium may help with the test set generator if you can extract the image coordinates relative to the browser window with that and then generate a screen shot of that application window. You may need to generate 100k or so such randomly generated pages with multiple rectangles to reach very good accuracy, but that number is very hand wavy.
E.g. you need to do your best to generate a uniform distribution of web pages, and then render them with a real browser, extracting the image rectangle coordinates somehow to generate the labeled test set. Then feed that into a convolutional network (or R-CNN) with back-prop's error metric being a function of the R-CNN's output objects vs. the training set labels. This should then converge on correct results and you should plot the convergence of the error metric to see when it asymptotes. If the quality is sufficient at asymptote you are done, if not you may have to change structure and/or hyper-parameters.
I've taken both classes. You can take Andrew Ng's class with no prior AI experience. Though Math is a nice addition. Every concept is broken down and explained. You actually get to build real models using python and numpy. This was probably the best course I took for an introduction.
Yann LeCun class has a ton of information and you can easily get lost. I spent a great deal of time after hours trying to figure out what was being said in class. I still can't tell you what energy based models are. Not to take something away from the class, but you will need a whole lot of resources to come out learned in these classes.
The option here is not either or. You will be better off starting with Andrew's class, then muster through Yann's.
I think it always helps to have a project to apply things to as you're learning something, even if it means coming up with something small. While preparing, I found it helpful to read for at least an hour each morning, and then divided the rest of the day into learning vs. "diving in" as I felt like it.
Getting deep into RL specifically wasn't so necessary for me because I was just replicating AlphaZero there, although reading papers on other neural architectures, training methods, etc. helped with other experimentation.
You may be well past this, but my biggest general recommendation is the book, "Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow" to quickly cover a broad range of statistics, APIs, etc., at the right level of practicality before going further into different areas (for PyTorch, I'm not sure what’s best).
Similarly, I was familiar with the calculus underpinnings but did appreciate Andrew Ng's courses for digging into backpropagation etc., especially when covering batching.
I'm one of the faces of the Coursera ads (they interviewed me about career change from drummer to deveoper). I feel exactly the same. When it was free, and when for profit institutions were experimenting more too, it felt like a revolution. The Stanford 'Startup Engineering' course on Coursera was only offered back then and it was amazing. Andrew Ng's ML, Prinston's Algorithms, all free including certificate.
I used Udacity CS101, Google Python Class and loads of other free resources and changed my life. It was true when I said Coursers changed my life. What has been slightly sad as it took Coursera years to start seriously advertising with my face, and even when they interviewed me they were almost beyond recognition.
It makes me sad I just happened to learn CS and software development during the online education wild west. That's what I dreamed of for everyone. Top level education for everyone, for free. I didn't care about certificates. I had knowledge and projects I could prove.
It may not all be doom and gloom, but when people ask me for recommendations after seeing advert, I sigh a little, because situation isn't as great as it was.
https://youtu.be/Z1lqnyEp38o
To think that i took the original ml-course Ng) and ai-course (Norvig) years ago. Great courses.
Currently I find Udacity model better than Coursera: I can get a course and take it in my own time without having to stick to a schedule. Also the fact that I pay for what I consume. I personally dont like subscription model for these things you may use once or twice every yesr
This was the first course I did on Coursera and it is by far the best introductary course for machine learning I have ever seen, but my sample size is pretty small ;)
I can think of many - I have taken several starting since 2013. The tricky thing is that Coursera classes seem to get merged, re-mashed or otherwise re-branded. And as such only one currently is listed in my "Completed" courses section of my profile.
Having said that and with the caveat that these probably changed since I taken them, I recommend the following:
- Cryptography - https://www.coursera.org/learn/crypto - great introduction to the fundamentals and math behind cryptography. A lot of theory but also some practical exercises. This is my top recommended.
- Machine Learning - https://www.coursera.org/learn/machine-learning - a good introduction to the basic of machine learning; focuses on octave/matlab and does not dive into frameworks like scikitlearn or tensorflow
- Introduction to Interactive Programming with Python - https://www.coursera.org/learn/interactive-python-1 - I took a course from Rice University on Python programming through making games that was fun. As far as I can tell, this is the modern incarnation in two parts.
- Software Security - https://www.coursera.org/learn/software-security - goes into stack / overflow exploits, tools for testing, and web-based attacks
- Functional Programming Principles in Scala - https://www.coursera.org/specializations/scala - this was a good introduction to scala and functional programming - it got me thinking in a different way
- C++ for C Programmers - https://www.coursera.org/learn/c-plus-plus-a - I think this was the first coursera class I took. This course dove into the C++ STL and a lot of modern features introduced in C++11.
It is never too old to transition and 36 is not old. Have a growth mindset. Don't listen to anyone who says it is too old to switch. You have one life to live, do what makes you happy.
Here is one advice, check that you want to move to AI/ML for the right reasons, not because it is the next cool thing.
One strategy: Take some of the courses in ML, for example Andrew Ng's course on ML and Deep learning. If you enjoy the course, next find a real project to do and see if you truly enjoy it. If you do enjoy it then switch, otherwise Java programming is always there. I have found real ML is a lot different from Andrew Ng's course for example, pretty much like doing algorithms in class is different from real world programming.
Maybe off topic ... Is Stanford ML expert some type of accreditation? How do you become a Stanford ML expert? :) Attending the (excellent) Stanford ML online course on Machine Learning or do I have to read an ML book on Stanford Campus?
I urge beginners not to learn these things from blog posts, even if it seems more digestible. Take a reputable textbook and you will be better off. Or take a MOOC, like Andrew Ng's or courses from MIT and Stanford on Youtube. It may look harder, but it has all been carefully didactically structured. It's harder on the long run to learn from a non-expert.
There's so much confusion out there, blog posts confusing cross-entropy with "softmax loss", gradient descent with backpropagation etc. etc.
Another resource I'd recommend is Andrew Ng's Machine Learning course on Coursera.
Same with MOOCs I think. It's too bad most of them are so bad they are not worth taking, but the ones that are good (e.g. Andrew Ng's machine learning MOOC) are having insane impact even though most people don't finish them.
I have completed both of the following:
1) https://www.coursera.org/learn/learning-how-to-learn
2) https://www.coursera.org/learn/machine-learning
I highly recommend both of them, if they cover a subject you're interested in, though if you only have time for one, they are listed in descending priority order.
Given how low it's time+energy requirements are, and how large the pay-off has been, I recommend Learning How to Learn by Dr. Barbara Oakley to everyone regardless.
years ago on the first ML hype wave I completed the excellent MOOC by Andrew Ng. In that course, he did go through the math and it was helpful to me to understand what was going on under the hood, but even then the value of ML wasn't understanding what it was doing, but understanding if your model was doing something well. I think your take that using packages created by others will be mostly what we do moving forward, and that's also true of pretty much all software development.
Machine Learning:
* https://www.youtube.com/watch?v=UzxYlbK2c7E: Andrew Ng's machine Learning course, the recommended entry point by most people
* https://mlcourse.ai/ : More kaggle focused, but also more modern and has interesting projects
Do both courses simultaneously, take good notes, write useful flashcards, and above all do all the exercises and projects
Deep Learning
* https://www.fast.ai/ - Very hands-on, begin with " Practical Deep Learning for Coders" and then "Advanced Deep Learning for coders"
* https://www.coursera.org/specializations/deep-learning : More bottom-up approach, helps to understand the theory better
Do those two courses in parallel (you can try 2 weeks of coursera followed by one of fastai in the beginning, and then just alternate between them), take notes, write good flashcards and above all do the exercises and projects.
After that you will be done with the beginning, your next step will depend on what area interested you the most, and getting way too many resources right now can be extremely confusing, so I would recommend doing a follow-up post after you worked through the above resources. Also as non-ML stuff I recommend Scott Young's Ultralearning and Azeria's self improvement posts (https://azeria-labs.com/the-importance-of-deep-work-the-30-h...)
I started with with the machine learning course[0] on Coursera followed by the deep learning specialization[1]. The former is a bit more theoretical while the latter is more applied. I would recommend both although you could jump straight to the deep learning specialization if you're mostly interested in neural networks.
[0] https://www.coursera.org/learn/machine-learning
[1] https://www.coursera.org/specializations/deep-learning
The answer to this question depends on your level of computer & math proficiency. Some folks here have been debating about the relative merits of practice vs. theoretical foundations, but this dispute makes some assumptions about where you are starting from and where you are most comfortable. The fastest way to learn something is to fit it into a framework that you already understand. If you have a PhD in theoretical physics/abstract mathematics (like a lot of ML researchers), then the more mathematical (theoretical) frameworks will be a good way to build deep intuitions. If, on the other hand, you are more into applied data analysis, then you will probably find that working on applications will be the easiest way to go.
Personally, I enjoyed both Andrew Ng's and Geoffrey Hinton's respective courses on ML and Neural Networks on Coursera. You may also want to check out Michael Neilsen's online essay on deep learning (http://neuralnetworksanddeeplearning.com). Ultimately I would also encourage you to supplement your understanding by applying this work to your own applications. The universe is often the best teacher.
Whoah!
Okay, counter-point: in my experience Udacity has the highest quality content on the toughest technical subjects. To me, it is evident that an enormous amount of work has gone into making the lectures and exercises effective.
Coursera has more courses, and the quality varies more but some of them are excellent. In any case, Coursera lectures come from university partners we are all familiar with, so people can decide if they like the Andrew Ng Machine Learning Class or Geoff Hinton on Neural Networks.
I am not affiliated with either of them.
I learnt eigen decomposition, Hessians, PCA, backpropagation, CNN, dropout, maxpooling etc.
The page you linked above is the derivation of PCA using linear algebra.
First part derives the encoding matrix from the decoding matrix. 2nd part derives the encoding matrix by minimizing the L2 norm.
If you find the math too heavy, you should take Andrew ngs course at Coursera (not his Stanford lectures, which follow a pattern similar to this book). Or pick up any book targeting programmers, machine learning for hackers etc.
If you want a gentler introduction, I recommend the first module of Andrew Ng’s Machine Learning class on Coursera. He also explains things intuitively before jumping into the math. Even when he uses math, he makes sure you understand the notations and what equation means.
I second this suggestion; I took that course when it was called "ML Class" during the Fall of 2011 (yep, I was one of the guinea pigs for what became one of the first courses of Coursera). It was an excellent course.
Here's an example of what one student of the ML Class built, after being inspired by what he was learning and videos that played during the course:
https://blog.davidsingleton.org/nnrccar/
It kinda shocked me at the time, because I knew quite a bit about ALVINN from books and articles I had read as a teenager in the 80s and 90s. This guy had created the same thing using a cell phone and a cheap RC vehicle! Ok, there was also an Arduino and computer involved - but it really hit home the fact that technology around neural networks had advanced quite a bit!
I also took the other course, "AI Class", but due to personal issues I had to drop out about halfway through.
The next year, after Udacity started, they introduced a course similar to AI Class called "How to Build Your Own Self-Driving Vehicle" (it's called something else today - something like "Robotics and Artificial Intelligence 302" or something like that).
That class was done in Python, and taught me even more about AI/ML - with a focus towards self-driving vehicles of course. Things I learned about that I struggled with or had no real concepts of before:
1. SLAM (Simultaneous Localization and Mapping) 2. Path Finding algorithms (A* and the like) 3. Kalman Filtering (what it is for, how it works) 4. PID Algorithm (how to implement and tune it) 5. More neural network stuff...
...and many other things. Another very excellent and free course to take if you're interested in learning this stuff.
This. I can tell anybody thinking about taking this course, do it! The way Jeremy Howard explains things is so down to earth and clear, you'll be surprised at how easy all this ML/DL nonsense is :)
But seriously, there's a lot to learn here, and if you are looking to make your entry into the hype field of the decade I would strongly suggest starting here first
(Alternatively, for the more math inclined, perhaps take Andrew Ng Coursera course first: https://www.coursera.org/learn/machine-learning)
Andrew Ng's courses on Coursera (Machine Learning) are the best ones IMO. Gives you the introductions to the math behind and some practice.
I enjoyed Andrew Ng's class on Coursera, which introduces basic concepts. For "real world," maybe find a dataset on data.world and try to figure out how to answer a question that is interesting to you?
https://www.coursera.org/learn/machine-learning
I'm same as you too but I did have some background in math from college but I've been rusty on them already.
- I know it's said over and over again but KhanAcademy is pretty good for high-school math - For probability, I'm also going over this website[1] at the moment. Maybe someone else has other recommendations too - Linear Algebra is next on my list but others have given some reference for it already - Andrew Ng's course on Machine Learning from Coursera is also another frequently recommended if you are trying to learn math for it
[1] https://www.probabilitycourse.com/
I read some tutorials on back propagation for this, but if you want to get into ML in general, I highly recommend fast.ai and Andrew Ng's free Stanford ML course.
What's the difference between Udacity and Udemy? Are they the same company?
Also, I'd love to hear about your favorite online courses. I took Andrew Ng's Machine Learning and Dan Boneh's Cryptography (both on Coursera) and they were excellent.
Understanding the notation is only the first little step in understanding the underlying principles. There are a lot of machine learning books/classes that will cover the underlying principles you need to understand what's covered in the book. Exactly which one to recommend depends on your background, but Andrew Ng's Coursera Course [1] has been a staple for people without a lot of math background.
[1] https://www.coursera.org/learn/machine-learning?authMode=log...
Here are some others that are also commonly recommended on HN:
Andrew Ng's Machine Learning MOOC: https://www.coursera.org/learn/machine-learning
Andrew Ng's Deep Learning Courses: https://www.deeplearning.ai/
David Silver's Reinforcement Learning Youtube Series: http://www0.cs.ucl.ac.uk/staff/d.silver/web/Teaching.html
Fast.ai Courses on ML and Deep Learning: https://www.fast.ai/
So we had this idea of a new feature for our product. The only way to quickly do it was to somehow implement a machine learning algo and that would give us the result that we wanted. Viola!! It seemed simple.
Now our company doesn't have any machine learning expert or a data science genius. Going for hiring one would take time. Taking someone up on contract would be very expensive (our CEO wasn't ready to shell out that kinda money). So the task fell on me. They asked me to go through the multitudes of Machine leaning MOOCs out there and get a working prototype ready in 2 weeks.
I had already done Andrew Ng's course back when it came out for the first time. But my memory had faded for the lack of practice.
I re-ran the course again. I went over a couple of online ML books too.
Then I started thinking of the problem at hand. Unfortunately, it turned out to be a chicken and egg problem. For the feature to work perfectly we needed a large amount of training data to train our models. But without the feature actually deployed, we didn't have any way to collect any training data.
So we ultimately fell back to simple algo, that took it's decisions based on a few hard coded rules. Things have been working fine till now.
Thankfully, someone has translated the assignment skeletons from Andrew's course into Python, so there's no need to use Octave if you go through that excellent course today:
https://github.com/dibgerge/ml-coursera-python-assignments/b...
Have you looked at fast.ai's ML course? It might also be worth a try. http://forums.fast.ai/t/another-treat-early-access-to-intro-...
I'd also recommend the Andrew Ng Coursera course, which I talked about here: https://harveynick.com/2018/04/25/some-notes-on-the-andrew-n...
I would not dive into GANs right off the bat. Ng's course is a good start, maybe also buy or borrow a copy of Russel and Norvig's "Artificial Intelligence: A Modern Approach"[0]. If you decide to look into deep learning, I've heard good thing about Bengio's book (see also Hugo Larochelle's lectures on YouTube) and if you want to learn more about reinforcement learning, "Reinforcement Learning: An Introduction" by Sutton and Barto is frankly amazing (and there's a great course on YouTube by David Silver).
From reading your code, you're probably familiar with most of the stuff you need to learn ML, so I thought I would take a stab at explaining how you get from "no machine learning experience at all" to "surely GANs will solve all our problems". Broadly speaking, a lot of machine learning is just deciding what your goal is (carve an image, play a game, classify objects, group similar data points, etc.) definining an objective function (e.g., number of sharp edge removed, player score, a classification loss function[1], distance over points in a cluster) and then maximizing or minimizing that objective function.
In classification, you want a function `f(.)` such that given a data point `x`, `f(x) ≈ y`, where `y` is the correct label for that data point. Say you have a bunch of these data points in a matrix `X`, and a vector of correct labels `y`–– how do you come up with a function that correctly classifies the data in `X`, and how can you do it so that data you might collect in the future would also be correctly classified?
You could do this with linear least squares (come up with weights `w` to minimize `|f(x) - y|`, with f(x) = x^T w) but unless your data is cleanly separable, that probably won't work so well. There are a lot of possible techniques people have tried, but lately neural networks have become popular because they are powerful approximators, relatively easy to deploy, and pretty versatile. But the training process[2] still boils down to tweaking weights until `|f(x) - y|` is as low as you can make it.
General adversarial networks are kinda a tweak on this idea-- you start with a bunch of images, and you train a network to generate images that are "similar" to them; call the initial images the "real" images and the generated ones "fakes". You want your network to generate fakes that are similar to the real images. How do you define similar? Well, you have a second network whose objective is to distinguish between real and fake images. You train both at the same time, using feedback from the second network to improve the first network, and using the images generated by the first network as training examples for the second.
Initially, the first network will generate splotchy garbage that the discriminator can easily tell apart from the real images. Over time, the first network improves, creating increasingly realistic fakes, while the second network gets better at determining whether an image is real or generated. After much training, you have two networks: one that is good at generating images that are similar to the seed images, and another network that is good at identifying if an image is real or fake.
––––––––
0. It even has a website and example code http://aima.cs.berkeley.edu/
1. https://en.wikipedia.org/wiki/Loss_functions_for_classificat...
2. See here for a nice explanation of the backprop algorithm, http://neuralnetworksanddeeplearning.com/chap2.html or if you want the OG (original Geoff) paper on the topic, it's in the proceedings for Parallel Distributed Processing 1986 -- "Learning representations by back-propagating errors", Rumelhart, Hinton, and Williams.
Recent report from my mother, who's a physicist with a strong math background, no statistics: Take the coursera intro course.
She said that the first hour or two were a little frustrating for the kinds of reasons you mention. After that, she could skip the "long" explanations & examples by going to the math directly. You'll progress 5X faster than normal after that.
She "got it" by creating ML "proofs" for math problems without doing math. She didn't like all the image recognition examples. Said it's a confusing place to start.
You are probably stuck with learning from a source that assumes you don't know math, because most people (including me) don't. This course is probably not for you if you want theory, as it's focused on "using" ML algorithms, not writing them. That said, this might be useful to you if you already have stuff that you want to do, and need a way of doing the "sticks and duct tape."
I've done a few from coursera and the best MOOC's I have taken are:
Machine Learning = really good overview of ML, well explained. I did this first but struggled with some of the maths, so took maths refreshers afterwards
Calculus 1 - I have never had a maths class where the Professors made the content so interesting and with such enthusiasm.
Human-Computer Interaction - worth doing if you are building websites / apps
Machine Learning by Andrew Ng on Coursera is the best MOOC I've taken so far. It has great explanations on complex topics, fun activities, and a really well put together curriculum on machine learning.
Andrew Ng's class was great! For a tougher class that focuses on one of the technologies, I recommend Geoffrey Hinton's "Neural Networks for Machine Learning" on Coursera. Really eye opening for me, and fairly close to the leading edge in deep learning, as far as I can tell. I felt that the exercises were more detailed and challenging than in Ng's class, and thus I ended up learning more.
The "Bitcoin and Cryptocurrency Technologies" on Coursera helped me gain an understanding cryptocurrencies. Until I took that course I knew very little about the subject.
It's possibly a little dated now, but it's a good primer.
https://www.coursera.org/learn/cryptocurrency/
I'd love to hear what other cryptocurrency courses others recommend.
As many others mentioned, Andrew Ng's course on Machine Learning on Coursera was also very good.
https://www.coursera.org/learn/machine-learning
Algorithms I and II by Tim Roughgarden on Coursera -- amazing teaching style with a lot of depth and math background.
ML by Andrew Ng is also another great example of a MOOC that could be executed very well.
From non CS related courses I really liked Intro To Finance by Gautam Kaul, also on Coursera.
Hey!
I'd suggest starting with this excellent course:
https://www.coursera.org/learn/machine-learning
And then dive into competitions on kaggle.com.
Then following up with the deeplearning.ai specialization here: https://www.coursera.org/specializations/deep-learning
and the http://www.fast.ai/ courses.
You'll be up to speed in no time :)
I did Coursera's "Introduction to Machine Learning" by Andrew Ng back in 2013 and loved it. Great tutor, good course material - and it looks like Coursera is still offering this course though I am not sure if it is still free. The course is language-agnostic and uses Octave (an open-source Mathlab clone) for assignments and examples.
Andrew Ng’s older machine learning MOOC class was excellent. I took it once, and took it again a few years later. In the last 8 months I also took his new deep learning set of courses. All really good stuf! (I have been working in the field since the 1980s, but constantly refreshing helps me. And Andrew’s lectures are great fun, he is a teaching artist.
I'd like to point out the fact that there is a split within Machine Learning into "classical" and "Deep Learning"-based one. While Deep Learning are just neural networks with Big Data on GPU, they obliterated many areas of classical ML. What you really want to learn is Deep (+Reinforcement) Learning, or what is coming known as differentiable programming. For classical ML, learn how to use Spark's MLlib and take Ng's Coursera course, that might be all you need from practical point of view.
Here's a hypothesis... When you have top school's materials online, it makes no sense to go to a second tier, or n-tier school other than for the networking. And by definition a 'n-tier' school is not going to be all that great at providing networking opportunities.
Would you take a machine learning course from an unknown, likely outdated professor when you could listen to Andrew Ng for free?
What the internet has brought is a 'winner takes all' for education.
Check out online courses. Coursera, EDX, etc. Andrew Ng's intro to machine learning course is a nice way to get up to speed with some basic concepts without diving too far into math. It doesn't cover any state of the art machine learning concepts, though.
Some other resources I have bookmarked:
- Convolutional Neural Networks for Visual Recognition Youtube playlist [1]
- Deep Learning for Self-Driving Cars [2]
- Natural Language Processing with Deep Learning [3]
[1]: https://www.youtube.com/playlist?list=PL3FW7Lu3i5JvHM8ljYj-z...
[2]: https://selfdrivingcars.mit.edu/
[3]: https://www.youtube.com/playlist?list=PL3FW7Lu3i5Jsnh1rnUwq_...
I took the 11-week Coursera course last year (and even paid for the 'verified' certificate). It was worth it, I think. I may not have a deep understanding of the different ML algorithms that were presented, but at least I know of their existence, and to what problems they are suited (and then I can look up the details when needed).
Granted, that's not enough to actually find work doing anything ML-related, as there are now plenty of experts with years of experience out there on the job market, but hey, I like learning stuff for the sake of it. I'm thinking of taking one of the follow-up courses on NN.
Really looking forward to getting stuck into this course. I’ve a background in fluid mechanics, in particular writing code (in Matlab but more recently with python) to identify structures in experimental and numerical data. In essence this is very similar to classical image analysis. Looking to machine learning seems a logical next step. I’ve been working through Andrew Ng’s course and while it is very good it suffers from fundamentalitis to a degree. However I do plan on completing Ng’s course first.
I'm starting the fast.ai courses as a recent stat grad looking to expand my knowledge. Heard many good testimonials. Besides that, I think Andrew Ng's Coursera offerings - intro to ML and the newer NN specialization - are great first steps. Personally I have a hard time learning from videos, so I refer to my copy of Pattern Recognition and Machine Learning by Bishop.
If you want a linear algebra text, I enjoy Strang's Introduction to Linear Algebra
I started with Andrew Ng's ML courses, which take a bottom-up approach beginning with the math. After finishing the deeplearning.ai track, I started rounding out my skillset with some data science and R programming classes, so I could be more comfortable working with unfamiliar data.
The fast.ai courses are a more top-down approach to ML, and there are plenty of good reasons for taking this approach. You'll start getting practice with libraries like Tensorflow right away. However, if you have a fairly strong math background and linear algebra doesn't give you nightmares, I highly recommend the Andrew Ng courses. A deep (ha!) understanding what's going on "under the hood" in ML will help your debugging, inform your strategy, and make your code better in the long run.
This is kind of a masters degree course i created for myself to get knowledge of Machine Learning from bottoms up
First, you need a strong mathematical base. Otherwise, you can copy paste an algorithm or use an API but you will not get any idea of what is happening inside Following concepts are very essential
1) Linear Algebra (MIT https://ocw.mit.edu/courses/mathematics/18-06-linear-algebra... ) 2) Probability (Harvard https://www.youtube.com/watch?v=KbB0FjPg0mw )
Get some basic grasp of machine learning. Get a good intuition of basic concepts
1) Andrew Ng coursera course (https://www.coursera.org/learn/machine-learning)
2) Tom Mitchell book (https://www.amazon.com/Machine-Learning-Tom-M-Mitchell/dp/00...)
Both the above course and book are super easy to follow. You will get a good idea of basic concepts but they lack in depth. Now you should move to more intense books and courses
You can get more in-depth knowledge of Machine learning from following sources
1)Nando machine learning course ( https://www.youtube.com/watch?v=w2OtwL5T1ow)
2)Bishops book (https://www.amazon.in/Pattern-Recognition-Learning-Informati...)
Especially Bishops book is really deep and covers almost all basic concepts.
Now for recent advances in Deep learning. I will suggest two brilliant courses from Stanford
1) Vision ( https://www.youtube.com/watch?v=NfnWJUyUJYU )
2) NLP ( https://www.youtube.com/watch?v=OQQ-W_63UgQ)
The Vision course by Karparthy can be a very good introduction to Deep learning. Also, the mother book for deep learning ( http://www.deeplearningbook.org/ )is good
I know how it feels. SVM's are an old and tried classification method that was in fashion before the DL craze.
One advantage of SVM's is that they don't use all data points to decide on the separation plane, just the closest points to the gap (the support vectors), making it more invariant.
Another advantage is that they can efficiently perform a non linear classification using the kernel trick, implicitly mapping their inputs into high dimensional feature spaces (here kernel means a distance function between two data points).
https://en.wikipedia.org/wiki/Support_vector_machine
https://en.wikipedia.org/wiki/Positive-definite_kernel
I recommend Andrew Ng's ML course for learning about SVM's.
From my personal experience I would recommend Andrew NG's courses. (I am on the second one right now). I have done fast.ai's first course which is low on math and more practical (you directly jump to coding, and then try to make some sense of things.) CS231n is definitely a great option, however I chose coursera because I needed deadlines/reminders of Coursera to keep me motivated and finish something. If you are looking for more options, there are lot more courses listed here in Machine Learning category : https://github.com/prakhar1989/awesome-courses however I don't know if someone has reviewed them.
While I agree with others that more is better, if you can take only one course, I strongly recommend taking Andrew Ng's. While it is true that you don't need to be able to design and understand 'nets from scratch to be able to use them, I agree with most of the brightest minds in DL that you won't get too far if you don't at least have an intuition for the math behind it. And Ng's course really only gives you that - an intuition. It does an excellent job at ensuring participants understand the bare minimum to do any kind of serious work. Learning ${your favorite framework}'s API will be a breeze if you understand the "why" already.
1. Start with the fast.ai courses. It's an applied deep learning course using state of the art techniques. 2. For classical machine learning (regression, etc...), Andrew Ng's course on Coursera is widely considered "the basics" 3. As you progress, check out CS231 and CS224 from Stanford for state of the art image processing and natural language processing techniques. The lecture videos are on YouTube and the course assignments are available online. The third course I recommend is Geoffrey Hinton's neural networks course on Coursera (he is one of the most important researchers in the field). 4. If you're an application engineer, focus on using existing tooling to build cool projects. Keras and scikit-learn are great out of the box tools. 5. If you are more research oriented, you can start reading papers. In Silicon Valley, there's a meet up group that reads papers every Monday and tries to implement the algorithms in the paper. It takes a while to get to this level, but try not to get overwhelmed. Experts spend 7 years studying this stuff full time to get a PhD. 6. You really don't need much math to get started with ML. A high school understanding of calculus and some basic understanding of numerical optimization are the two main concepts you need to know. If you want to get into the research, there'll come a time when you will need more advanced math, but in my experience you can pick that up as you go along if you are curious.
Maybe you could start an AI study group online? The Silicon Valley study group was great, but I was just visiting.
Here are the resources I found useful: ========================================== Advices from Open AI, Facebook AI leaders
Courses You MUST Take:
Machine Learning by Andrew Ng (https://www.coursera.org/learn/machine-learning) /// Class notes: (http://holehouse.org/mlclass/index.html)
Yaser Abu-Mostafa’s Machine Learning course which focuses much more on theory than the Coursera class but it is still relevant for beginners.
(https://work.caltech.edu/telecourse.html)
Neural Networks and Deep Learning (Recommended by Google Brain Team) (http://neuralnetworksanddeeplearning.com/)
Probabilistic Graphical Models (https://www.coursera.org/learn/probabilistic-graphical-model...)
Computational Neuroscience (https://www.coursera.org/learn/computational-neuroscience)
Statistical Machine Learning (http://www.stat.cmu.edu/~larry/=sml/)
From Open AI CEO Greg Brockman on Quora
Deep Learning Book (http://www.deeplearningbook.org/) ( Also Recommended by Google Brain Team )
It contains essentially all the concepts and intuition needed for deep learning engineering (except reinforcement learning). by Greg
2. If you’d like to take courses: Linear Algebra — Stephen Boyd’s EE263 (Stanford) (http://ee263.stanford.edu/) or Linear Algebra (MIT)
(http://ocw.mit.edu/courses/mathematics/18-06sc-linear-algebr...)
Neural Networks for Machine Learning — Geoff Hinton (Coursera) https://www.coursera.org/learn/neural-networks
Neural Nets — Andrej Karpathy’s CS231N (Stanford)
http://cs231n.stanford.edu/
Advanced Robotics (the MDP / optimal control lectures) — Pieter Abbeel’s CS287 (Berkeley)
https://people.eecs.berkeley.edu/~pabbeel/cs287-fa11/
Deep RL — John Schulman’s CS294–112 (Berkeley) http://rll.berkeley.edu/deeprlcourse/
I am sitting in the same boat. Being a web developer for a couple of years, I wanted to try out a different domain. So I started to take Andrew Ng's course on Coursera [0]. Highly recommended. I supplement the course with audio and text by listening to the Machine Learning Guide Podcast [1] and by reading The Master Algorithm [2].
In addition, I started to apply my learnings in JavaScript [3]. Even though it's not the best language for ML, it makes it simpler to learn only one new thing and stick to known technologies for the rest. I have lined up ~7 articles about ML in JavaScript, so if you are interested, you can keep an eye on it :)
- [0] https://www.coursera.org/learn/machine-learning/
- [1] http://ocdevel.com/podcasts/machine-learning
- [2] https://www.goodreads.com/book/show/24612233-the-master-algo...
- [3] https://www.robinwieruch.de/linear-regression-gradient-desce...
Andrew Ng's courses are excellent. Another pretty good Coursera course is Machine Learning Foundations from the University of Washington. It is very high level and novice friendly. While it covers non of the math and very little programming it does give a nice quick introduction to the most popular ML techniques out there and when to use them. It all depends on what level you are interested in starting. They also have follow up courses that go deeper into the different techniques.
I highly recommend Andrew Ng's Coursera courses for both Machine Learning and Deep Learning. Good for beginners, Math is taught along with the course, and gets you a solid foundation:
https://www.coursera.org/learn/machine-learning/
https://www.coursera.org/learn/neural-networks-deep-learning...
No.
I recommend the ML course by Andrew Ng (I did it on coursera) and the AI course by Sebastian Thrun and Peter Norvig.
If you can spend 5 or 6 hours a week, take Andrew Ng’s machine learning class on Coursera.
https://www.coursera.org/learn/machine-learning
Not a book but excellent primer course
Great reading! i would like to kindly suggest the following:
"The minimum values the error would take can be computed using gradient descent, an optimization algorithm that is capable of finding local minima of a given function"
i think it's better to clarify that gradient descent "can find" global minima, there are some neural networks cost functions that are non convex, in that case the gradient descent can be stuck in local minima, as Andrew Ng explains in his coursera free course is that this local minima normally doesn't represent a big problem in neural networks, but as a general concept the idea behind optimization algorithms in cost functions is to find the global minima, for example in linear regression and logistic regression we have convex cost functions so the Gradient descent can find pretty close values to global minima.
Also i will suggest that the main task of Gradient Descent is not to compute the minimum values the error would take, it's to find the values of W that will get you the minimun error.
I think MOOCs are the future based on my own experience. They allow the best teachers to scale their reach. Instead of aristotle giving a lecture to a room it allows an unlimited audience who can also rewind and pause if they don't understand something.
I've been able to learn machine learning and deep learning from Andrew Ng, one of the best AI researchers in the world, among other world class researchers, for free. It's incredible when you think about it, probably one of the most significant things in human history. I've been able to learn how to code by picking and choosing the best classes from MIT, Stanford, Harvard, etc without spending a penny. Why settle for mediocre teachers at state universities when these options are available?
The major thing we need to do is find some other way to grade people other than using a college degree as a benchmark.
Have you checked out itunes u? There you can find actual lectures from too universities recorded. Mostly they require video to get the most of them but i find that just hearing the sound can still be very informative.
Specifically, stanford has some great cs material, and you could also look into andrew ng's machine learning course (i think that one was on coursera).
Totally depends on where you want to land on the engineering-AI-products to pure-AI-research spectrum.
So, what do you mean by "pursuing"?
But even still, I would caution against trying to upload a bunch of new math concepts into your brain without first understanding the ML/AI context.
I would say go through both of Andrew Ng's ML and DL courses on Coursera.
Then, pick a domain/ problem that you're interested in.
Then, read papers about how ML/AI is applied in that domain.
Then, try to reproduce a paper that you understand and are interested in.
For those who like videos, I would highly recommend utilizing Andrew Ng's Coursera ML videos for step one. I found his lectures to be good high level overviews of those topics.
The course in general lacks rigor, but I thought it was a very good first step.
So awesome. I just completed Andrew's excellent 2011 Machine Learning coursera course, and was looking where to go next. I am trying to go through the courses on fast.ai, but I don't enjoy them nearly as much compared to Andrew's teaching style and structure. Will be signing up for this!
Yes for sure. I am very math-averse (almost failed calculus 1, skated by pre-calc in high school... not my intelligence area) and though I don't have a full time job doing something in AI, I do feel like I can build models that solve real problems (and am doing so in an internship right now).
Probably start with Andrew Ng's Machine Learning course. It has a significant amount of Math in it-- try to understand it, but seriously do not worry about it. Just get the high level concepts, try to get some intuition on machine learning ideas and techniques. You don't need to do the assignments or work too hard on the course (but obviously it's helpful if you do).
Then read these. Don't worry if you're still confused at first. It's fine. Just go through 'em kinda slowly and try to see what's going on. https://iamtrask.github.io/2015/07/12/basic-python-network/ http://karpathy.github.io/neuralnets/
By now if you're still into it, I highly recommend Chris Olah's blog: http://colah.github.io/ It has some pretty complicated ideas in there, but the articles are illustrated and explained very well so you can get more of a feel for Neural Networks while also getting very excited about them.
Then it's probably time to start really building things. I would use Keras (https://keras.io/) at first, because it's very easy to get cool results without understanding everything under the hood. Do a tutorial or two, it's pretty intuitive and if you've done everything above you should understand more or less what's going on. Then try to find a cool dataset to work with that relates to something you're interested in. If you can't find anything you want to work with, then just use a classic dataset (imagenet is fine, even MNIST when you're just practicing) which will probably be less fun but will still let you learn. With whatever dataset you choose, just implement a simple model on your own without a tutorial (of course, if you get stuck referencing a tutorial is totally fine). Then see if you can tweak your model to get better and better scores. Start reading papers, you can find the newest ones on Twitter from ML researchers (Karpathy, Sutskever, Hinton, LeCunn are some names you could start with, there's probably a Twitter list out there somewhere) and then you can look at the references in those papers to keep finding more and more good ones. Implement any ideas in the paper you think are useful. Often Keras will have functionality to let you implement them easily. If it doesn't, then feel free to dip down into Tensorflow if you feel ready!
From there the world's yours. Find cool data to work with, implement papers to get a baseline measurement, and iterate in any way you can think of. It's very fun :)
The one thing is if you want to get state of the art results or do novel research you might need better hardware. AWS/Google Cloud/Floydhub are all options if you're willing to spend a little money, or you can just keep your expectations low ;)
Wow that turned out to be more of a roadmap than I wanted it to, sorry. The reason you don't need great math skills is that a lot of AI research is very intuitive-- Gradient descent can be internalized as a ball rolling down a hill, Momentum in training neural nets is like momentum in the real world, Neural nets are just manipulating data in high dimensional space... it's all stuff you can visualize instead of use mathematic symbols to depict, but since it's so much easier to write with symbols than it is to create a powerful image, symbols are used often.
To be honest I still usually skip over some equations in papers if they look daunting. Most of the time I don't need to understand them. Most of the time, I can read the abstracts, look at some figures, look at the results, and that's everything I need.
No, start with fast.ai's deep learning course. They have a top down approach so you start with connecting layers to solve deep learning problems and then dig down into the theory behind the thing.
If you want a bottom up approach, go look at Ng's coursera course.
http://course.fast.ai
The advice you've been given so far is sound. Do the Andrew Ng Coursera course. It will give you a great introduction to key areas of machine learning. Andrew Ng is a terrific educator.
The level is pitched just right - enough maths to understand what's going on and implement it yourself - but not so much that it detracts from the key focus of the course.
The exercises give you a great opportunity to actually implement the algorithms covered and really aid understanding.
ML really requires more than a tutorial. Just bite the bullet and take Andrew Ng's ML class.
To understand ML you've got to have at least a basic understanding of the math. And it's really not that difficult, especially if you find the right class/book/professor/etc.
The problem is that there are a ton of terrible writers and instructors out there.
I think it's just as important to ignore the terrible stuff (pretty much any blog post on ML) as it is to learn from the good stuff (e.g. Ng's ML course).
Andrew Ng has an excellent ML course on coursera [1] that is very light on math.
[1]: https://www.coursera.org/learn/machine-learning
I actually did take a data mining course a couple years ago in uni, and also at the time I was following Andrew Ng's course. TBH, I learn better by working on a project. Otherwise I lose interest midway.
I'd be curious to know why myself?
I'm about 2/3rds of the way thru the second term (I'm in the November cohort), and I plan to finish the nanodegree.
What I have found, though, with this second term, is that it seems to be a rehash of the CS373 course (I took that back in 2012, when Udacity first started) - though it also has taught a few new concepts for me (I like the fact that it is mostly done in C++ - which had gotten really rusty for me - but now I am liking C++ more).
I'd also recommend the Coursera ML course - from what I know, it is basically identical (if not the same material) as the original ML Class from 2011 (I took that one, which was my intro to MOOCs). If you can wrap your mind around the programming language used (Octave - basically an open-source clone of MatLab - where the main primitive is a vector/list), you'll find it does a great job at explaining many concepts that you might have had trouble with in the past (when we got to neural networks and backprop, it finally clicked for me - same with how think in terms to figure out how to parallelize processes).
Yes, I did my research but there is no such interactive tutorial online like Treehouse or Codecademy. There are so many tutorials but none of it tells you the whole path.
Here are the resources I found useful:
========================================== Advices from Open AI, Facebook AI leaders
Courses You MUST Take: Machine Learning by Andrew Ng (https://www.coursera.org/learn/machine-learning) /// Class notes: (http://holehouse.org/mlclass/index.html)
Yaser Abu-Mostafa’s Machine Learning course which focuses much more on theory than the Coursera class but it is still relevant for beginners.(https://work.caltech.edu/telecourse.html)
Neural Networks and Deep Learning (Recommended by Google Brain Team) (http://neuralnetworksanddeeplearning.com/)
Probabilistic Graphical Models (https://www.coursera.org/learn/probabilistic-graphical-model...)
Computational Neuroscience (https://www.coursera.org/learn/computational-neuroscience)
Statistical Machine Learning (http://www.stat.cmu.edu/~larry/=sml/)
From Open AI CEO Greg Brockman on Quora
Deep Learning Book (http://www.deeplearningbook.org/) ( Also Recommended by Google Brain Team )
It contains essentially all the concepts and intuition needed for deep learning engineering (except reinforcement learning). by Greg
2. If you’d like to take courses: Linear Algebra — Stephen Boyd’s EE263 (Stanford) (http://ee263.stanford.edu/) or Linear Algebra (MIT)(http://ocw.mit.edu/courses/mathematics/18-06sc-linear-algebr...)
Neural Networks for Machine Learning — Geoff Hinton (Coursera) https://www.coursera.org/learn/neural-networks
Neural Nets — Andrej Karpathy’s CS231N (Stanford) http://cs231n.stanford.edu/
Advanced Robotics (the MDP / optimal control lectures) — Pieter Abbeel’s CS287 (Berkeley) https://people.eecs.berkeley.edu/~pabbeel/cs287-fa11/
Deep RL — John Schulman’s CS294–112 (Berkeley) http://rll.berkeley.edu/deeprlcourse/
From Director of AI Research at Facebook and Professor at NYU Yann LeCun on Quora
In any case, take Calc I, Calc II, Calc III, Linear Algebra, Probability and Statistics, and as many physics courses as you can. But make sure you learn to program.
There are a few ways to go about this.
One is very programmer heavy. There's a lot of data processing that goes into machine learning, but you can't quite separate the two. Often, the programmer who prepares and processes the data needs to write the code that actually runs the model and parses the result, and this means you benefit from more understanding of machine learning. That role is most likely the best one available to programmers who don't have much mathematics background in this area.
To really get into machine learning itself as a data scientist, though... I do think it requires some math. There's a reason a large percentage of people who work in this field have an MS or PhD in a very quantitative field. And I don't just mean algorithm designers - to really be able to explain the difference between naive bayes, random forest, neural nets, and logistic regression, it helps enormously to have a background in math.
To illustrate this, I've taken two coursera courses on data science. They were both excellent, but approached from different angles. Bill Howe's data science class involved an exercise to use a random forest to do some classification, but the focus was on calling the scikit-learn library. We did of course review the algorithm, but not in mathematical depth.
Andrew Ng's course on machine learning got into implementing the algorithms (with a language called Octave, which honestly I didn't like much, but that's a completely different topic where plenty of people would disagree with me). To do that class, honestly, I'd just ask if the terms "vector calculus", "matrix of second order partial derivatives" or "logistic function" mean something to you. It's ok if you can't define these things on the spot, but was there a time when you could? You can get up to speed, but I'd say if you haven't taken basic calculus through differential equations (with linear algebra), then you won't be able to understand this material.
I've been impressed with how well people learn on their own, picking up a lot of math as they go along. And I don't think you need to be able to implement these algorithms yourself to use them meaningfully. But if you're going to be deciding what kind of model to use, even if you're using libraries to do it (and most people who can implement these algorithms would still use a library), I think that you do need to be able to describe how a neural net work vs random forests vs logistic regression vs naive bayes. There is a side of this that is very math-y as well.
On the bright side, we live in an era of amazingly available learning material. I personally think a dedicated person can probably learn calc, linear algebra, and differential equations through web-based coursework now.
SO overall, I'd say - start on the data side as much as possible, leveraging your programming skills. While doing this, keep getting more exposure to ML algorithms, and make sure you are taking a coursera or other web-based class on the side.
There's lots of books, courses, etc out there, much more than you can possible get around to reading. Lots of people recommend Andrew Ng's ML course, and it's a good introduction to the basic ideas, but it's showing it's age a bit IMO and doesn't prepare you to be a practitioner at all.
Python is pretty much the lingua franca of machine learning, so expect to use that, and I would recommend Keras as a framework for getting started with deep learning; it uses TensorFlow (or Theano) under the hood. scikit-learn it the main non-DL python ML library. You will almost certainly want to use Jupyter as an interactive Python environment.
Things like WEKA/deeplearning4j exist for the JVM, and they may be necessary for work, but are not where I would recommend starting.
If you're not aware of Kaggle, it's an ML competition website which hosts datasets, but also publishes descriptions of winning entries, though you'll find that deep learning is not what wins everything.
I would also suggest looking at some of the academic papers from conferences like ICML/NIPS/ICLR or just uploaded to Arxiv (though figuring out which ones are interesting will be harder to start); many papers are surprisingly approachable and knowing what mathematical topics are mentioned in cutting edge papers can help guide your learning.
I'm currently working through Andrew Ng's ML course on Coursera. It's definitely high-level (though a fantastic overview of the fundamentals), and I plan to take something a little more mathematically rigorous at some point, but I'll probably want to take some refresher courses in Linear Algebra and calc before doing so.
I'm not trying to learn about ML for purposes of employment, It's somewhat relevant to my current job, and I may have some interest in using it on my personal projects. But mostly I'm just learning for the 'fun' of it :)
I don't have the time, money, or inclination to pursue a MS in data science atm (My current 'formal' education consists of a BS in Comp Sci and an MBA), but I may go back to school when the kids are grown, more for personal edification than anything else, however. A big shift in career, from software engineer to 'data scientist (or whatever they call it)' is probably not possible at my advanced age (37).
The fundamental techniques of machine learning will always be there. For that reason, Ng's ML course will always be the prime entry point for people new to ML. As someone who has taken both the Ng course and has roughly watched the videos of the fast ai course and done some work, I would recommend first doing the Ng course. Things will be much more calm if you then approach the fast ai DL course. The fast ai course is amazing and hats off to Jeremy and Rachel for the great work. But without any idea of machine learning, I would say you will need to put at least 20 hours of work each week to work through it, and that is if you are also a really proficient programmer.
This course was discussed here on HN 3 months ago: https://news.ycombinator.com/item?id=13224588
thrawy45678: I see a lot of criticism about tmux and other non-core items being included in the overall curriculum. I think the author is trying to portray the workflow he is currently on and exposing the full tool kit he uses. I don't think he is saying - this is "THE" approach one has to follow.
derekmcloughlin: If you've no experience with ML stuff, you might want to start with Andrew Ng's course [...] Paperspace (https://www.paperspace.com/ml) have ML instances starting at $0.27 per hour.
webmaven: Any recommendations for the cheapest (but not "penny wise, pound foolish") HW setup that meets these requirements for course completion? - (answers: https://news.ycombinator.com/item?id=13227437 )
The course is also quite easy to follow without buying the book. I love the exercises in which you are programming an intelligent agent to move through a maze. It reminded me of how we learned programming in university using Karel The Robot.
This alongside Andrew Ng's Machine Learning course was my first exposure to the field. https://www.coursera.org/learn/machine-learning
I can also recommend Sebastian Thrun's Artificial Ingelligence for Robotics course: https://www.udacity.com/course/artificial-intelligence-for-r...
I've had exactly the same struggle. Andrew Ng was truly awesome, better than any lecturer I had. Jeff Hinton's course is just a real struggle to complete.
It's more than just the lectures, the thought and relevance that goes into the problem sets is a big difference too.
It's great to hear that Jennifer Widom's course is on a similar level. Perhaps I should dust up on my database knowledge.
I've done Udacity's ML and doing the SDC in free time.
Short answer: Yes - I would do it again.
ML (Mid 2016): Cost-wise you may be able to gather the similar quality of materials for free, nothing are too unique and in many cases additional intensive research are still required. But, the Udacity provides nice structure and helps you to keep motivated.
SDC (2016): Much better experience for students, the quality of materials are higher, and amount of support from the peer group are extraordinary. Just one of the five projects in the first term are more complex than the whole ML nanodegree.
Haven't tried the AI nanodegree myself, but because it's a part of the new batch of programs I believe the experience would be quite positive (closer to SDC).
If you will decide to proceed with self-learning path, here is a nice multi-month study plan: https://github.com/ZuzooVn/machine-learning-for-software-eng...
In any case I strongly advice to take a look into Andrew Ng's coursera courses and Andrew Karpathy's CS231n on CNN (http://cs231n.stanford.edu/) as a supportive materials.
Also, if money is the main concern, you may want to apply to sponsorship, or maybe discuss this with your current employer.
Feel free to PM me on twitter (same nick) if you have any other questions, would be glad to assist.
Do this course and you'll get a very good idea of what's happening, and it's free: https://www.coursera.org/learn/machine-learning
If it isn't too introductory, I'd say take the Andrew Ng course on Machine Learning on Coursera. If that's too introductory, try the Geoffrey Hinton course on Neural Networks on Coursera, or the Google / Tensorflow course on Deep Learning on, er, I think it's EdX.
I am taking Machine Learning on coursera by Andrew Ag. Initially I was intimidated by the idea of ML as I had no prior programming experience. I started learning data science stuff less than 6 months ago. I started to feel motivated and confident about ML by taking this course. I highly recommend it.
https://www.coursera.org/learn/machine-learning
I'm currently enrolled in the OMSCS program, and so far I would call it so-so.
I joined the program because I come from a non-CS background - undergrad in math, work in an unrelated field: consulting. I'm trying to pivot into a ML Engineer career. If you want to learn ML, you're better off going through the Ng Coursera course and from there pursuing some personal projects. The primary value of the program is the ability to get past recruiting coordinators simply due to the fact I'm enrolled in CS program.
The two undergrad CS courses I took at Berkeley were more rigorous, and were superior from a skills development perspective. But at the price, the OMSCS program is definitely worth it for someone coming from a different background.
- Take Andrew Ng's ML class on Coursera.
- Install TensorFlow and TFLearn.
- git clone something based on TF from GitHub and hack it.
- Do some pet project from scratch.
- profit
I had taken the Ng course, which I recommend to get a basic understanding of ideas, but, the fast.ai course enables you to compete in Kaggle competitions in fairly short time.
Just watch the course.fast.ai intro/overview video, and decide from there.
Without doubt do the Andrew Ng course on Machine Learning.
https://www.coursera.org/learn/machine-learning
It's excellent.
So far, yes - but that has a few caveats:
See - I have some background prior to this, and I think it biases me a bit. First, I was one of the cohort that took the Stanford-sponsored ML Class (Andrew Ng) and AI Class (Thrun/Norvig), in 2011. While I wasn't able to complete the AI Class (due to personal reasons), I did complete the ML Class.
Both of these courses are now offered by Udacity (AI Class) and Coursera (ML Class):
https://www.udacity.com/course/intro-to-artificial-intellige...
https://www.coursera.org/learn/machine-learning
If you have never done any of this before, I encourage you to look into these courses first. IIRC, they are both free and self-paced online. I honestly found the ML Class to be easier than the AI class when I took them - but that was before the founding of these two MOOC-focused companies, so the content may have changed or been made more understandable since then.
In fact, now that I think about it, I might try taking those courses again myself as a refresher!
After that (and kicking myself for dropping out of the AI Class - but I didn't have a real choice there at the time), in 2012 Udacity started, and because of (reasons...) they couldn't offer the AI Class as a course (while for some reason, Coursera could offer the ML Class - there must have been licensing issues or something) - so instead, they offered their CS373 course in 2012 (at the time, titled "How to Build Your Own Self-Driving Vehicle" or something like that - quite a lofty title):
https://www.udacity.com/course/artificial-intelligence-for-r...
I jumped at it - and completed it as well; I found it to be a great course, and while difficult, it was very enlightening on several fronts (for the first time, it clearly explained to me exactly how a Kalman filter and PID worked!).
So - I have that background, plus everything else I have read before then or since (AI/ML has been a side interest of mine since I was a child - I'm 43 now).
My suggestion if you are just starting would be to take the courses in roughly this order - and only after you are fairly comfortable with both linear algebra concepts (mainly vectors/matrices math - dot product and the like) and stats/probabilities. To a certain extent (and I have found this out with this current Udacity course), having a knowledge of some basic calculus concepts (derivatives mainly) will be of help - but so far, despite that minor handicap, I've been ok without that greater knowledge - but I do intend to learn it:
1. Coursera ML Class 2. Udacity AI Class 3. Udacity CS373 course 4. Udacity Self-Driving Car Engineer Nanodegree
I honestly think it will - but I also have over 25 years under my belt as a professional software developer/engineer. Ultimately, it - along with the other courses I took - will (and have) help me in having other tools and ideas to bring to bear on problems. Also - realize that this knowledge can apply to multiple domains - not just vehicles. Marketing, robotics, design - heck, you name it - all will need or do currently need people who understand machine learning techniques.
I believe you could, depending on your prior background. That said, don't think that these courses could ever substitute for graduate degree in ML - but I do think they could be a great stepping stone. I am actually planning on looking into getting my BA then Masters (hopefully) in Comp Sci after completing this course. Its something I should have done long ago, but better late than never, I guess! All I currently have is an associates from a tech school (worth almost nothing), and my high school diploma - but that, plus my willingness to constantly learn and stay ahead in my skills has never let me down career-wise! So I think having this ML experience will ultimately be a plus.
Worst-case scenario: I can use what I have learned in the development of a homebrew UGV (unmanned ground vehicle) I've been working at on and off for the past few years (mostly "off" - lol).
No problem, I hope my thoughts help - if you have other questions, PM me...
i was quite put off by it. i feel like the teaching technique is pretty poor and the focus in on all the wrong things. mainly the tech gets in the way for learning. i don't want to figure out how to learn numpy when i'm trying to learn how to understand deep learning, that in itself is hard enough. i quite after a week (i did the stanford course first and this was going to be my second).
i would recommend the coursera course by andrew ng. i had an amazing time. the code stays out of your way and he walks you through the algorithms and explains the theory very well.
i just started the fast.ai by jeremy howard, and literally have been blown away but the course. it is AMAZING! by lesson 3 i'm able to build cnn models and score on top 20% in kaggle competitions. not bad for a complete novice. HIGHLY RECOMMENDED.
once im done with the fast.ai course i may look back around to google's deep learning course. i think it may be easier for more experienced users to digest its info.
https://www.coursera.org/learn/machine-learning
http://course.fast.ai/
Edit: added fast.ai link
The main reason we see all these relatively information-less blog posts / tutorials is because most people on HN are afraid of the math. If you're not afraid of the math (or have a relatively strong math background), then honestly you can jump into the CS 229 Stanford lecture notes and homework assignments. If that's a little too heavy for you, you can also start with the MOOC version of that course (which is a very watered down version of CS 229).
The amount of free resources now available for learning machine learning/deep learning nowadays is robust and easy to comprehend. (indeed, Andrew Ng's Coursera class is very good). And running running ML code is even easier, with libraries like Tensorflow/Theano to abstract the ML gruntwork (and Keras to abstract the abstraction!)
I suspect that there may be machine learning knowledge crash, where the basics are repeated endlessly, but there is less unique, real world application of the knowledge learned. I've seen many Internet testimonials saying how "I followed an online tutorial and now I can classify handwritten digits, AI is the future!" The meme that Kaggle competitions are a metric of practical ML skill encourages budding ML enthusiasts to look at minimizing log-loss or maximizing accuracy without considering time/cost tradeoffs, which doesn't reflect real-world constraints.
Unfortunately, many successful real world applications of ML/DL are the ones not being instructed in tutorials as they are trade secrets (this is the case with "big data" literature, to my frustration). OpenAI is a good step toward transparency in the field, but that won't stop the ML-trivializing "this program can play Pong, AI is the future!" thought pieces (https://news.ycombinator.com/item?id=13256962).
If you've no experience with ML stuff, you might want to start with Andrew Ng's course, which has a small bit on neural networks (MLP and backpropagation), with examples in Matlab/Octave. I found it useful, along with "Make Your Own Neural Network" by Tariq Rashid. Very good intro to coding MLPs directly in Python.
As long as you know a lil bit of math and coding, you're good to go. If you're watching the cs231n videos and don't understand something, you can always check out their amazing class notes.
The cs231n course covers a lot of material, including some (or most) of the material from the Andrew Ng's Coursera course. You can totally do them in parallel though.
Try Andrew Ng's ML Coursera class. The programming is in Octave/Matlab which means you've got all the necessary visualizations and math libraries ready to go.
Ng's Machine Learning course is an outlier I think. It's easily the best online course I've ever taken.
The scikit-learn tutorials are great. Another nice thing about scikit-learn is that the api for a lot of different ML algorithms is very similar, almost identical.
This means that you can set up a train and and test set and swap in and out random forest, svm, naive bases, logistic regression, and various others.
Read about them one by one, try to understand the algorithms generally, test them out, see how they perform differently on different data sets.
It all depends on how you like to approach a new subject, but I think this is more fun and motivating than going straight into the mathematics behind the algorithms right away (which is more along the lines Andrew Ng's excellent course). I'd say once you're into it and using the algorithms, then dig deeper into the core mathematics, you'll have a better context for it.
You need to learn the basic concepts before you start with coding. Take your time and view Andrew Ng's course. It is good for the first timers.
Get some background knowledge; I think with a topic like machine learning it's important to understand why certain algorithms work better than others on different kinds of data. I would recommend following a structured course. Andrew Ng's, or the UC berkley one are good. Tom Mitchell's Machine Learning book is a great intro too to supplement the online course of your choice.
If you're a python dev, maybe download scikit-learn and see what kinds of things you can put together after a few lectures.
You can start with free coursera course https://www.coursera.org/learn/machine-learning/, it starts 17 oct
and then continue with https://www.coursera.org/learn/neural-networks/
Andrew Ng's machine learning course on coursera is great for beginners. It'll show you some theory backed by intuition, and if you do the assignments you'll implement ML algorithms yourself (in Matlab).
After that, I recommend Stanford's CS231n. It's focused on computer vision tasks, but you can learn a lot about neural nets in general from it. The videos were recorded around a year ago I think, so it's up to date with the latest deep learning techniques and libraries. For some reason the videos were taken down from the CS231n site but you can watch them here: https://www.youtube.com/watch?v=g-PvXUjD6qg&list=PLlJy-eBtNF...
If you want to do Andrew Ng's ML course, but want to do it in python: https://github.com/icrtiou/coursera-ML
It really depends on how you learn.
Traditionally the best answer is to do Andrew Ng's Machine Learning course[1]. It's a great course, and you won't regret doing it, but it is kind of annoying that it is in a language (Matlab/Octave) you'll (hopefully) never use again.
A lot of people now recommend working through CS229[2]. I haven't looked at it depth, but I've been impressed with a lot of the class projects.
If you like books, then Statistical Learning in R[3] is generally well regarded.
If you like doing stuff, then Kaggle and SciKit-Learn will throw you in the deep-end. Just be aware you can't just program, though - you really do need to understand some theory. It's good to run into a problem, and then really, really understand the reasons behind what you are seeing.
[1] https://www.coursera.org/learn/machine-learning
[2] http://cs229.stanford.edu/
[3] http://www-bcf.usc.edu/~gareth/ISL/
So I've been feverishly researching both topics, and how to learn them. I've decided to take these 2 courses from Coursera (starting soon, in Oct 2016). Both are free, and for the ML course, one can pay after completing it and get a certificate (there's no certificate for the Bitcoin course).
https://www.coursera.org/learn/machine-learning
https://www.coursera.org/learn/cryptocurrency#
Hope you too will join these, and post updates on this thread right here.
Yes, the Coursera ML course is equally hand-holdey. (I'd still recommend it anyway.)
I've been doing the Coursera ML course the past few weeks(started on the 22th of August). I can only give it my highest recommendation, it really is a great course. I did computer science in university(2.5 years) and have a decent grasp of calculus and linear algebra. I've also been programming for 9 or so years. I've found the course to be surprisingly simple and intuitive, but this probably varies with how comfortable you are with the math in the course.
The part about the Coursera course's steep learning curve feels so familiar. I'm doing the course as well, but it's pretty tough for me. I suck at maths and I don't know Octave or Python. It's a very good course though, I still try to watch some lessons every week, but I'm way behind schedule.
I'm impressed by the author's ability to recognize his own failure in completing a course, because for me it often just keeps dragging on while I still think "some day I will finish it". I can't accept the fact that I just failed it. Sometimes I wish I was still in university, being able to dedicate 40 hours per week to just learning this kind of stuff and having teachers around.
I completed Andrew Ng's Machine Learning course on Coursera over this summer and thoroughly recommend it. The course is free unless you pay for the optional certificate.
Andrew's course makes extensive use of GNU Octave which is also free (you can use Matlab), but the material is easily transferable to other environments.
Andrew spends a lot of time explaining how and why the algorithms work, so you are fully prepared when it comes to implementing them for the assignments, rather than just showing you how to call a method or use a wizard.
This course will help you understand the mathematical principles behind the ML techniques that are covered, which I found particularly helpful.
First, one must understand that these things are not just over, but over-over hyped. It is a socially constructed meme that Deep Learning is only for real geniuses, math PhDs, etc, build by Deep Learning guys themselves.
The problem with AI is that one need enormous money and resources of a big corporation to produce results one could see on AI competitions. It their "research" they do progress, like everyone else, by trial and error (well, augmented with a decent heuristic search process) and the more people and hardware resources they could put in it, the more chances they will outperform other teams. It is resources, not "smartness".
I have completed the very first AI MOOC by Andrew Ng years ago - 780 out of 800 or something. The only difficulty is that it combines methods from math and programming, so one need to have some background to really understand hows and whys. Also really decent knowledge of English is required, otherwise one might miss the nuances in a very dense, loaded with terminology lectures, so, the entry level is quite high (of course, one could always copy-paste code without understanding and use ready-made toolkits and tutorials).
Apart from that it is nothing special. Basically, it is a function composition, with linear algebra and some numerical optimizations. When you have understood the basic building blocks - mathematical functions, processes and algorithms involved, there is nothing much else to do - one has to apply the theoretical knowledge which is, again, not a big deal, to real problems, and this is where big corps with resources took the advantage.
As long as you manage to get inside one of bigcorp AI lab, you become a star, simply because of the well funded PR machine of the corp. Everything that comes out is super cool, of course, so even being mentioned in the context makes one super cool too.
In my opinion, the guys from upper middle class families, who went through a decent technical school (with mom and dad's money) which taught them the basis needed for entering AI, are not that special. I probably could beat one or two of such snobs, having no high school education at all, never studied English or programming in a school and being raised in an impoverished family, but this is another story.
As for ML, take Andrew Ng's course, it is pretty accessible, and then take the one on the Udacity (with all these arrogant hot shots) and realize that there is really no magic in it. It is not that hard.
When you see a cool paper or video about some "breakthroughs" it ML, take into account that it is mostly due to resources spent on it, not some kind of extraordinary genius of the authors of the paper. Remember, all they do is basically a heuristic search and constraint satisfaction problems - train, test, change function (layer) composition (usually, without real understanding of whys), re-train, re-test. The problem is that there are very few of such slots in megacorps.
The guys like Andrew Ng himself are, of course, the real stars. But this kind of success comes from years of rigorous training similar to what Olympic champions have undergo. I personally don't think I have such ambitions or wish to embrace such lifestyle.
The people doing this aren't magical geniuses; they've just put the time and work into the subject and have been able to get themselves into a position they can do this all day surrounded by others they can collaborate with.
As with most human endeavors, the trick is to just get started, and not get frustrated and give up when it turns out you don't know anything at first. Some people don't mind starting with a ton of abstract learning about the subject, others prefer trying to accomplish specific tasks, learning the theory along the way.
For your specific question, as with all software, there's likely a lot to be done that has little to do with the main task of the tool and is just the everyday tasks of ease of use, interoperation with other tools, testing, etc. If on the other hand you want to get into the scary world of AI, like many others I'd recommend the Coursera machine learning course as a great place to start[1]
[1] https://www.coursera.org/learn/machine-learning
Check out the stanfard MOOC class taught by Andrew Ng... that's where most people (I know of) started. https://www.google.com/search?client=safari&rls=en&q=Andrew+...
Web developer to ML? Doable but you may have to work harder than some other folks have a background in statistics, probability, signal processing etc. Plus, you don't have to be an algorithms developer... doing the backend, compute cluster is hugely valuable, too.
Do what feels good for you :)
Ng is a fine place to start, you get some pretty quick wins, doing MNIST from first principles within a month or two. You just need to know or get comfortable with matrix multiplication. It strikes a reasonable balance between being rigorous and approachable for a committed student at an undergrad level.
Principles of Statistical Learning is easier https://lagunita.stanford.edu/courses/HumanitiesandScience/S...
LAFF linear algebra is just starting http://www.ulaff.net/
Hinton's Neural Networks is offered in the fall https://www.coursera.org/learn/neural-networks
For my money, I wouldn't do something like Practical Machine Learning in R, because I think you'll learn more R than machine learning. I wouldn't do the Udacity TensorFlow course because I think it assumes a lot of stuff you would learn in Ng's class ... I think Ng is a fine place to start.
I liked this class and Andrew seems like a great guy but I'd like to point out that it's no longer free to take the evaluations for Coursera courses. Coursera is a startup and needs to find a way to monetize the courses to stay in business. I have no problem with that but I think its a little disingenuous to present Coursera as a free service when its clearly not.
these are things I might consider if I was in your situation: 1) build something that you think will force you to learn whatever data structures and algorithms you want to learn. Two benefits; learning and having something to show for it 2) ask someone to do mock interviews with you to practice 3) take the coursera ML course - fascinating, useful, marketable skill and hard to do unless you have a couple of months free to dedicate 4) network. Coffee with people IRL regularly
it's very good, but personally I'd start with the Andrew Ng and Hinton courses
https://www.coursera.org/learn/machine-learning https://www.coursera.org/learn/neural-networks
I think the Udacity course is best if you know principles of machine learning and want to apply them in a more professional toolchain and learn Tensorflow
I disagree re: ESL and RMS, but Harrell's book is superb. It's really more aimed at biostatisticians and clinicians, though.
If the math in ESL gives you trouble, you might prefer http://www-bcf.usc.edu/~gareth/ISL/ISLR%20First%20Printing.p... (and I'm not just saying this because the first author was one of my advisers, although I do think that he and Daniela are particularly gifted teachers).
If the math in ESL is too trivial for you, there's always https://web.stanford.edu/~hastie/StatLearnSparsity_files/SLS... , which covers some graphical modeling strategies in later chapters and even kicks the tires of the autoencoder (imho perhaps the greatest recent advance in neural networks for practitioners) along the way.
Koller's course and Ng's course are also good.
Ultimately I feel like you have to get the math right or you'll never acquire the intuition that helps you design your own approaches. But you also have to put in the work.
That reminds me, tibshirani's Stanford course (accompanies ISL and ESL) is terrific. Better than those other two, actually. I wish Harrell would offer one.
If you ask me about ML I'll say that starting with convolutional nets for images is starting on the wrong side (end instead of beginning). Andrew Ng course is better, but still somewhat lacks theory and general ML insights.
I'd recommend new Coursera Washington University ML course (https://class.coursera.org/machlearning-001/lecture) as a solid foundation. You'll learn key ML concepts (bias vs variance, bayes theory etc.) and the lector is simply the best I've seen.
Following that Ian Goodfellow book http://www.deeplearningbook.org/ is very good.
If you want to learn foundations of Reinforcement Learning I'd recommend Sutton & Barto https://webdocs.cs.ualberta.ca/~sutton/book/ebook/the-book.h...
You should start with an intro calculus class (e.g. Calculus I). Andrew Ng's Coursera course teaches you the necessary linear algebra. After his Coursera course it'll be worthwhile to take a linear algebra class.
I took the AI for robots on Udacity, the Machine Learning at Coursera and the Control of mobile robots on Coursera. They are all very good in my opinion. Of those, I would start with the Machine Learning course. Not only is it an excellent course and has material found in the others, but if you are at all rusty on your Linear Algebra it will help with that, in addition to the programming of course all of these course make heavy use of linear algebra. The coursera course both use Matlab, the Udacity course uses Python.
Of the robotics course I would say the one on Courera is better in terms of the basics of a robot, like odometry, PID control, etc. The one on Udacity I found more informative in terms of higher level concepts like route planning and localization.
If you are interested in learning the basics of Machine Learning I really recommend Andrew Ng's course on Coursera[0]. It starts off very basic and requires almost no prior knowledge other than some linear algebra. It evolves to more advance topics like Neural Networks, Support Vector Machines, Recommender Systems, etc. It is amazing how much you can do after just a few weeks of lessons. It's a very practical course with real world examples and exercises each week. The focus is mostly on using and implementing machine learning algorithms and less so on how and why the math behind them works. (To me this is a negative, but I'm sure some people will appreciate that.)
[0] https://www.coursera.org/learn/machine-learning
It is truly sad to see Coursera getting greedier by the day. I can honestly say this website changed my life, I was living in a third world country and still in high school when I enrolled in Andrew Ng's machine learning class and thanks to that MOOC I was able to get a machine learning job building recommender systems for a Canadian company straight out of high school. There are plenty of amazing MOOCs that Coursera has completely removed from the website or are only available for people who want to pay upfront. Please don't be like Udacity Coursera.
BTW you have until June 30 to download your courses.
It sounds like you've got the beginnings of Machine Learning (classification & regression), but I'd still recommend checking out Coursera. The JHU Data Science series is good, but more geared towards DS than AI. There are others on there as well; if you're looking to get into deep learning, I echo the recommendation to check out Ng's course, though this might be good, too: https://www.coursera.org/course/neuralnets
After taking Andrew Ngs course I highly recommend you take cs231n afterwards.
Assuming you know a little bit of calculus you'll be able to create and train fully connected networks, convolution networks, recurrent networks, and long short term memory networks using only numpy. You will feel very comfortable with deep learning after taking this class.
http://cs231n.stanford.edu/syllabus.html
The lecture videos for the course got taken down due to complaints, but you can recover them via google
I'm in the same boat - so far it seems like the plan would be 1) Get more familiar with Linear Algebra; 2) Take Ng's coursera course "Intro to Machine Learning" and 3) start trying some kaggle.com challenges.
I believe that the best way to learn ML is by first learning to program the algorithms and then learning the math. This is the opposite to what people is used to do, but I think it's better. The reason is because programming ML is easy, but the math behind it is very complex. I would suggest to start with scikit tutorial http://scikit-learn.org/stable/tutorial/ and later with Ng course https://www.coursera.org/learn/machine-learning. Then a good book is Pattern Recognition and Machine Learning, from Bishop.
For a good conceptual outline you should definitely check out neuralnetworksanddeeplearning.com by Michael Nielsen. Then if you want to go deeper, look for Machine Learning course on Coursera by Pedro Domingos.
I also just finished the Coursera ML course, now I'm doing some stats and data science classes on there, and the Google deep learning class at Udacity.
I kinda think that the rise of MOOCs and "on demand education" is the best thing since sliced bread. :-)
The lecture that covers gradient descent in the Youtube list you linked there is the first time gradient descent actually clicked for me, and I made it through the entire Andrew Ng Coursera ML course. Highly highly recommend it.
It's good to see people write up their experiments, it's useful for the rest of us to test how we understand neural nets.
I think there are a few mistake in your maths though. You can learn a 1-1 discrete mapping through a single node where you are using a one-hot vector. You just assign a weight to each of the input nodes, and then use a delta function on the other side. If I understood correctly, this is what you are doing.
Also, if you use a tanh in your input layer, but keep a linear output layer (as you start off with), you are still doing a linear approximation because you have a rank H (where H is the hidden layer) matrix that is trying to linearly approximate your input data. This is done optimally using PCA.
I'd second the advice to look into the coursera courses, or the nando de freitas oxford course on youtube (that actually has a really nice derivation of backprop).
The possibility of deep NLP suggest that there are tremendous opportunities for building a personal assistant. For Deep NLP, I would suggest course CS224D at Stanford on YouTube (going on now but also delivered last year: https://www.youtube.com/watch?v=kZteabVD8sU
For Deep NLP, you will need to be solid on linear algebra and machine learning. For introduction to machine learning, check out Andrew Ng at Coursera: https://www.coursera.org/learn/machine-learning and my favorite talks on Linear Algebra are the ones done by Gilbert Strang: http://ocw.mit.edu/courses/mathematics/18-06-linear-algebra-...
For web crawling, there are plenty of open source libraries. If you are not familiar with it, check out the Common Crawl: http://commoncrawl.org/ This is a great source of data to crawl.
If you focus solely on NLP and data from the Common Crawl (or even Wikipedia), then you will see where you stand as the smoke clears and you feel comfortable with the state of the art techniques.
Ignore all the naysayers. The good news is that it has never been easier to get started in deep NLP. Once you have the experience of training a model and seeing how well it works, you can decide on the next steps. Perhaps, you will find a niche that is not well-served that you can go after as a first step of a personal assistant.
Good luck.
Andrew Ng's Coursera course on Machine Learning:
https://www.coursera.org/learn/machine-learning
There's also this course on Neural Networks by Geoffrey Hinton:
https://www.coursera.org/course/neuralnets
If you do the Andrew Ng course, you'll have to learn/know Octave (or Matlab). Otherwise, Python is very heavily used in ML these days and you could do a lot worse than learning Python. R and Octave are both very useful as well. And, as you note, there is a lot of machine learning software available in the JVM world as well. Scala in particular seems to be gaining some ground in this world.
I'm just finishing up Andrew Ng's Machine Learning course and would definitely say it's worth it (depending on how much, if anything, you already know about ML).
I'm also about halfway through the Johns Hopkins Data Science specialization, and I've found all of those courses to be worthwhile. But then again, I'd never programmed in R before and that's something I very pointedly wanted to pick up. So far, this series has been invaluable in terms of learning R (much like I had to learn a fair amount of Octave for Andrew Ng's class).
I enjoyed Andrew Ng's Machine Learning course on Coursera. Why don't you give it a shot.
Sure, a couple things.
(I'm assuming you're comfortable with multivariable calculus.)
Andrew Ng's coursera course is good.
PRML (pattern recognition and machine learning) by bishop is good, and has a useful introduction to probability theory.
You also want a good grounding in linear algebra. Strang is basically the authority on linear: http://ocw.mit.edu/courses/mathematics/18-06-linear-algebra-...
You want a strong grounding in probability theory and statistics. (This is the basic language and intuition of the entire field.) I don't have as many preferences here (although its the most important); someone in this thread pointed to a course on statistical learning @ stanford that's good.
A good understanding of optimization is helpful. Here's a link that leads to a useful MOOC for that: http://stanford.edu/~boyd/cvxbook/
there's a lot of other stuff (markov decision processes, gaussian processes, monte carlo methods come to mind) that is useful that I'm not pointing to, but if you've hit the other stuff here then you'll probably be able to find out those things.
If you're into it, https://www.coursera.org/course/pgm is good but not vital.
You may want to know about reinforcement learning. This answer does better than I can: https://www.quora.com/What-are-the-best-books-about-reinforc...
Deep learning seems popular these days :) (http://www.deeplearningbook.org/)
Otherwise, it depends on the domain.
For NLP, there's a great stanford course on deep learning + NLP (http://cs224d.stanford.edu/syllabus.html), but there's a ton of domain knowledge for most NLP work (and a lot of it really centers around data preparation).
For speech, theoretical computer science matters (weighted finite state transducers, formal languages, etc.)
For vision, again, stanford: (http://cs231n.stanford.edu/syllabus.html)
For other applications, well, ask someone else? :)
Also:
arxiv.org/list/cs.CL/recent arxiv.org/list/cs.NE/recent arxiv.org/list/cs.LG/recent arxiv.org/list/cs.AI/recent
EDIT: unfortunately, there's also a lot of practitioner's dark art; I picked a lot up as a research assistant, and then my first year in industry felt like being strapped to a rocket.
If you haven't already, you should check out the first and second week of Andrew Ng's Coursera Course on Machine learning. He exclusively talks about gradient descent the first few weeks. https://www.coursera.org/learn/machine-learning
(Disclaimer: I have only seen part of the material, for both courses) This seems true in terms of difficulty, mostly because CS229 assumes a much stronger math background and provides more of the theory for a lot of the ML techniques being used. However, if you want to know the "technique space" of usual ML algorithms and when/how to apply them, the Coursera course seems to be 70-90% of the way there in terms of breadth of the material (with a bias towards the most commonly used methods, so perhaps 70% of the techniques, applicable to more than 90% of the use cases).
So, if you want to do ML in industry, applied to common problems, the Coursera course will get you there. If you want to become an ML researcher, the Coursera course will fall way short of that, while CS229 might be at least a first step (followed by CS229T/STATS231 [1]).
[1] Lecture notes online, too: https://web.stanford.edu/class/cs229t/
Assuming that you consider Machine Learning to be either a subset of AI (as I do) or a sibling field, and want to learn aspects of ML, then consider Andrew Ng's Machine Learning course on Coursera. It's a great introduction and doesn't require a ton in terms of prerequisites. You'll see some multi-variable calculus and linear algebra, but he does the calculus derivations for you, and there's a pretty adequate review of the relevant parts of Linear Algebra.
In addition, if you don't already have a background in Calculus and Linear Algebra, then supplement the Ng course with the Khan Academy stuff on Calculus and Linear Algebra, or other courses you can pick up on Coursera or Edx or whatever.
If you get really interested in neural networks (which are all the rage these days) after the Ng class, there's a freely available book on Neural Network design that you could look at. It doesn't cover all the very latest techniques, but it would help you build the foundation of understanding.
http://hagan.okstate.edu/nnd.html
There's also a MOOC around the Learning From Data book that you could check out.
http://amlbook.com/
https://work.caltech.edu/telecourse.html
OTOH, if you're making a sharp distinction between "Classic AI" and "Machine Learning" and you really care mainly about the classical stuff, then you might want to start with the Berkeley CS188 class. You can take it through EdX (https://www.edx.org/course/artificial-intelligence-uc-berkel...) or just watch the videos and download the notes and stuff from http://ai.berkeley.edu/home.html
And if you just want to dive into reading some classic papers and stuff, check out:
http://publications.csail.mit.edu/ai/pubs_browse.shtml
and/or
http://ijcai.org/past_proceedings
Another good resource is
http://aitopics.org/
Definitely linear algebra, but also some calculus and some probability theory / statistics. By way of illustration, if you were to go through the Andrew Ng course on Machine Learning, you'll encounter mathematical explanations involving partial derivatives from differential calculus
As it is, he gives you the derivations you need, so you can complete the class without needing to find partial derivatives. But the point is, a certain level of understanding of calculus will help with understanding exactly what is happening.
As for probability / stats, you'll find more than a few uses of probability notation and basic ideas from statistics scattered throughout machine learning.
No. Honestly, no. Do it because you think it's interesting. Very few companies do deep learning (Google, Facebook, and Microsoft come to mind - it might be useful if you work at these companies already). That number will undoubtedly grow, but the vast majority of ML/data science positions deal with stuff like linear regression, PCA, logistic regression, decision trees, random forests, maybe svms/boosting if you want to get fancy. Take an ML survey course like Andrew Ng's course to learn about these. Also basic statistics/probability.
Haven taken this course (though I will), but check out Andrew Ng's Coursera's ML course. He gives a crash course in linear algebra. It helped me a lot in my endeavours in ML & neural nets. I also like to learn as I go along, and this intro was just about enough to get started.
Two courses I've found are very good for Neural Networks & Deep Learning are Karpathy's CS231n from Stanford[0] and Nando de Freitas's Deep Learning class from Oxford[1][2].
Been through parts of both and they seem really good. That said, I'd bet that both of these classes would be a lot more meaningful after either Hastie's class (listed here) or Andrew Ng's course.
[0] http://cs231n.github.io/
[1] https://www.youtube.com/playlist?list=PLE6Wd9FR--EfW8dtjAuPo...
[2] http://www.cs.ox.ac.uk/teaching/courses/2014-2015/ml/
A couple of friends of mine and I have agreed to go through the first few lectures of this still super-relevant and foundational machine learning class:
http://openclassroom.stanford.edu/MainFolder/CoursePage.php?...
Why not join us with Swift implementations of some of the discussed algorithms? Simultaneously, you could also experiment with some of the cool, puristic functional programming libraries recently coming up for Swift. Check out awesome-swift:
https://github.com/matteocrippa/awesome-swift
In any case server side application of machine-learning has its drawbacks in terms of privacy, reality of losing internet connectivity, etc.
Hit me up on twitter or github if you are interested.
Have fun and happy holidays (don't forget to relax) :)
Read AIMA, Know your maths, discrete maths, probability & stats. Take some machine learning courses (free ones at coursera, edx, etc), read up on Deep learning, you should be good to go.
Find a problem to work on in a domain you find interesting. By reading published papers and trying to attack the problem, you'll be forced to pick up a lot of other knowledge not commonly discussed like: feature extraction and selection, dimensionality reduction, dealing with sparsity, common metrics for that problem, recent work, etc.
I was forced to learn a massive amount in a short period of time for work, but I'd previously watched Andrew Ng's lectures, as well as majored in Math/CS. I can also generally recommend Hinton's NN lectures, Socher's Deep learning for NLP, Andrew Ng's Machine Learning, and a few books.
TIME SENSITIVE note --
The guide's primary recommended course is Andrew Ng's Machine Learning course. Current session started November 2nd, you must enroll by the 7th. Another session is starting November 30th.
https://www.coursera.org/learn/machine-learning
Cheers!
A lot of people are suggesting some bad things.
Some people might take issue with this, but as far as resources/classes/research groups in academia/textbooks go, AI != machine learning. And neural networks are a subset of machine learning.
The AIMA book is the best introduction to AI, but only to traditional AI, which consists mostly of planning/search/inference algorithms (brute force algorithms, albeit clever brute force algorithms). It is not a book on machine learning, even if it talks a bit about machine learning.
The Deep Learning book that people mention is not an introductory book on the subject of neural networks or machine learning.
Your best bet is Andrew Ng's Coursera course as an introduction to ML and neural nets.
I have recently wrote an article collecting the best AI resources:
https://medium.com/@rayalez/best-deep-learning-resources-76b...
Specifically, I would reccommend AIMA as the best introduction to AI in general, and a fantastic video course from Berkeley:
https://www.youtube.com/channel/UCshmLD2MsyqAKBx8ctivb5Q/vid...
and also Andrew Ng's course on coursera:
https://www.coursera.org/course/ml
For neural networks there's an awesome course by Hinton:
https://class.coursera.org/neuralnets-2012-001/lecture
and UFLDL tutorial:
http://deeplearning.stanford.edu/wiki/index.php/UFLDL_Tutori...
Thanks for this. Incidentally, the second paper you link to is co-auhored, among others, by Daphne Koller, who teaches [this great course on probabilistic graphical models](https://www.coursera.org/course/pgm) and Andrew Ng, who teaches [the best-known intro MOOC in machine learning](https://www.coursera.org/learn/machine-learning/home/welcome).
I am learning the course of Machine Learning [1] at Coursera. I didn't know he co-founded Coursera. Can't believe this awesome course is free. Andrew Ng is really a good teacher. Thank you Andrew Ng and Coursera.
[1] https://www.coursera.org/learn/machine-learning/
I'm taking his machine learning course [1] and it's absolutely fantastic. It's one of the mondern wonders of the internet that you can have such a tutor for free. Plus it's fascinating to see what people go on to do after the course [2].
[1]: https://www.coursera.org/learn/machine-learning/
[2]: https://syrah.co/joshdickson40/5604e5e10fc1786b0152a51a
For those that don't know-- although Octave is interpreted, the interpreted part acts as an interface to highly optimized vector operations under the covers. Signal processing and a lot of other tasks come down to manipulating large arrays of data using similar operations across the entire array. Although Octave is interpreted, these kinds of operations execute very nicely inside Octave. Octave also lets you write those kinds of operations at a very high level, like a = b * c, and under the covers it's doing millions of operations.
If you'd like to give Octave a try and learn about Machine Learning at the same time, try this excellent and fun free course! https://www.coursera.org/learn/machine-learning
One of the later lessons in that class is a mind-blowing signal processing thing. In that exercise, they show how you can put multiple mics in a room, with multiple people talking over each other, and the computer separates out the different sources of sounds, and outputs a clean wav file for each sound source with the sounds isolated from each other. It blew my mind, and was implemented in Octave. And they teach you exactly how to do it yourself.
All the above concepts can, of course, be done in C++ or NumPy or whatever, they just use Octave to teach the course.
Andrew Ng, the man behind the excellent ML course on coursera, has an introduction to Deep Learning using Matlab.
[1]: Wiki with code, exercises and explanation
[2]: Video lecture one with a recap on back-propagation
[3]: Video lecture two on Sparse Auto Encoders
[4]: Handouts
In terms of books, Bayesian Reasoning and Machine Learning [5] is Matlab based. So is the Handbook of Monte Carlo methods [6].
[1]: http://ufldl.stanford.edu/wiki/index.php/UFLDL_Tutorial
[2]: http://www.stanford.edu/class/cs294a/video1.html
[3]: http://www.stanford.edu/class/cs294a/video2.html
[4]: http://www.stanford.edu/class/cs294a/handouts.html
[5]: http://web4.cs.ucl.ac.uk/staff/D.Barber/pmwiki/pmwiki.php?n=...
[6]: http://www.maths.uq.edu.au/~kroese/montecarlohandbook/
This course is great: https://www.coursera.org/learn/machine-learning It's all done in GNU Octave, which is mostly compatible with MATLAB.
I agree. Andrew Ng's course is a great place to start with ML. We've got a list of great online ML/DL courses here: http://deeplearning4j.org/deeplearningpapers.html
If this feels like jumping in the deep end, I highly recommend Stanford's online course on this subject:
http://coursera.org/learn/machine-learning/
I recommend this course for a general overview on machine learning: https://www.coursera.org/learn/machine-learning
There are also some good resources in edX...
I really have to take exception to this (characterizing MOOCs as some sort of useful-only-for-signalling high-tech "vocational training" for the privileged).
I've been working through Andrew Ng's Machine learning class (https://www.coursera.org/learn/machine-learning/home/welcome) on Coursera, and I think it's wonderful. It begins a bit slow-paced, and I definitely supplement the lectures with readings and the class notes for CS 229 (http://cs229.stanford.edu/materials.html) that broadly match the content of the online course, but I absolutely love the course.
The lectures are well-paced and have 'in-line' quizzes to test your understanding as you go. The problem sets are easy to do at any time, with optional components and instructions that let you understand the material better if you want to deep dive. The course itself is available to start whenever you choose, which means barrier for entry is very low.
MOOCs may not be everything that they first promised, but they definitely win in terms of the sheer accessibility of content and the flexibility of the format. I would not be learning machine learning right now if the only available format was dead-tree or (worse) putting together information piecemeal from blog posts.
This is fantastic! I've been meaning to make an RNN to generate EDM.
For begineers, I'd like to summarize some excellent resources to start:
- Coursera course by Andrew Ng (he explains everything magically. The best course I've done online) https://class.coursera.org/ml-003/lecture
- Neural Networks and Deep Learning neuralnetworksanddeeplearning.com/ (I highly recommend trying to write your own backprop and MNIST dataset classifier. I wrote in JS and gave me a lot of confidence)
- Oxford ML class (2015) https://www.cs.ox.ac.uk/people/nando.defreitas/machinelearni... (perhaps the the most recent MOOC on ML online. Things are progressing so fast in deep learning that it's worthwhile to do multiple ML courses to get different perspectives)
- I also enjoyed http://karpathy.github.io/2015/05/21/rnn-effectiveness/ for inspiration and the video https://www.youtube.com/watch?v=xKt21ucdBY0 is a FANTASTIC summary. HN earlier pointed our the course videos are out http://cs224d.stanford.edu/syllabus.html
- Again for inspiration into big challenges, this talk from Andrew Ng is a good one https://www.youtube.com/watch?v=W15K9PegQt0
Machine learning is a pretty big field. The Coursera course is very good. It uses Octave not Python, but what you learn will be easy to transfer. It is mostly focused on neural networks. If you don't already know linear algebra you should probably learn that first.
These are three very good O'Reilly books that all use Python:
- Programming Collective Intelligence: A broad and shallow survey of automated machine learning techniques.
- Data Analysis with Open Source Tools: Also a survey. More focused on manual data exploration.
- Python for Data Analysis: A pandas tutorial (and more). Very helpful to learn the ML tools in the python ecosystem.
Fitting all that into two months sounds challenging.
There is a nice book by my profs (Y. Bengio, A. Courville) and a former student of the lab (I. Goodfellow) here: http://www.iro.umontreal.ca/~bengioy/DLbook/ The chapter on RNNs is pretty enlightening. You probably want to start with something the goes through feedforward networks, convolutional, etc. first though.
Andrew Ng's Coursera course does a super basic NN in MATLAB which might be good to shake the rust off [1]. Hugo Larochelle's youtube course [2], Geoff Hinton's Coursera course [3], and Nando's youtube Deep Learning course [4] are all very good. There are also some great lectures from Aaron Courville from the Representation Learning class that just finished here at UdeM [5]
If you want to do this for real (not self teaching RNN backprop on toy examples) tool-wise you generally go with either Theano (my preference) or Torch - implementing the backwards path for these (and verifying gradients are correct) is not pretty.
Theano does it almost for free, and Torch can do it at the module level. Either way you definitely want one of the higher order tools!
[1] https://www.coursera.org/learn/machine-learning
[2] https://www.youtube.com/watch?v=SGZ6BttHMPw&list=PL6Xpj9I5qX...
[3] https://www.coursera.org/course/neuralnets
[4] https://www.youtube.com/watch?v=PlhFWT7vAEw
[5] https://ift6266h15.wordpress.com/
Thanks! I'm really enjoying seeing the huge variety of domains people have applied char-rnn to. I haven't anticipated response at this scale, to both the original blog post and the code release. (I try to keep a somewhat up to date list of responses on the bottom of the blog - cooking recipes, music, obama speeches, magic cards...)
In terms of learning, I would also encourage people to try the materials from our CS231n class - this is a class I taught at Stanford last quarter with my adviser. It's technically about Convolutional Networks, but most of the class material is building up generic Neural Networks, backprop, and so on. You can also try our IPython Notebook assignments.
Course notes: http://cs231n.github.io/ Syllabus with slides too: http://cs231n.stanford.edu/syllabus.html
Another good pointer is Andrew Ng's Coursera class - that's a thorough introduction as well.
Pedro Domingos also has a fantastic mooc at https://www.coursera.org/course/machlearning
I recommend starting with Andrew Ng's ML course on Coursera. He teaches and presents the subject in a way that is both fun and non-intimidating to a beginner. You get a good overview of techniques, and get to try out fun exercises like digit recognition.
I think you're in a position to learn a lot in a week. My advice would be to create a simple data set and go through what you learned in Andrew Ng's course with scikit-learn (even better, since you already prefer Python).
http://scikit-learn.org
Much of what you learn in Ng's course is how to implement these algorithms - there's less (no?) emphasis on using existing libraries in R or Python. I think that implementing your own code base for logistic regression, neural net, random forest, and so forth is an extremely valuable exercise, but I'd recommend you put that aside just for the moment. Instead, try using some of the existing libraries.
For instance, use scikit-learn's logistic regression, neural net, and random forest (not covered in Ng's class) libraries to do a classification. You don't want to use these with no understanding of how they work, but you've done the coursera, so see if you can use your knowledge of how these algorithms differ to create dataset that will highlight the benefits each approach (i.e.., can you create a dataset that works great for logistic regression but poorly for neural nets, or random forest?) Think about how you'd use an unsupervised approach to classification, and run it through k-means. I really think that applying different techniques to the same dataset, on a high level, combined with general knowledge of what's going on under the hood, can be a great way to understand how/when to use these algorithms.
I would recommend going through the CalTech Course - Learning from Data run by Yasir Abu Mustafa as the first step. https://www.edx.org/course/learning-data-caltechx-cs1156x#.V...
It is one of the best places to start. Please see that the course will require you to spend considerable time. If you find this challenging, you can also look at the Machine Learning class by Andrew Ng on Coursera.
Once you have undergone these courses, you can take up Coursera course on Neural Nets or look at tutorials on deeplearning.net
I think these courses are awesome as well:
6006 Introduction To Algorithms from MIT
http://ocw.mit.edu/courses/electrical-engineering-and-comput...
Machine Learning from Stanford: Learn about the most effective machine learning techniques, and gain practice implementing them and getting them to work for yourself. https://www.coursera.org/course/ml
My guess is that you won't find any course that explains all the prerequisite math. It's probably more useful to build a solid foundation in probability theory (and therefore calculus) before going on.
For machine learning, a good place to start is Andrew Ng's course on Coursera:
https://www.coursera.org/course/ml
It's pretty light on math, while at the same time giving you experience in implementing and understanding these techniques.
From there, I might recommend Learning from Data and the associated video lectures:
https://work.caltech.edu/telecourse.html
It is a bit of a jump, but it is a great course in presenting the field of machine learning and explaining the mathematical and statistical underpinnings in a systematic way.
Bookwise, Yoshua Bengio, Aaron Courville, and Ian Goodfellow are nearly finished with their MIT Press book on deep learning: http://www.iro.umontreal.ca/~bengioy/dlbook/ . It is pretty strong on the true theory of what is going on in deep networks, and has fairly good intuition for how and why things work. Paired with the deep learning tutorials http://www.deeplearning.net/tutorial/, as well as the content from UFLDL it is a pretty strong foundation for advanced study.
Michael's book seems to target a more introductory level - a beginner might be better off to start with that, follow with Andrew Ng's ML course, which has a section on neural nets including an assignment implementing backpropagation, then continue with the deep learning book and the {deep learning, UFLDL} tutorials. This should be solid enough to at least read most of the cutting edge work and papers, if that is the aim.
Hugo Larochelle's youtube course https://www.youtube.com/playlist?list=PL6Xpj9I5qXYEcOhn7Tqgh... and Hinton's coursera course https://www.coursera.org/course/neuralnets are also great references.
In red is your model whereas in green is the real one, M being the number of parameters.
The technical term for the last one is "overfitting" if I remember correctly. But in the case you have an enormous amount of data, it is unlikely to happen.
It reminds me of this awesome course: https://www.coursera.org/course/ml
edit: The parent's parent's parent mention overfit for the MIT work, I don't think it'd be the case if you have that amount of data in hands
The videos are available here: https://class.coursera.org/ml-005/lecture
I think it is worth mentioning that "beginners" should be understood as beginners in machine learning and not beginners in computer science. Ng's course more or less assumes students have the equivalent of an upper division undergraduate or graduate background in computer science.
While it may be a valuable learning experience for a person who isn't comfortable knocking out algorithms, academic success will be rather difficult to achieve.
Do this Coursera course, targeted towards beginners and its taught by Andrew Ng (Co-founder of Coursera) - https://www.coursera.org/course/ml
There is a nice tutorial here [1] about max-ent classifiers. Ultimately in neural networks there are usually a number of cost functions you can use for the last layer - softmax or cross-entropy are other possibilities that may be easier to understand, though possibly less performant for NLP tasks.
I thought the introduction to neural networks from Andrew Ng's coursera course (even though it meant writing MATLAB) was quite good, and allows you to implement backprop, cost functions, etc. while still having some other helper code to make things easier. I highly recommend working through that course if you are intersted in ML in general [2].
[1] http://www.cs.berkeley.edu/~klein/papers/maxent-tutorial-sli...
[2] https://www.coursera.org/course/ml
My favorite course so far has been MITx - 15.071x The Analytics Edge on edX. Everything I've seen from MITx has been above average. Andrew Ng's Machine Learning course on Coursera is quite nice as well.
Its because textbooks are usually more in depth than lectures can go due to time considerations, this is why any graduate program is 90% papers and textbooks.
A great example of this is Andrew Ng's course, even though he is the co-inventor of LDA (complicated Bayes network) he does not explain Bayesian analysis in his course.
I did Stanford's online ML course (taught by Andrew Ng, quoted in the article), and heartily recommend it. I found it to be a nice balance of theory and practice - it's also not a bad way to learn Octave (the GNU equivalent of matlab) if one wants to do that. http://online.stanford.edu/course/machine-learning
(I don't know if there's a way to just access the materials and videos though, rather than signing up for the course.)
Great introduction to ML algorithms. I would also like to recommend the Coursera course taught by Andrew Ng for a practical starting up in this powerful field.
There is also a great course running on coursera right now covering this:
https://class.coursera.org/ml-006
I followed Andrew Ng's Machine Learning course on Coursera. The course itself was great, but halfway through I realized he had to be from Malaysia or Singapore, because he started talking about "alphabets" instead of "letters".
If you're good with math and interested in machine learning, I'd recommend checking out Theano's Deep Learning Tutorials: http://deeplearning.net/tutorial/
Theano is a python library that basically compiles graphs of mathematical functions into highly-optimized C code. For computationally-intensive AI applications (ie, all the fun stuff), there's nothing that I've found better than Theano.
If you work through the tutorials (I re-wrote each example from scratch using them as a guide), you'll get a pretty good feel for how to use it.
I would strongly recommend Andrew Ng's coursera course on ML as a starting point, though.
Hello,
I think it's great that you are motivated to study such topics. First, I think it would be useful for you to define what you actually want to learn. There is a difference in what has traditionally been called "AI" and what is called "Machine Learning" today. While these two fields are related and some people say that ML is a subfield of AI, both tend to focus on very different problems. Today, "Machine Learning" has taken a lot of the attention away from traditional AI (mainly due to the lack of results in making truly intelligent machines)
Here are some topics I think of when hearing AI vs. Machine Learning:
AI
- Robotics
- Intelligent Machines, e.g. for question answering
- Natural Language understanding (Not NLP, I mean understanding)
- Game playing/planning
Machine Learning
- Making predictions (Often synonymous with "big data analytics" these days)
- Recommendation Systems
- Finding Patterns (Data Mining)
- "Data Science" techniques
In Machine Learning, Python is the de-facto standard language both in Academia and Industry, mainly because because of its excellent libraries. In terms of resources, I can also highly recommend the Coursera ML class, as well as statistics classes to get started. From there you can dive deeper into any of the topics you are interested in.
I think Python has become a de-facto standard language for machine learning (not necessarily academic AI research). The book Collective Intelligence by O'Reilly is a great overview of ML techniques, and it uses Python. You might also want to take the Coursera ML class. If you sign up today, you might not even miss the deadline for Week 1 coursework. (And if you do it's not that bad.) That uses Octave, not Python, but you'd still learn a lot. You might also meet like-minded people in the course forums.
I doubt the CPU costs will be measurably different.
I suggest watching: https://class.coursera.org/ml-003/lecture/36 and you can jump to 7 minutes if you don't want to watch the whole thing.
The only difference is instead of merely doing the matrix multiplication of your observation times your coefficients (the scalar product of θ and x) you have to take the output and use it as the input to the logistic function. The matrix multiplication will grow in complexity with the number of non-zero features (either non-zero coefficients in your model or non-zero features in your observation, presumably the latter) and the evaluation of the sigmoid function will be constant time, and on many architectures will be a very small number of instructions.
If you're operating in very small numbers of dimensions what you're doing may still be interesting but I'm not sure it qualifies as "Big Data," regardless of the number of available observations. After all, if you're producing something with actual utility, you'll probably converge sooner rather than later...
There's a great Machine Learning course up on Coursera from Stanford: https://www.coursera.org/course/ml
Not a journal or blog, but I highly recommend Andrew Ng's Machine Learning course on Coursera.
i'd suggest following prof ng's free online course instead, just finished it a few weeks ago and it's really good!
https://class.coursera.org/ml-004/
I also finished the last of the videos over the weekend. I thought the course was excellent -- just the right amount of detail to get you started, and yes, his enthusiasm for the material is contagious.
Ng's machine learning is the first class on Coursera I actually saw through to completion, and I'm a little sad now that it's over. It reminded me of all of the best classes I took in college, how excited I was about the material and the inevitable letdown when the class ended.
Don't look at the prerequisites and dig right in. I don't think you'll have any problems. Linear Algebra is something that kinda stand on its own.
You should not think like that when learning. If you want to be efficient just take shortcuts.
Learn machine learning (btw there are great courses on coursera or Stanford about that), and if you're stuck on something just check on wikipedia or another resource.
You should check out the Coursera ML class videos.
100% agree except that I think courses like this are great for people who want to bluff their way through a job interview to get one of those $150K-$250K big data jobs that are the rage right now (watching Andrew Ng's machine learning lectures beforehand as well would be the pro move in my book). In my experience so far, most of these positions appear to be Java programming gigs where, sadly, issues like SIMD, cache locality, and the GPU are actively ignored or dismissed. Autoboxing alone to use Java generics pretty much destroys any hope of efficient cache use.
I've walked out of job interviews over this sort of thing wherein it's been described as a "performance programming" position for big data and/or machine learning except that everything has to be in Java. And sure, Java performance programming is a thing, but compared to what one can achieve with SIMD, attention to cache-locality and/or running on the GPU for the 20% of the code that eats 80-99% of the cycles, it's laughably uninteresting to me, big bucks aside.
For some more elementary material, I also recommend Andrew Ng's machine learning course on Coursera. He's a great teacher.
Definitely a solid foundation in linear algebra and statistics (mostly Bayesian) are necessary for understanding how the algorithms work. Check out the wiki portals (http://en.wikipedia.org/wiki/Machine_learning) and (http://en.wikipedia.org/wiki/Artificial_intelligence) for overviews of the most common approaches.
Also, Andrew Ng's coursera course on machine learning is amazing (https://www.coursera.org/course/ml) as well as Norvig and Thrun's Udacity course on AI (https://www.udacity.com/course/cs271)
I'm currently going back and learning linear algebra from the excellent MIT OCW course [1]. I somehow didn't have to learn linear in the course of a CS degree at a big state school. Focusing solely on learning this one subject has allowed me to get the subject at a much deeper level than learning enough to pass a test.
I'm hoping to move on to statistics, which I slept through because I was an immature shit-head my sophomore year.
Then a review multivariable and diff-eq, and the end goal is to do the full Stanford Machine Learning class, taught by Andrew Ng [2]. I also really want to tackle "Underactuated Robotics", which is another MIT OCW course.[3]
[1] http://ocw.mit.edu/courses/mathematics/18-06sc-linear-algebr...
[2] http://see.stanford.edu/see/courseinfo.aspx?coll=348ca38a-3a...
[3] http://ocw.mit.edu/courses/electrical-engineering-and-comput...
I'm taking Coursera's Introduction to Functional Programming with Scala and Machine Learning classes and Udacity's Intro to Parallel Programming class. All out of interest.
I really recommend Machine Learning, this is my second time taking it (taking as a refresh). It really gives a good introduction on how to predict or classify data.
If you want something more practical, Introduction to Functional Programming with Scala is really informative, teaches you Scala and gives you another perspective on how to make functions and classes. It's definitely a great course for recursion lovers.
If you're interested in Machine Learning, the outstanding Coursera course on machine learning just started a couple of days ago. It covers a variety of machine learning topics, including image recognition. The first assignment isn't due for a couple of weeks, so it's a perfect time to jump in and take the machine learning course!
https://www.coursera.org/course/ml
For a glimpse into machine learning, check out Professor Yaser Abu-Mostafa's "Learning From Data" course from Caltech. The videos are online for free (http://work.caltech.edu/telecourse.html, https://www.edx.org/course/caltechx/cs1156x/learning-data/11...), and its corresponding book is on Amazon (http://www.amazon.com/Learning-From-Data-Yaser-Abu-Mostafa/d...).
Also Professor Ng's course from Stanford (http://cs.stanford.edu/people/ang/?page_id=22).
Agreed. Take the Machine Learning class on Coursera if you need an introduction to them.
Here's a comment which I had written earlier on another article. The context was about learning ML with Python - of course the objective of Hal is more generic but some parts of it apply here too.
https://news.ycombinator.com/item?id=5802968
The book details building ML systems with Python and does not necessarily teach ML per se. It is a good time to write a ML book in Python particularly keeping in mind efforts to make Python scale to Big Data [0].
What material you want to refer to is entirely dependent on What you want to do?. Here are some of my recommendations-
Q : Do you want to have an "Introduction to ML", some applications with Octave/Matlab as your toolbox?
A :Take up Andrew Ng's course on ML in Coursera [1].
Q : Do you want to have a complete understanding of ML with the mathematics, proofs and build your own algorithms in Octave/Matlab?
A : Take up Andrew Ng's course on ML as taught in Stanford; video lectures are available for free download [2]. Note - This is NOT the same as the Coursera course. For textbook lovers, I have found the handouts distributed in this course far better than textbooks with obscure and esoteric terms. It is entirely self contained. If you want an alternate opinion, try out Yaser Abu-Mostafa's ML course at Caltech [3].
Q : Do you want to apply ML along with NLP using Python ?
A : Try out Natural Language Tool Kit [4]. The HTML version of the NLTK book is freely available (Jump to Chapter 6 for the ML part) [5]. There is an NLTK cookbook available as well which has simple code examples to get you started [6].
Q: Do you want to apply standard ML algorithms using Python?
A : Try out scikit-learn [7]. The OP's book also seems to be a good fit in this category (Disclaimer - I haven't read the OP's book and this is not an endorsement).
[0] http://www.drdobbs.com/tools/us-defense-agency-feeds-python/....
[1] https://www.coursera.org/course/ml
[2] http://academicearth.org/courses/machine-learning/
[3] http://work.caltech.edu/telecourse.html
[4] http://nltk.org
[5] http://nltk.org/book/
[6] http://www.amazon.com/Python-Text-Processing-NLTK-Cookbook/d....
[7] http://scikit-learn.org
I've taken the Coursera ML class. It's very easy if you have the adequate math background. And it's not very comprehensive, there are lots of machine learning methods that are not covered. So it's more like an introductory course to machine learning.
But it's absolutely commendable how Andrew Ng takes the topic to such an understandable level that a clever high schooler who knows a little about programming could take the course. There's even extra videos serving as a crash-course to linear algebra and octave programming in the first week.
So he really manages to make the topic accessible to a very large audience.
If you are interested in this you might want to also look at Andrew Ng's (Stanford) Machine Learning course that is starting soon on Coursera.
https://www.coursera.org/course/ml
I'm considering doing this. Have you tried playing around with simple neural nets? Andrew Ng's machine coursera course is really phenomenal and drops you into doing neural nets using octave, which makes the understanding really easy. After doing it in octave, and writing some simple discriminators, I was able to really rapidly write neural nets in several languages - I even wrote one in python - a language I don't 'know' - to play around with in quantopian. Needless to say, the neural net lost a lot of virtual money, but I figured out why. =)
As I discovered myself, Hawkins' algorithm is not the best place to start learning about machine learning. I would suggest getting an overview elsewhere, such as maybe Andrew Ng's Coursera course, then coming back to Hawkins.
You get annotated as well as original PPT-slides along with clear text transcripts of what he says in the videos. Can be a bit awkward as it is not a textbook text but it gets the job done. I honestly think it is hard to do a better course than what you get from Ng's Machine learning on Coursera.
The book details building ML systems with Python and does not necessarily teach ML per se. It is a good time to write a ML book in Python particularly keeping in mind efforts to make Python scale to Big Data [0].
What material you want to refer to is entirely dependent on What you want to do?. Here are some of my recommendations-
Q : Do you want to have an "Introduction to ML", some applications with Octave/Matlab as your toolbox?
A :Take up Andrew Ng's course on ML in Coursera [1].
Q : Do you want to have a complete understanding of ML with the mathematics, proofs and build your own algorithms in Octave/Matlab?
A : Take up Andrew Ng's course on ML as taught in Stanford; video lectures are available for free download [2]. Note - This is NOT the same as the Coursera course. For textbook lovers, I have found the handouts distributed in this course far better than textbooks with obscure and esoteric terms. It is entirely self contained. If you want an alternate opinion, try out Yaser Abu-Mostafa's ML course at Caltech [3].
Q : Do you want to apply ML along with NLP using Python ?
A : Try out Natural Language Tool Kit [4]. The HTML version of the NLTK book is freely available (Jump to Chapter 6 for the ML part) [5]. There is an NLTK cookbook available as well which has simple code examples to get you started [6].
Q: Do you want to apply standard ML algorithms using Python?
A : Try out scikit-learn [7]. The OP's book also seems to be a good fit in this category (Disclaimer - I haven't read the OP's book and this is not an endorsement).
[0] http://www.drdobbs.com/tools/us-defense-agency-feeds-python/...
[1] https://www.coursera.org/course/ml
[2] http://academicearth.org/courses/machine-learning/
[3] http://work.caltech.edu/telecourse.html
[4] http://nltk.org
[5] http://nltk.org/book/
[6] http://www.amazon.com/Python-Text-Processing-NLTK-Cookbook/d...
[7] http://scikit-learn.org
Here's some free book lists
http://www.kaggle.com/wiki/Tutorials
http://metaoptimize.com/qa/questions/186/good-freely-availab...
I think the best single resource is Kevin Murphy ML text, but there's lots of relevant stuff on linear algebra, Bayesian analysis etc
______________
Here's a "curriculum"
http://blog.zipfianacademy.com/post/46864003608/a-practical-...
And look at the coursera courses by Koller, Andrew Ng, NLP by Collins etc
Keep it going! I attended the very first one (~75%), and then did a second time last year. It was then easy to get to a perfect 100% score, having done most of the exercises during the first time.
It is one of the best Coursera classes. I had a blast, and strongly recommend it. I decided to continue learning ML, mostly because Prof. Ng.
I have to blame you, because now I had to take the course in coursera! I always wanted to learn machine learning. Better late than never!
That's awesome to hear! I'm taking the Coursera class now, and it's been great so far. It just started a couple weeks ago, so it definitely isn't too late to join! https://www.coursera.org/course/ml
One of the main problems the author has with Coursera or EdX is that it is "too fast". To address this problem, I urge the author to try Udacity [0]. You can set learning at your own pace.
That said, not all material on Coursera is "amazing". Some of the classes have very high completion rate like Functional Programming with Scala has a 19.2% completion rate [1]. Similarly, the class taken by Andrew Ng on Machine Learning is fantastic. However, many of my peers had bad reviews on Daphne Koller's Probabilistic Graphical Models class. Last year, I myself registered for one of these MOOC's and found that half of the course was good while the other half was quite bad - both halves had different professors.
At some point of time, universities would have to realize that great researchers do not make great teachers. Some excel in both - researching & teaching while some in just one of those two fields.
PS - Other problems on reengagement do stand though.
[0] https://www.udacity.com/ [1] http://www.katyjordan.com/MOOCproject.html
I would also recommend the Coursera course on Machine Learning by Andrew Ng and Probabilistic Graphical Models by Daphne Koller. I would also go over some basics probability and statistics review. Maybe some linear algebra too. Python is a great language to do data analysis in. I recommend the scikit-learn and pandas packages and using ipython notebooks. Another book is the Elements of Statistical Learning (http://www-stat.stanford.edu/~tibs/ElemStatLearn/). There are also Kaggle contests for testing your chops.
Yes, I found the class somewhat slow going, and the programming environment fairly unpleasant (I’m not used to working in Eclipse). I ended up dropping the class somewhere in the middle.
Generally, I learn better from books than from lectures, but I found that some of the lecturers in online courses are truly outstanding (Prof Agarwal in Circuits and Electronics at MITx, Profs Ng (Stanford, Machine Learning), Egerstedt (Georgia Tech, Control of Mobile Robots), Pattison (Berklee, Songwriting)), so listening to their lectures was worth it. I did not quite find Odersky as strong as a lecturer.
"As least in Scandinavia, we don't feel MOOCs are a huge threat to us, because there is no tuition, so there is no real financial reason for students to prefer MOOCs."
At least the best Stanford MOOCs, Machine Learning and Databases, were taught much better, and the material and exercises were much more carefully planned, than most of what I'd expect at least in any university in Finland.
Now, I hear Coursera has also some bad quality MOOCs, but as long as the very best courses by the very best lecturers can be offered to the whole world as MOOCs, their better quality should be a strong reason the prefer them over some mediocre course in a local uni.
How about tapping into the local universities around you ? Plenty of universities nowadays teach intro-level courses in machine learning. Someone who has gone through such courses would know more than enough to satisfy your needs. I'd love to help, but I'm not based in Amsterdam :)
Failing that, there're always online resources to turn into. I highly recommend this https://class.coursera.org/ml/lecture/preview. This is how I got started with machine learning too. It's not hard to hack together a basic recommendation system quickly ... although for the long term, you might need a dedicated person to work on it since providing good recommendations on favourites seems to be your core value proposition.
Would 1 week of java training turn a novice into a expert programmer? I took a quick peak at the course page and it does look very interesting. However, I dont think you can call yourself a data scientist after taking this course. If you want some free resources, check out coursera's machine learning course. You can also find some free map reduce training from google. In my opinion, becoming a data scientist takes years - not days.
What exactly do you want? Do you want a resource that "clicks" with you that you can use to learn about machine learning, or do you want a witty piece of art like "_why's poignant guide to ruby"?
If its the former you are looking for, I have several recommendations. There are a plethora of online resources to learn about machine learning from. In video form, my favorite resource is Yaser Abu-Mostafa's (Caltech prof) video lectures, available here: http://work.caltech.edu/teaching.html
You can also actually enroll in the next version of CS 156 (Abu-Mostafa's class) and do the class online with problem sets very similar to that of Caltech students actually enrolled in the class. (It starts January 8th).
Coursera/Udacity contain several classes that involve machine learning; Andrew Ng's class is a good place to start.
If you (like me) prefer text to video, you can buy Abu-Mostafa's book (I hear it follows the course very closely, and the book costs way less than $500 :D ), read Andrew Ng's online lecture notes, or read one of several freely available ML books (such as "Elements of Statistical Learning" or "Bayesian Reasoning and Machine Learning", but I've found these to take more mathematical maturity that the other resources recommended).
Note: One reason that I've highly recommended Abu-Mostafa's resources (besides having taken the class at Caltech and loving it) is that his class is more focused on the theory of machine learning (ie hammering in over-arching principles like avoiding overfitting by regularizing) rather than just covering as many algorithms as possible. Also, I believe Mitchell of CMU has good online resources for his machine learning course.
Cool stuff... Andrew Ng's excellent Machine Learning course on Coursera also had a programming exercise that involved using k-means to reduce an image's palette (in Octave, though, rather than Python), so if you find this interesting you might consider signing up for his course the next time it comes around:
https://www.coursera.org/course/ml
Not a book, but Andrew Ng's coursera course (or Stanford class video lectures) are great and have lots of practical tips.
I'd love to learn which government official and department is responsible for this action so we can direct our dislike of this decision towards them.
P.S. The ML class Andrew Ng is teaching on Coursera is phenomenal by the way.
have you checked out coursera?
I started when I was 8, first professional programming job in 10th grade...
The jump from intermediate to "complete" is indeed very large, so expect a journey.
You need to find something you enjoy working on that you evolve over the next couple of years to learn about software engineering and what automation is important etc. This could be a game or an extension to an open source game, it doesn't really matter. It will almost certainly be a failure in some sense, so it's good to get that first failure out of the way so you can scrap it, revisit, etc.
I think coursera would be a great place for you to start, though it may require you to learn a new language (this is one step on the journey towards advanced in X language, I expect it would be octave/matlab) I'd recommend doing the introductory machine learning course and whatever other stuff interests you.
From the quality of your writing, you're quite bright for 14 - so stick to it and don't neglect your physical health and artistic/emotional/cultural education because you need to be free of distractions to reach highest level.
Take a machine learning class on coursera - you'll have a big leg up on the math. If you get the math, the algorithms are straightforward and the field is growing.
I'm looking at the same problem at the moment. PGM sounds really interesting, but I think that the time investment just isn't going to be workable for me unless I drop several of my other classes. My current plan is to watch the PGM videos and try to keep up with the programming assignments as long as I can, but if it comes down to a choice of one or the other, PGM will be the one to go.
As far as "dumbing down", I've found that the Coursera classes that I've taken (Compilers, Automata Theory, Algorithms 1, SaaS and Machine Learning) have varied in difficulty quite widely. Compilers and Automata were both challenging and enjoyable, Algorithms 1 was about what I'd expect from a freshman/sophomore algorithms class and SaaS and Machine Learning were easy enough that they should be approachable to anyone with basic programming experience.
I don't feel that the difficulty in the classes that I've taken had any particular correlation with teaching effectiveness. I found Andrew Ng's ML class to be simple, but still interesting and informative - you come out of it with enough of a basic understanding to implement simple ML techniques as well as a place to start if you wish to learn more. I think that while a theory-centric class would be a nice thing to have, he's done an amazing job of making a class that can appeal to a wide range of potential students and introduce them to a field that's usually very difficult to approach.
Hi,
I am also taking the course by Andrew Ng and understand your complaint that the programming assignments aren't as interesting ( from your perspective). Being quite comfortable with linear algebra, I was able to complete the assignments easily.
But when I go through the course forums, I find that for many people taking the course, the intuition behind the use of linear algebra in ML doesn't come as easy as it does for us. I think when Andrew Ng designed this online course, he must have had those people in mind also. I think he mentions it at the start of the course that it's more about understanding the concepts and the implementation details should come later. The programming exercises are designed keeping that in mind, I think.
I tried to make the programming exercises interesting for myself, by first thoroughly understanding the code that they had provided and tweaking it here and there. Once you have done that, you could apply what you've learnt on real world datasets from sources like Kaggle and see how you fare :)
I'm taking the Stanford Coursera class on Machine Learning with Andrew Ng, and I wholly recommend it. It just started this week, so it's not too late to join if this blog post has whet your appetite and you're looking for more.
https://class.coursera.org/ml-2012-002/class/index
Yeah, definitely.
If you're just getting started, I highly recommend Andrew Ng's online ML class. I had started reading up before this was available, but it really tied a lot of basics together that I was confused about.
From there, read papers. For me, my primary interest is in neural networks. Geoffrey Hinton and Yoshua Bengio have two very good groups that both have contributed a great deal to research in this area, and their websites provide lots of good stuff.
After spending a bit of time on a survey of the field, try to come up with a practical goal as quickly as you can: I want to use ML to do X. Then, try to do that. When you get stuck, get back to reading until you find the answer. Rinse and repeat. The math naturally falls into this--you'll get stuck on things that you can't fix without a decent understanding of the math. So figure out how to formulate the question you're really asking, hit google, and read. Then try again. Rinse and repeat.
If you're interested in neural networks, I can also recommend the deep learning tutorials associated with Theano, a library for compiling python code down to CUDA code to get speed increases on certain operations that will let you train your models about 10-40x faster on GPU than if you tried it on CPU.
For me, putting things into practice has helped me make the biggest leaps in understanding, but of course I wouldn't have been able to do that at all without getting a basic grasp of the mechanisms involved. So it's a bit of push and pull between practice and learning, like anything worth doing.
I am doing the https://www.coursera.org/course/ml from Stanford by Andrew Ng & I definitely recommend it.
I'm really excited by all of this free university level material flooding the web as I never even started college due to financial concerns (aka I didn't want to get any loans).
This seems very related to this http://www.youtube.com/watch?v=ZmNOAtZIgIk speak by Andrew Ng. It is a 40min speak, but he explains very simply how all this works for images and some examples about the audio case. It is incredible how using this deep learning techniques we can teach this "neural networks" to recognize such complicated patterns. It is like reverse engineering the brain's algorithms.
BTW I took his Coursera's course about Machine Learning and it was great! I also recommend it A LOT to gather basic ML knowledge.
I'm almost through Andrew Ng's course. Did you do all the programming activities to reinforce the lectures? I've been keeping pace with the lectures, but hadn't done the homework/programming. Now I'm going back and completing them one-by-one.
I was thinking about watching Tom Mitchell's CMU course online next. Have you checked that out?
PS. Is it just me or is Andrew Ng incredible? I thought I had good professors in college, but he is on another level.
I would recommend to pick the free Stanford class... great intro to (classical) ML: https://class.coursera.org/ml/class/index
It looks like Coursera is trying to scale rapidly. If you look at the potential course offerings, Coursera has much more. The problem is it's harder to control quality. The Machine Learning course is great, but the Computer Vision is much less engaging.
On the other hand, Udacity seems more focused on a couple of core offerings.
Agree, the first AI class was horrible, while Andrew Ng's Machine Learning class on Coursera was top notch. I think both platforms have advantages and things to improve, and I'm glad they are both activelly working on it.
Not sure we need yet another platform with eDX, but let's see. So far it seems more of a West versus East type of thing..
why not work within the system you're in? At my university a bunch of grad students got together wanting to do ml/pgm/nlp-class, so we put together a small group, added a semester project plus paper presentations to make it worthy of a grad class. It's a bummer that the coursera courses started so late, but otherwise it's working out great. We help each other out and teach each other new stuff, and in the end get credit ;)
If you can't get departmental approval for something like that, then you should at least be able to swing an easily justifiable independent study out of one of the free classes.
But online courses actually give you more options as far as personalizing education. At most universities, unless they're very large or very specialized, you're likely to have one professor from a specific subject in a field. With online lectures, even from a reasonably small number of sources you can mix and match. Two personal examples I can think of is supplementing my algorithms class with MIT's algorithms class on OCW, and watching Tom Mitchell's lecture on kernel functions to add depth to the topic as covered in Ng's Ml-class.
Even at great universities there are professors who are better researchers than teaches that still may be a students only option. If you took the top 10, or even 5 teachers on a given subject you could give students the freedom to mix and match as suited their learning style.
Personally I actually find the lectures a poor substitute for a text. I find the Probabilistic Graphical Models textbook a great addition to the pgm class on coursera. Even though the lectures for pgm-class are long and detail, there's still only so much information you can get into a few hours. The information density of a textbook is hard to beat.
Andrew Ng's course[1] covers the same ground and was excellent.
[1] http://www.ml-class.org/course/auth/welcome
the best aspect of the andrew ng course was the homeworks using octave. Yes, it's watered down and not as mathematically rigorous as the real course, but you learn a lot of the essentials from the experience of coding machine learning algorithms, that I can't imagine learning as easily from doing non-programming homework.
It would be great if they can coordinate their classes with Coursera. I signed up for algo-class.org, but now I see that Udacity will also have similar class.
Edit: I completed ML (Coursera) and AI Intro (Udacity) classes and I have to admit that execution of Coursera was way better. Not sure how things will be for algo classes.
I could not disagree more. I have an MIT degree, and I thought it was remarkably refreshing to have the difficult made easy, rather than vice versa. Not that I have huge complaints with my MIT education, but having taken some classes at Harvard too, where the emphasis was sometimes different, I can appreciate that there's often a great deal of value in a class that can draw people in and make them interested, rather than send them fleeing in terror. Personally, I thought that Prof. Ng's lectures where brilliant in their ability to make difficult material completely approachable.
I do think that he occasionally would belabor the obvious, but for the intended purpose of the class, I think that it is better that he erred on the side of being too clear, rather than on the side of being opaque.
Unfortunately the "difficulty" of the ML class assignments was not in the material application of the things learned in the lecture but instead in the gotchas of Matlab/Octave vectorization. Most of the assignments were in the format of: "Here's a forumla that applies to the element, generalize it to the matrix so there's no for-loops." Which, while challenging for those not accustomed to thinking that way, was not a challenge of applying what one learned from the lectures.
The course was awesome for what it was: a chance for the average programmer to get their feet wet in ML, but the fact that it was hosted by Stanford seems to lead to a somewhat humorous irony that it was pretty accessible whereas Stanford's real-life rigor is supposed to be nothing of the sort.
I share your opinion.
I wish I could spend the whole week to dive into AI, IMHO without previous knowledge a regular person can't catch up with course progress. Unfortunately I can't leave everything behind to study the deeps of AI and in this case the ML does a great job, Professor NG presets lectures and a few quizes to see if you understand that sensible topic, then you can apply those concepts with programming exercices and homework.
I'm almost giving up about my dream that is attend a course in Stanford. I'm a 28 college dropout who had to start working to help family. I don't live in US, I'm married, I have a great job (that I believe is far more interesting than most of you guys). But I have this itch for Academics. *
I'm glad I've been playing around with both ML and AI classes but I think that Stanford should stick with Professor NG didactics.
ps: I know that my statement sounds like the "occupies" but is hard to see dreams not becoming reality. I feel sorry for those who dropout Stanford/MIT to create a landing page startups.
truly amazing!
about the stanford ML course (i'm not taking any others) -- i especially like the fact that i can always rewind, re-watch, pause-take notes-play, answer questions during the "lecture" without the embarrassment of getting it wrong the first time, unlike in real lectures. the forums are there in case of questions/problems. the content is presented in a clear and concise manner. and the length of each "lecture" is 10-15 minutes, no need to focus heavily for an hour straight.
for people who have not seen the ted talk by salman khan http://www.ted.com/talks/lang/eng/salman_khan_let_s_use_vide... which, along with increasing number of online courses from prominent universities, suggests that the educational system is changing.
people who are behind this, i salute you.
Bump. In the past, I've been through Stanford's IOS dev course and ML course (via iTunes). Currently, I'm doing the AI course. It is truly amazing that Stanford is giving away this content for free.
As an academic, it makes me wondering how education will change over the next 10 years. If a prof gives a fantastic lecture in the Fall of 2010, does he need to repeat himself in 2011? I had the experience of lecturing an undergrad course in Programming Languages many years ago. Honestly, the content didn't change much (I was teaching Scheme, ML and Prolog). I did get a bit better answering and anticipating student questions. I'll admit that in some cases, content must be updated fairly frequently. For instance, Prof. Ousterhaut's web app course on the site seems to be covering Rails 2.x.x. Teaching it today, perhaps 3.x or 3.1 would be used.
However, when I look at how well the community can improve the quality of a lecture, it blows my mind. In the Stanford AI class, for instance, the lecturer made a slight error where he defined the admissibility criteria of the h function (estimated cost) in A* search as less than the true cost rather than less than or equal to the true cost. Well ... this was quickly spotted by students and a correction was promptly issued. This blows my mind!
This also has the potential of making things worse on the supply side of education. In the last decade, it has undoubtedly gotten harder to get a faculty position in Computer Science. Will the advent of online education make that situation worse? As someone who has been an educator in the past, my personal opinion is that education should be available for free. But, I worry that it might not be sustainable.
I am most excited about the machine learning course, since I could use the structure to go through Andrew Ng's lectures; also the ones from 2008 are already out of date.
As for the AI course, I'm skeptical, frankly. Sign-up right now has meant putting your name and email address. There is a big difference between spending 2 second to fill that in, and spending the amount of time necessary to finish the course. You come to a website that sounds like making robots and turns out the class is about developing pruning heuristics for search trees, do you stick with it? Maybe...but I'm going to save my awe for how many people complete.
I praise Stanford (CS) for thinking up this idea. Like Khan, they are going to reach a lot of people, and I think its particularly useful for motivated learners who, for whatever reason, are stuck in a place that you can't take a good AI course or can't afford one. Ultimately, though, the course is going to require drive that a lot fewer than 65,000 have.
I highly recommend Andrew Ng's lectures also. I was going through them myself, and he does a good job of explaining the intuition behind many things (of which the math, although important, is only a formalization of).
One piece of advice I would give from my own experience is that you have to play with data to get practical experience applying the methods. Machine learning is not a set of plug and play blackboxes that you can feed random input into and get clean output. You have to spend a lot of effort understanding your data and how they relate to the specific method you're using. For example, if you use linear regression as your learning model, you have to understand what kind of relationship is assumed between the inputs and outputs (in this case, that the output is a linear combination of the inputs).
I know this because when I started, I would just toss unclean, unfiltered, and untransformed data into a method and hope for good results. Of course I fed garbage in, so I got garbage out.
Another word of advice is to watch out for overfitting. Often, you'll find that your training gives you good in-sample statistics (for example, with linear regression you'll get great R^2 with high p-values). However, when you test out of sample, you'll realize quickly that most of the models you've fit are overfit to the data that you trained on. Just something to be aware of.
I guess both of these may be very abstract and useless for you right now, but hopefully one day you'll look back and able to find use for it.
Take a look at Machine Learning video lectures by Professor Andrew Ng (Stanford). Highly recommended. http://www.youtube.com/view_play_list?p=A89DCFA6ADACE599
You'll also find a bunch of resources on this page - http://www.quora.com/Machine-Learning/What-are-some-good-res...
In case you see the cheat sheet and think, "Wow, I'd love to understand that," there's an excellent (albeit challenging) complete course on machine learning in Stanford's "engineering everywhere" online repository. http://see.stanford.edu/see/courseinfo.aspx?coll=348ca38a-3a...
I'd suggest you look into machine learning. It's probably going to be the single most important area of computing in the years to come. Start with the Stanford course online.
Also, if you haven't already, learn functional programming.
That's a good set of lectures.
My girlfriend recently did several as she was learning about CS as a postgrad, and found them very useful.
They, and many more, are also available on 'iTunes university' from within the iTunes interface. The Stanford CS courses that are up are also well worth a look; there's a good Machine Learning course up there.
The only issue is that the videos often aren't properly edited for the Internet. They don't strip out the class administrative parts, and the lectures would go a lot better with a navigable text index to allow you skip to different parts of the video. The Khan Academy is also worth a look as a similar project.
With a little more care and attention to editing and presenting for the web, this could have a big impact on education.
If your math background is weak enough that you don't feel totally at home using calculus and linear algebra, I'd recommend "Principles of Data Mining", which they use for the intro to ML class at Princeton. The ratio of English to equations is pretty high, which might help you to get into the swing of things.
If you think that you can learn enough math to follow the material, you should watch the Stanford lectures given by Andrew Ng. They're available freely online through iTunes U. Andrew is also one of the most respected experts in the field. As he points out, you will eventually need to understand the math behind the methods. Otherwise, a day will come when you start wasting time coding algorithms that plainly don't apply to the problem at hand.
I would rather recommend Machine Learning video course from Stanford by Andrew Ng.
Pattern Recognition and Machine Learning by Christopher M. Bishop is good too
I second the nomination of Bishop. It is the standard text. It is only two years old, and Bishop will teach you machine learning the way that the field practices it nowadays. In my lab of fourteen people, we must have six or so copies of Bishop.
I don't understand what is impractical about Bishop. If you are looking blindly to use an off-the-shelf machine learning implementation, that's one thing. Machine Learning has been described as the study of bias. If you want to understand when to pick certain techniques, and develop appropriate biases, then read Bishop.
"The Elements of Statistical Learning" by Hastie, Tibshirani and Friedman gives more of a statistician's approach. The treatment is simply less broad, and also more dated.
You can also look at Andrew Ng's video lectures: http://www.youtube.com/watch?v=UzxYlbK2c7E He is very well-respected in the field. For certain students, watching a lecture may be preferable to reading a book.
IMO that's a bit of a strange video for Karpathy to start with, perhaps even to include at all.
Let me explain why ...
Neural nets are trained by giving them lots of example inputs and outputs (the training data) and incrementally tweaking their initially random weights until they do better and better at matching these desired outputs. The way this is done is by expressing the difference between the desired and current (during training) outputs as an error function, parameterized by the weights, and finding the values of the weights that correspond to the minimum value of this error function (minimum errors = fully trained network!).
The way the minimum of the error function is found is simply by following its gradient (slope) downhill until you can't go down any more, which is hopefully the global minimum. This requires that you have the gradient (derivative) of the error function available so you know what direction (+/-) to tweak each of the weights to go in the downhill error direction, which will bring us to Karpathy's video ...
Neural nets are mostly built out of lego-like building blocks - individual functions (sometimes called nodes, or layers) that are chained/connected together to incrementally transform the neural network's input into it's output. You can then consider the entire neural net as a single giant function outputs = f(inputs, weights), and from this network function you can create the error function needed to train it.
One way to create the derivative of the network/error function is to use the "chain rule" of calculus to derive the combined derivative of all these chained functions from their own individual pre-defined derivative functions. This is the way that most machine learning frameworks, such as TensorFlow, and the original Torch (pre-PyTorch) worked. If you were using a machine learning framework like this then you would not need Karpathy's video to understand how it is working under the hood (if indeed that is something you care about at all!).
The alternative, PyTorch way, of deriving the derivative of the neural network function, is more flexible, and doesn't require you to build the network just out of nodes/layers that you already have the derivative functions for. The way PyTorch works is to let you just use regular Python code to define your neural network function, then record this python code as it runs to capture what it is doing as the definition of neural network function. Given this dynamically created neural network function, PyTorch (and other similar machine learning frameworks) then uses a built-in "autograd" (automatic gradient) capability to automatically create the derivative (gradient) of your network function, without someone having had to do that manually, as was the case for each of the lego building blocks in the old approach.
What that first video of Karpathy's is explaining is how this "autograd" capability works, which would help you build your own machine learning framework if you wanted to, or at least understand how PyTorch is working under the hood to create the network/error function derivative for you, that it will be using to train the weights. I'm sure many PyTorch users happily use it without caring how it's working under the hood, just as most developers happily use compilers without caring about exactly how they are working. If all you care about is understanding generally what PyTorch is doing under the hood, then this post may be enough!
For an introduction to machine learning, including neural networks, that assumes no prior knowledge other than hopefully being able to program a bit in some language, I'd recommend Andrew Ng's Introduction to ML courses on Coursera. He's modernized this course over the years, so I can't speak for the latest version, but he is a great educator and I trust that the current version is just as good as his old one that was my intro to ML (building neural nets just using MATLAB rather than using any framework!).
- AIMA by Russel and Norvig is a classic but I would say is more of overview of the field and for most topic areas isn't quite deep enough imo.
- For deep learning specifically, a more applied text that is beautifully written and chock full of examples is Francois Chollet's Deep Learning with Python (there a new second edition out with up to date examples using modern versions of Tensorflow). The first 3 chapters I would give as required reading for anyone interested in understanding some deep learning fundamentals.
- Deep Learning - goodfellow and bengio - seems like it would be hard to get through without a reading group not exactly a APUE or K&R type reading experience but I haven't spent enough time with it.
If you haven't taken a Linear Algebra or Differential Equations class its useful stuff to know for ML/DL theory but not fully necessary to do applied work with modern high level libraries, but definitely having a strong understanding of basic matrix math is useful.
If you have interests in natural language processing theres a couple good books:
- Natural Language Processing with Python - Bird Klein, Loper, is a great intro to NLP concepts and working with NLTK which may be a bit dated to some but I would definitely recommend, and its online for free. Great examples.(https://www.nltk.org/book/)
- Speech and Language Processing - Dan Jurafsky and James H. Martin - is good, though I have only spent much time with the pre-print
And then theres a lot of papers that are good reads. Let me know if you have any questions or want a list of good papers.
If you just want to get off the ground and start playing with stuff and building things I'd recommend fast.ai's free online course - its pretty high level and a lot is abstracted away but its a great start and can enable you to build lots of cool things pretty rapidly. Andrew Ng's online course also is quite requitable and will probably give you a bit more background and fundamentals.
If I were to choose one book from the bunch it would be Chollet it gives you pretty much all the building blocks you need to be able to read some papers and try to implement things yourself and I find building things a much more satisfying way to learn than sitting down and writing proofs or just taking notes but thats just my preference.
I don't doubt that there are people working at EdX / Coursera with a passion for education. I just think maybe these companies are moving in a direction that is at odds with the goal of providing free education, everywhere, to everyone, at any stage in their life.
I enrolled in some of the earliest MOOCs. Sebastian Thrun's original ai-class.com which now redirects to Udacity. I took the first iteration of Andrew Ng's "Machine Learning" on Coursera, as well as Geoffrey Hinton's original NNML course. Back then, everything was open. Course materials were shared freely, and the archives were available for years after the course concluded. There was an autograder for coding assignments that didn't get in your way too much.
Slowly, more and more roadblocks were put in place.
What was your experience like at Coursera? Did you get a chance to see how decisions about the UI and structure of courses were made? Did you get a sense of how much the marketing / business side of things interfered with the education side?
For who? Maybe these sites have created a product that works well for a certain niche of people, and they've hyper-optimized for that. Great. But that's not really the dream we all had for it ten years ago.
Like I said in a sibling comment: I've already been through school, and already know my own learning process. I find that the practices Coursera / EdX actively get in the way of my learning.
Language changes. Most people include in their meaning of "dark pattern" things like "artificially restricting you from performing actions that the website is fully capable of performing, with dubious or justification or malicious intent".
I don't think EdX is malicious, just that their reasons for restricting usage of course materials are dubious, and conflict with their stated mission.
Why can't it be a content library? I learn a lot at libraries!
This structure helps some people, sure. But some people like me are not full-time students. Some weeks I have lots of time to dig in, other weeks I don't have time to even watch a lecture. Moreover, I'm learning for myself, not for credentials, so why should I care what a website thinks of my progress?
I don't have definitive proof, but every time I visit the EdX or Coursera sites it just so happens that the exact course I was searching for started within a week of the current date. Maybe I'm being paranoid.
This was mostly a joke :)
Again, I'm not a student. I trust my own learning process, which is impeded by constant quizzes. I'm doing this to broaden my knowledge. I don't have time to enroll in a college class, but I have time to listen to a few lectures when doing dishes, and read a couple book chapters per week.
Coursera and others are technically capable of opening up their service to this use case -- it doesn't cost them anything -- so why not do it?
In a certain sense, online education is thriving! There are tons of video lectures on YouTube available for free and I can easily pirate any textbook I want to with a quick Google search. It's just that Coursera / EdX / etc don't really fit into that for me. I really wish they did.
You are definitely onto something though. No matter how many points I make, musicians do like Spotify less than prior solutions, and musicians have a really hard time turning Spotify into a living. I was really only arguing against some causes for that observation (specifically, allocating a pool of funds based on some "usage" metric seems reasonable), but they're not necessarily off the hook. Here are a couple thoughts more in line with what I think your position is:
(1) Inflation adjusted, per capita we spend 5x less for music now than we did 20 years ago. Is that because Spotify doesn't charge power users more? Is it because users won't accept higher rates for song rentals? Regardless of the cause, there's less money to distribute.
(2) Popular music is more popular than ever. For a variety of reasons (some potentially Spotify-induced -- e.g., it's pretty hard for me to get it to recommend somebody who isn't soundly leading a genre it thinks I might be interested in), the top X% of artists are more listened to than ever, and that's going to exacerbate any other problems for the long tail of artists.
(2a) That idea is affecting a lot of industries right now. At some point in recent decades it was easy to be the best <thing> in your local small town or group of peers and get all the business. Plenty of formerly in-person industries are now competing globally though. E.g., I would have a hard time recommending the best ML practitioner in my hometown as a teacher rather than just taking Andrew Ng's courses. I'm not sure how that's going to play out in general, but if the top 1% of the population in any field can make everything we want/need then the idea of having to work for a living might need to be revisited? Maybe we'll eventually all transition to biomedical research when nothing else is scarce? In any event, in the short term a lot of professions are being disrupted.
I've taken almost 10 MOOCs and 5 courses in GT's OMSCS program, spanning from a member of Ng's first ML course, to a currently lackluster offering I'm taking for credit.
To take a step back, I think what makes a course great is just its ability to transfer knowledge to students and to spark curiosity (further learning after course finishes). Usually, when you take a course in person, the folks teaching the class have ample opportunity to ensure that transfer is happening, in online courses, the onus is really on the student.
Thus, in an online course, you need to provide clear learning goals, for instance, "dynamic programming", a set of lectures that teaches those concepts, then homework or project based work that directly refers to what has been said in the lectures. Decide what your resources are, say, a book, your videos, and a few homework assignments, then try to have the resources build off each other, and make sure information is available in multiple places. Sometimes with difficult concepts, you need things explained twice, from two different sources!
The next aspect of a great course is the fostering of a learning community. Make sure the students have a place where they can engage (slack, discord, et cetera) and ask questions. Having students make a social connection has been shown to increase their outcomes in online learning, and for me, the early Coursera courses were amazing for this.
Finally, there is the aspect of of running the course. Some students don't do well learning online, and I some of this can be explained by the loss of human to human contact and relationship building and mentoring that can help struggling students during in person office hours. A percentage of students truly rely on this, and there is no online alternative. If you don't grok them pythonz, data science can be a nightmare (R 4 lyfe!!!).
Someone much smarted than me figured out you need 3 things to learn: time invested, a consistent environment, and feedback. Make sure you have enough TAs to actually give student feedback, or figure out an autograder for problems of varying difficulty.
Many times a recommendation system is opaque and not adjustable because the people who made it don't know how it make it transparent and adjustable. A good example of such a system was given in Andrew Ng's old machine learning MOOC. I'll describe it below.
Suppose we want to do a movie recommendation system, and we have a bunch of data consisting of anonymous user identifiers and for each user a list of movies they have seen and their rating of that movie.
Imagine that we had a list of movie attributes that we thought might be relevant to whether or not someone would like the movie. This might be things like running time, if it is funny, if it has car chases, if it has romance, if there are horses in it, if there is profanity, and so on. Let's say we've got 50 of these attributes.
Now suppose we had for each movie a vector of length 50 whose components were how well the movie fit each of those attributes, on a -1 to 1 scale.
If we had that movie attribute data, then what we could try is modeling the users as if each user had a 50 component vector telling how important each attribute is for that user, and for each user we could go over the list of movie ratings and try to find a set of weights for the components of that user's vector such that the dot product of their vector with a given movie's attribute vector correlates with that user's rating.
That works pretty well, but there are two practical problems. First, we need to come up with that attribute list for the movies. Second, once we've got our attributes someone has to go through each movie and figure out the vector.
So forget about that approach for a moment. Suppose instead we came up with a list of attributes, somehow, but instead of figuring them out for the movies and then inferring the user weights, suppose we told the users the attributes, and asked them how important each was, and assumed that the users are actually right about what is important to them?
Then maybe we could take all the movies, and try to assign attribute weights to them that lead to consistent predictions of user scores!
It turns out that works, but we still have the problem of guessing what attributes matter, and depend on the users actually somewhat knowing what makes them like a movie.
So...if we knew the movie attribute weights we could infer the user preference weights, and if we knew the user preference weights we could infer the movie attribute weights.
The brilliant solution is to get rid of the attribute labels. All we decide on is the number of attributes! So we might decide that there are going to be 50 attributes, and we can assign the movies random weights for all those attributes. We can assign the users random preferences for the attributes.
Then you can go through an iterative processes where you tune the movie attribute weights to better predict user scores, and you tune the user preferences to better match the movies. This ends up converging to a set of movie attribute weights and user preference weights that do a good job of reproducing the user's scores for the movies, and makes good recommendations...
...and you have no idea when it is done what the attributes mean!
Unless I knew for certain that I couldn’t do solo work, or knew for certain that I couldn’t muster the discipline, I would say “work on your own” in the beginning.
If I am to take your harmonious financial arrangement at face value, which you reaffirm in another comment, understand that your situation is probably very enviable to many. Make the most of it.
You didn’t specify but I assume you expect this situation to hold for at least a year, maybe two or three years? That’s great. And you already dabbled in ML, and you “really like it”? Well, its the remote option, but this is what I would do :
1. I would brainstorm a project about applying ML to law, or my field of law[0]. This is not completely unheard of, as I’m sure you’re aware.
2. If I come up with some holy grail ideas, I would put them aside for later. I would then try to find something to start with that is very simple, maybe something that would have helped me day-to-day in my previous lawyer life. Maybe something much smaller that would be a necessary but not sufficient item for one of the holy grail ideas, or like I said some small but obvious value added day-to-day lawyer stuff.
3. After I decided on the project, I would apply the Amazon way of working backwards, a. Create the press release: Describe in simple terms what you’re going to build b. 20 most frequent questions: Answer in clear, simple terms (& iterate until it is very clear what you’re going to build) c. UX Document: How are my users going to interact with this d. Keep the above simple, don’t overdo it.
4. So you came up with your first project, now you have to skill up. Since you dabbled in ML maybe you are already familiar with the ML learning angles people recommend on this site (fast.ai, Coursera etc). I would research and make a decision which ones to start with first (maybe you’ve already completed some) and focus on completing them together with all the relevant exercises etc. In parallel, I would be on the lookout about what I need to learn for the project and continuously apply what I learn to the project.
5. Out of necessity, you are probably going to need to dabble in integrating frontend web and mobile stuff. Research and decide on learning path based on what concrete things you actually need in your project.
6. Once you feel you know enough to build the shittiest and dumbest version of your idea build the shittiest and dumbest version of your idea before continuing a new online course in your studying plan.
7. Put your code on the web when you have the shittiest and dumbest version of your idea working and keeping updating it. I would upload two separate things as projects a. the software/product that the end user will use that has everything in it. b. The ML component as a separate project. Don’t let the frontend stuff detract from the ML stuff. Imagine someone with better web and mobile skills wanted to use your ML-law work to make something more polished. They could use this.
8. Now you have a portfolio, don’t you? You could go on building the next item on the road to a holy grail, or you could continue iterating on this as you learn.
9. Meanwhile, you could start applying with your portfolio for remote work at AI tech companies targeting law or NLP areas (more general overlapping domains) , or law firms that are scared of tech who don’t want to be the nokia or blackberry of law in the future (this is probably the unlikelier of the two, since this tech would disrupt billable hours). In the former, you would be the “domain” guy for the tech, in the latter you could be the “tech” guy for the domain. Your skills and background will be unique.
10. Alternatively, or in parallel, start selling what you built. Cause looked at it another way, this is identical to have received seed funding for 1-3 years to start a company.
Sure when you start working with others you may soon feel you’ve missed out on some fundamentals, but this is what I would do in terms of opportunity cost. By the end, you will have a unique background and you will be working in a niche area suited to your background, your weaknesses will matter less. After this, if I still felt iffy and it still didn't matter if I got a job or not, then I would consider a online degree through a US institution, but if its a very good program.
Anyway you asked me what I would do, obviously might not be the best advise (disclaimer).
[0] assuming doing anything related to law or your field of law doesn’t instantly fill you with dread or actively demotivate you in whatever you’re doing. If I hated law, the general outline would still be the same. Coming up with a project idea that kept me motivated and that I thought would be of use to others might take a little more time to find. I wouldn’t be able to use my law background for applying for work for maximum effect, so there would be no binding narrative between my previous lawyer life and what I’m doing now, that might make it more difficult to sell myself to potential employers. But with a portfolio that would matter less.
This raises an interesting question about MOOCs for me. Several people here are objected to violating the terms of a free online course. At what point is it ok to start creating derivative works? I've noticed many of the comments discuss etiquette or ethics "what's right is right", rather than legality. The whole question is interesting to me.
I actually do think that one of the drawbacks of Andrew Ng's course is that it uses octave rather than python. My point here isn't to kick off another debate about this topic - there have been plenty of those already. Rather, let's start with the assumption that it is reasonable for someone to feel that the course is brilliant, but would be improved by the option to do the assignments in python. As it is acceptable by the terms of the site to audit the class for free (no credit is given), many people have chosen to quietly do the exercise on their own in python.
At some point, they may want to get together to share, discuss, correct, and debate their solutions - and for these discussions to happen properly, this must involve sharing code. A lot of good could emerge from something like this.
I suppose you could say "then create your own course", but that comes with some issues of its own. First, the course is a classic - it was one of most famous early MOOCs, and it is a seminal course in machine learning generally. Under certain rules, it's ok to create derivative works of art from the public domain, and it's even ok to create derivative works of art from copyrighted material, provided you follow certain restrictions and abide by a (fairly complicated, inanal) set of royalty sharing regulations.
This is just a thesis, and I'm sorting it through mentally, so I don't want to come off as pushing any particular solution or angle here... but I am leaning toward the notion that 1) original content creators for online courses need to be credited, and in some cases paid, for their work, and 2) restrictions on creating derivative works from these courses may be very harmful, especially if they get to the point where simply re-implenting things in a different language and sharing the solutions (in ways that are essential for meaningful discussion) is disallowed legally or through etiquette or custom.
Well, that's about it, I'd be interested in hearing thoughts on this.
NOTE: I do want to be clear, this is a general question prompted by this discussion - the site in question is not a place where people post and share solutions to problems for a coursera course.
CNN == Convolutional Neural Network
That...is not a good approach.
Don't get me wrong - learning this stuff is hard, and implementing it (on your own - that is, a completely from-scratch not-using-any-libraries neural network) is even more non-trivial. You need to know and understand linear algebra for this; knowing a bit of calculus (derivatives mainly) can also help, but isn't absolutely necessary.
Trying to build it yourself, while admirable, will likely only lead you down paths already traveled and found wanting (if you are lucky). Once you understand the basics of neural networks and other machine-learning tools (gradient descent, state vector machines, etc) - and have implemented them yourself - you'll have a foundation on which to build with tools like TensorFlow, Keras, and others, without them feeling like such "black boxes".
Remember: Abstraction is a good thing. If you are doing any kind of programming that isn't hand-assembly or plugboards - you are using an abstraction to the machine. One thing I have learned in listening to people like Andrew Ng and Sebastian Thrun - is that even they rely and use those black-box abstractions, and don't have a "not-invented-here" attitude. There's nothing wrong with wanting to look under the hood for understanding, but don't let too much hubris get in the way of thinking that you can do it better without understanding what has come before.
You can take this advice or leave it - it's up to you of course; but if you want to have a better base on how all of this works, then my suggestion would be for you to gain an understanding first of linear algebra (just the basics - vectors, matrices, mathematical manipulation and calculation, etc) - Khan Academy or other resources can help there.
Then, go to Udacity, Coursera, and/or some of the other MOOC providers, and find a class or two on machine learning, and take them. Most will take (if seriously studying, and depending on existing work load) about 6-8 weeks - but you can usually go at your own pace (so shorter or longer learning terms are possible). I'm partial to Coursera's and Udacity's courses, mainly because in 2011 I took Andrew Ng's "ML Class" MOOC and Norvig/Thrun's "AI Class" - from both of which Coursera and Udacity (respectively) were founded.
Later, in 2012, I took Udacity's CS373 course (it was titled differently than today's offering, but I highly recommend it regardless). Today, I am enrolled in and progressing through the Udacity Self-Driving Car Engineer Nanodegree (so I am a bit biased, I suppose).
The tough thing about learning this stuff on the internet, without going thru a well-known MOOC or other vetted source, is knowing what you're learning is valid to learn. There is a ton of information out there purporting to teach you about neural networks or other "AI" tools - and they are actually teaching out-of-date technologies and techniques. But without having a grounding in the long history of machine learning (and AI) as a field, you wouldn't know this. It can be very confusing. There have been some good postings here on HN that have given some details on good resources, so you might want to check those out. There's nothing wrong with learning historical methods (in fact, I encourage it - I believe that one can't really understand a subject until they understand the paths that were taken in the past - whether they led to failures or to success for the period). But you don't want to learn something that is actually sub-optimal for the task.
Finally a couple of other points: Do you know what MNIST or ImageNet is? If not, then you need to learn about datasets and where to get them. Also, for a real CNN, look into the history of character recognition using the MNIST dataset - eventually, you will find a guy named "Yan LeCun" - who is one of the original people who led us down this whole CNN and Deep Learning path (along with plenty of other names).
There is so much out there (much more than I can summarize here) - you have the interest, I can see that. Take the time to properly learn this stuff, and you'll be rewarded with some interesting results.
Frankly as a grad student (The kind that the author apparently considers "dim witted"), the entire article is meaningless babbling without any underlying theme.
I wonder if the author truly understands "Machine Learning", what are his qualifications? A degree in Art History, and some "programming experience" aren't very assuring. E.g.
WTF?? The author should enroll in a beginner Machine Learning course on Udacity or Coursera before making philosophical statements about fields he has zero clue about.
It seems the only skill the author has is piecing together meaningless arguments that appeals to average HN users incapable of distinguishing between informed opinions and pseudo-scientific rants. Hell at least bad graduate students have to give examinations, read papers and make original contributions that get peer reviewed (otherwise they fail/get-kicked-out/drop-out). Not like this guy who does not understands difference between "supervised" and "unsupervised" machine learning, yet feels comfortable in making "prophetic" statements about machine learning.
Also
What does he means by "same generic approach". If we assume he is implying specific algorithms then we have a good "No free lunch" theorem that shows that a single algorithm is not effective across all domains. Now if by "generic approach" the author mean "machine learning" in general then its as ridiculous as saying
"Mathematics is effective, but the fact that the same Mathematical approach works across a wide range of domains should make you suspicious about how much insight it's adding."
The entire article is filled with "truthiness" and "feel-good" statements, which fall apart on closer examination.
"Once you realize that computing is all about constructing, manipulating, and reasoning about abstractions, it becomes clear that an important prerequisite for writing (good) computer programs is an ability to handle abstractions in a precise manner. Now that, as it happens, is something that we humans have been doing successfully for over three thousand years. We call it mathematics. This suggests that learning and doing mathematics might play an important role in educating future computer professionals. "
The conclusion is not good. The logical conclusion would be that if we define mathematics as 'handling abstractions in a precise manner' then programming is part of mathematics. In this case his conclusion becomes:
'learning and doing some classical parts of mathematics might play an important role in educating people who will do this another part of mathematics'
Which might be true to an extent, but this statement is very broad.
It is not a question that some amount of math is helpful. But after a while there are diminishing returns, because you simply learn different skills than what you need.
After having some very basic math knowledge the best strategy is to try to learn a lot of practical 'engineering' from very good books/courses, and only go back to math on-demand.
For example if you want to have some knowledge on machine learning or computer graphics, don't start to learn math for years first and then start learning about machine learning/3d graphics. Start to learn abut the engineering field first (take Andrew Ng's coursera class, or some equivalent in 3D graphics) and only go back to learn some linear algebra or very basic calculus after you see why it will be useful and to what degree you need it (you mostly don't need most of the theorems if you are learning the basics of these engineering fields).
Being an engineer is very different than being a mathematician. If you want to be a good engineer then mostly learn a lot from good engineers (or researchers in the engineering field) and learn some math along the way, but that will be secondary in my opinion. It's good to learn tons of math, but only if you learn even more engineering besides the math (to be a good engineer).
I will try to list resources in a linear fashion, in a way that one naturally adds onto the previous (in terms of knowledge)
[PREREQUISITES]
First things first, I assume you went to a highschool, so you don't have a need for a full pre-calculus course. This would assume you, at least intuitively, understand what a function is; you know what a polynomial is; what rational, imaginary, real and complex numbers are; you can solve any quadratic equation; you know the equation of a line (and of a circle) and you can find the point that intersects two lines; you know the perimiter, area and volume formulas for common geometrical shapes/bodies and you know trigonometry in a context of a triangle. Khan Academy website (or simple googling) is good to fill any gaps in this.
[BASICS]
You would obviously start with calculus. Jim Fowlers Calculus 1 is an excellent first start if you don't know anything about the topic. Calculus: Single Variable https://www.coursera.org/course/calcsing is the more advanced version which I would strongly suggest, as it requires very little prerequisites and goes into some deeper practical issues.
By far the best resource for Linear Algebra is the MIT course taught by Gilbert Strang http://ocw.mit.edu/courses/mathematics/18-06sc-linear-algebr... If you prefer to learn through programming, https://www.coursera.org/course/matrix might be better for you, though this is a somewhat lightweight course.
[SECOND STEP]
After this point you'd might want to review single variable calculus though a more analytical approach on MIT OCW http://ocw.mit.edu/courses/mathematics/18-01sc-single-variab... as well as take your venture into multivariable calculus http://ocw.mit.edu/courses/mathematics/18-02sc-multivariable...
Excellent book for single variable calculus (though in reality its a book in mathematical analysis) is Spivaks "Calculus" (depending on where you are, legally or illegally obtainable here http://libgen.org/ (as are the other books mentioned in this post)). A quick and dirty run through multivariable analysis is Spivaks "Calculus on Manifolds".
Another exellent book (that covers both single and multivar analysis) is Walter Rudins "Principles of Mathematical Analysis" (commonly referred to as "baby rudin" by mathematicians), though be warned, this is an advanced book. The author wont cradle you with superfluous explanations and you may encounter many examples of "magical math" (you are presented with a difficult problem and the solution is a clever idea that somebody magically pulled out of their ass in a strike of pure genius, making you feel like you would have never thought of it yourself and you should probably give up math forever. (Obviously don't, this is common in mathematics. Through time proofs get perfected until they reach a very elegant form, and are only presented that way, obscuring the decades/centuries of work that went into the making of that solution))
At this point you have all the necessery knowledge to start studying Differential Equations http://ocw.mit.edu/courses/mathematics/18-03sc-differential-...
Alternativelly you can go into Probability and Statistics https://www.coursera.org/course/biostats https://www.coursera.org/course/biostats2
[FURTHER MATH]
If you have gone through the above, you already have all the knowledge you need to study the areas you mentioned in your post. However, if you are interested in further mathematics you can go through the following:
The actual first principles of mathematics are prepositional and first order logic. It would, however, (imo) not be natural to start your study of maths with it. Good resource is https://www.coursera.org/course/intrologic and possibly https://class.stanford.edu/courses/Philosophy/LPL/2014/about
For Abstract algebra and Complex analysis (two separate subjects) you could go through Saylors courses http://www.saylor.org/majors/mathematics/ (sorry, I didn't study these in english).
You would also want to find some resource to study Galois theory which would be a nice bridge between algebra and number theory. For number theory I recommend the book by G. H. Hardy
At some point in life you'd also want to go through Partial Differential Equations, and perhaps Numerical Analysis. I guess check them out on Saylor http://www.saylor.org/majors/mathematics/
Topology by Munkres (its a book)
Rudin's Functional Analysis (this is the "big/adult rudin")
Hatcher's Algebraic Topology
[LIFE AFTER MATH]
It is, I guess, natural for mathematicians to branch out into:
[Computer/Data Science]
There are, literally, hundreds of courses on edX, Coursera and Udacity so take your pick. These are some of my favorites:
Artificial Intelligence https://www.edx.org/course/artificial-intelligence-uc-berkel...
Machine Learning https://www.coursera.org/course/ml
The 2+2 Princeton and Stanford Algorithms classes on Coursera
Discrete Optimization https://www.coursera.org/course/optimization
Convex Optimization https://itunes.apple.com/itunes-u/convex-optimization-ee364a... https://itunes.apple.com/us/course/convex-optimization-ii/id...
[Physics]
http://theoreticalminimum.com/courses
http://ocw.mit.edu/courses/physics/
I also live in the Bay Area (SF). Here's a small sampling of things I do when I'm not working:
* Write. Every weekday morning, I go to a coffee shop downtown and write for 30-60 minutes. I've written two full-length novels and two novellas so far, though they are all as-of-yet unpublished. Someday...
* Run. I run 3-4 nights per week, either along Embarcadero or through Golden Gate Park. I started off with the Couch to 5K program in January 2012 and I've worked up to four half-marathons in the last 12 months.
* Disc golf. There's a disc golf course in Golden Gate Park. Grab a beginner set (20 bucks from Amazon: http://www.amazon.com/Innova-Disc-Golf-3-Disc-Colors/dp/B000...), grab a friend, and go play. It's super fun.
* Nerd Nite. http://sf.nerdnite.com - 8 bucks. It's like the Discovery Channel -- with beer! (Side note: it's also where I met my girlfriend, and I go every month.)
* Rock climbing - check out Planet Granite (http://www.planetgranite.com/) or Mission Cliffs (http://www.touchstoneclimbing.com/mission-cliffs)
* Meetups - I've attended a dozen Meetups, including the SF Rust meetup. Very fun way to meet new people.
* Biking - Go to Roaring Mouse bikes in Crissy Field (http://www.roaringmousecycles.com/) and buy a bike. Then bike to work, bike to Golden Gate Park, bike across the GG bridge to Sausalito, to Tiburon, to Marin. There are a million awesome places to bike in the Bay Area, and there are huge communities around it. It's something you can do by yourself or with a group. Super fun.
* Anything from sf.funcheap.com. Like the site says, it's a list of fun, cheap things to do in the Bay Area.
* Take online classes (http://coursera.org). I'm currently taking the Machine Learning course and the Compilers course. It's very fun to learn new things!
* Become a Stormtroooper (http://www.goldengategarrison.com) - I'm not a member, but I know a lot of people who build their own Stormtrooper armor and then do volunteer activities with the 501st (the official Stormtrooper organization). Very cool way to give back to the community, especially if you're a Star Wars geek!
* Volunteer! There's a great organization in the city called One Brick (http://sfbay.onebrick.org/). You basically pick an event and just show up. It's volunteering a la carte! You're not making long term commitments -- just one event at a time. And you meet TONS of good, kind-hearted people. It's a win/win!
* And being Hacker News, I would be remiss if I didn't mention: start your own business. Find a problem domain you know something about, and solve that problem. Get out there, meet some co-founders, crank out some code, save the world, and get the cheerleader. Or maybe it was save the cheerleader? Whatever.
* And so much more. I do all of the above things regularly (sometimes daily, sometimes weekly, sometimes monthly), but there are a huge list of things I just don't have the time or memory to mention. There's a whole world out there. Get outside and explore!
Good luck!
Re: the original post IMHO, I find it has many logical flaws and while it is trying to draw attention to the fact that programmers need to learn math it may end up scaring people away.
Background: I have a BS in EE from IIT Bombay, An MS in Applied Math from USC and an almost PhD(everything but thesis) in Pure Math.
I also loved teaching math as a TA and was able to teach engineers multivariable calculus and differential equations so that they lost their fear of it.
But for the last 20 years have been in the database business. I am very enthusiastically returning to math in the form of data science in the last year.
I find that especially today it is necessary for math to be made accessible without having to prove theorems - that's for professional mathematicians.
Programmers use hardware often without knowing diddly-squat about semiconductors. And that's just fine if you don't want to or need to. We drive cars, use mobile phones and all sorts of machinery without needing to know anything about internal combustion or the CDMA algorithms or even how theory of relativity figures into GPS.
I think the insistence on setting a high bar for people to learn/use/apply math is unnecessary mystification and obscurantism.
Math is beautiful, useful and powerful. If you know math you should be wanting to simplify it's teaching and instruction, thinking hard about making it useful for the non-mathematician and also making it easier to access in a breadth-first fashion rather than the depth-first fashion in which it is currently practiced.
I know many people, engineers and mathematicians who believe that simplification is "dumbing down". I disagree vehemently.
Today if you are a programmer and want to learn math I have a few of suggestions
a) If you want to apply it in engineering take a couple of classes on Coursera/Udacity - it will be a slow an steady way to re-activate your math neurons (it's also insurance against Alzheimers to keep learning new and hard stuff).
b) Learn some statistics, learn to use R. (Coursera again)
c) Learn some linear algebra, minimal, then learn MATLAB (Andrew Ng's Machine Learning class on Coursera or a similar one by AbuMustaffa (sp?) of Caltech)
d) If you absolutely don't want to learn a new language then look at Machine Learning by Marsden which uses Python and then pick up NumPy/SciPy and look up the exciting stuff that's happening with the IPython Notebook.
e) If your focus is on pure CS the take the Algorithms track(s) by Roughgarden (Stanford) or Sedgewick (Princeton) on Coursera
This way you dont have the notation issues - R has a syntax, MATLAB has a syntax just learn it. Python hopefully is note even an issue.
Start using your math brain a little at a time - absolutely no proofs required - and THEN once you see the beauty and want to learn more and want to learn why, THEN dig deeper with the fundamentals. If you don't that's fine too - use it, put its power to use and become a better developer and engineer. Math is not trivial but it's not as hard as it's made out to be. There's way too much noise in the channel.
Good luck.
(feel free connect with me on twitter if you want to get more help (@nitin))
Contrary to the other advice around here, I would strongly advise NOT taking a course. I think it is a good idea at some point, but it is not the first thing you should be doing.
The very first thing you should do is play! Identify a dataset you are interested in and get the entire machine learning pipeline up and running for it. Here's how I would go about it.
1) Get Jupyter up and running. You don't really need to do much to set it up. Just grab a Docker image.
2) Choose a dataset.
I wouldn't collect my own data first thing. I would just choose something that's already out there. You don't want to be bogged down by having to wrangle data into the format you need while learning NumPy and Pandas at the same time. You can find some interesting datasets here:
http://scikit-learn.org/stable/datasets/
And don't go with a neural net first thing, even though it is currently in vogue. It requires a lot of tuning before it actually works. Go with a gradient-boosted tree. It works well enough out of the box.
3) Write a classifier for it. Set up the entire supervised machine learning pipeline. Become familiar with feature extraction, feature importance, feature selection, dimensionality reduction, model selection, hyperparameter tuning using grid search, cross-validation, ....
For this step, let scikit-learn be your guide. It has terrific tutorials, and the documentation is a better educational resource than beginning coursework.
http://scikit-learn.org/stable/tutorial/
4) Now you've built out the supervised machine learning pipeline all the way through! At this point, you should just play:
4a) Experiment with different models: Bayes' nets, random forests, ensembling, hidden Markov models, and even unsupervised learning models such as Guassian mixture models and clustering. The scikit-learn documentation is your guide.
4b) Let your emerging skills loose on several datasets. Experiment with audio and image data so you can learn about a variety of different features, such as spectrograms and MFCCs. Collect your own data!
4c) Along the way, become familiar with the SciPy stack, in particular, NumPy, Pandas, SciPy itself, and Matplotlib.
5) Once you've gained a bit of confidence, look into convolutional and recurrent neural nets. Don't reach for TensorFlow. Use Keras instead. It is an abstraction layer that makes things a bit easier, and you can actually swap out Tensorflow for Theano.
6) Once you feel that you're ready to learn more of the theory, then go ahead and take coursework, such Andrew Ng's course on Coursera. Once you've gone through that course, you can go through the course as it actually has been offered at Stanford here (it's more rigorous and more difficult):
https://see.stanford.edu/Course/CS229
I will also throw in an endorsement for Cal's introductory AI course, which I think is of exceptionally high quality. A great deal of care was put into preparing it.
http://ai.berkeley.edu/home.html
There are other good resources that are more applied, such as:
http://machinelearningmastery.com/
I hope this helps. What I am trying to impart is that you will understand and retain coursework material better if you've already got experience, or better yet, projects in progress that are related to your coursework. You don't need to undergo the extensive preparation that is being proposed elsewhere before you can start PLAYING.
I've taken both, and the code is in fact not that much simpler than it was in the original class. There are, however, two huge differences: the algorithm is spoon-fed to you, and there is no math.
Firstly, think about how much more difficult the assignments would be if, for example, the steps weren't broken out and we didn't get any advice on how to vectorize. Of course, it would still be short work for anyone who (a) knows Matlab/Octave and/or (b) understands the material well, but it would also be an order of magnitude harder.
Secondly - and this is by far the larger point - the original CS 229 was really about math; the programming assignments were more of an afterthought. The lectures and homework mainly focused on the theoretical derivations and corollaries of the math that led to the algorithms. Once you'd done your bit on the math and cried to your classmates and the TA about it, you could go and implement the beautiful and extremely succinct result in Matlab.
As for my perspective on the difference, I believe it is a deliberate choice made with full knowledge of the difficulty drop. For starters, there are (with regards to homework help) no TAs in this course, so the absolute difficulty would have to decline to create an equivalent experience. More significantly, the enrollment has increased by a factor of about 700. If Stanford students had trouble with the original, you can bet that the median student in the course doesn't find it as easy as either of us does. If the goal is to generate the greatest benefit for the most people, and delivering the algorithms with a good intuition on their proper use will do so, then this course has succeeded marvelously. Of course, the smartest and most dedicated students will want more, which remains available through textbooks as well as the original course handouts (http://cs229.stanford.edu/materials.html). However, I would argue that the goal of most MOOCs (massive open online courses) should be to kindle interest and foster basic understanding, both of which the Coursera version achieves.