#93 overall
Learning from Data
edX
4 recommenders · 4 threads · recommended in 4 years, 2015–2023 · score 4.8
A Caltech machine learning course by Yaser Abu-Mostafa covering the theory and principles behind learning from data, including generalization, overfitting, and the bias-variance tradeoff. It is offered on EdX and also on the instructor's own site, and is based on a real Caltech undergraduate course.
Commenters call it far and away the best MOOC they have taken, praising its challenging, mathematically grounded approach over library-API tutorials.
Machine Learning & AIintermediatefreevideo lectures#machine learning#theory#generalization#caltech#abu-mostafa
Recommendations by year
20062026
It's not on Coursera, but Caltech's Yaser Abu-Mostafa offers "Learning from Data" on his own website https://work.caltech.edu/telecourse.html and it's intermittently on EdX https://www.edx.org/course/learning-from-data-introductory-m...
This is far and away the best MOOC I've ever taken. The class is genuinely challenging. It's a real Caltech undergraduate course, and you can't get away with copy-pasting code or just keep resubmitting until you pass the grader. The course is focused on real understanding of what's going on mathematically, not just learning to use some library API.
I would recommend going through the CalTech Course - Learning from Data run by Yasir Abu Mustafa as the first step. https://www.edx.org/course/learning-data-caltechx-cs1156x#.V...
It is one of the best places to start. Please see that the course will require you to spend considerable time. If you find this challenging, you can also look at the Machine Learning class by Andrew Ng on Coursera.
Once you have undergone these courses, you can take up Coursera course on Neural Nets or look at tutorials on deeplearning.net
This is off the beaten path, but consider Abu-Mostafa et al.'s "Learning from Data". https://www.amazon.com/Learning-Data-Yaser-S-Abu-Mostafa/dp/...
I adore PRML, but the scope and depth is overwhelming. LfD encapsulates a number of really core principles in a simple text. The companion course is outstanding and available on EdX.
The tradeoff is that LfD doesn't cover a lot of breath in terms of looking at specific algorithms, but your other texts will do a better job there.
My second recommendation is to read the documentation for Scikit.Learn. It's amazingly instructive and a practical guide to doing ML in practice.
edX has quite a few good and very good courses on the subject. For example, right now "MITx: 6.008.1x Computational Probability and Inference" is being offered (other courses are available in archived mode), as well as Caltech's "Learning from Data" (theory heavy) course. There also is a lot on Coursera, but the more theoretical university courses on probability theory are on edX. Everything is free - if you ignore the certificates (if you really need to prove you took a course take a screenshot of the progress page).
"closures were accomplished with anonymous objects until version 8!"
Java has had closures through inner classes since the beginning, albeit admittedly in a verbose way. This is a quote from page 540 of "Concepts, Techniques and Models of Computer Programming": Full lexical scoping. Full lexical scoping means that the language supports procedure values with external references. This allows a class to be defined inside the scope of a procedure or another class. Both Smalltalk-80 and Java support procedure values (with some restrictions). In Java they are instances of inner classes (i.e., nested classes). They are quite verbose due to the class syntax (see the epigraph at the beginning of this section).
But I think this is already enough to do high order programming, given that it effectively allows having functions as values (and hence high order functions), and support all 4 basic techniques high order programming builds upon: procedural abstraction (which relies on closures), genericity, instantiation and embedding [1].
I have to admit that I am not sure if I understand what you mean by "There is simply no way to infer where a value was set without tracing the entire code, which slows development time by a factor of 10 to 100 over more functional techniques that use methods to derive state instead of values". My (possibly wrong) interpretation is that you are referring to lazy execution, given that this technique associates a variable with a function that creates its value but whose execution is delayed until triggered when the value is actually needed.
Is this correct? If it is, then explicit lazy execution can certainly be accomplished in Java, given that, as I have said before, it allows embedding procedure values in a data structure. Of course, this is not as convenient as implicit lazy execution as in Haskell with its non strict evaluation, but it achieves the main purpose of laziness all the same, which is to optimize resources by allowing consumers rather than producers to drive value creation and hence generate data structures incrementally (as they are needed) rather than eagerly (as in simpler data driven techniques): stream programming; another advantage of laziness is of course purely functional persistent data structures in some cases with good performance ([3] and [4]).
"It would also be nice to have the numerical computing techniques from MATLAB"
I couldn't agree more with this. For instance, if possible, for a ML project I think it's better to do as much research/experiments/development as possible in Matlab/Octave, and then translate it to Python or whatever the production language is (would love to hear opinions on this). It would be so nice if such translation could be avoided (maybe a preprocessor could automate it, but still ...).
I believe that the reason is that Matlab is just a much more natural language for mathematical programming. I did the wonderful mooc [2] in Matlab, and much prefered it compared to the Python solutions I saw.
[1] https://ocw.mit.edu/courses/electrical-engineering-and-compu...
[2] https://www.edx.org/course/learning-data-introductory-machin...
[3] http://lambda-the-ultimate.org/node/2665
[4] https://www.cs.cmu.edu/~rwh/theses/okasaki.pdf