#29 overallEvergreen
Probabilistic Graphical Models
Coursera
15 recommenders · 15 threads · recommended in 9 years, 2012–2025 · score 19.1
An advanced Coursera course by Daphne Koller on probabilistic graphical models, covering Bayesian networks, Markov networks, inference, and learning. It is based on the Koller and Friedman textbook and is noted for demanding homework.
Commenters call it a very good, rigorous deep dive into graphical models, with several saying it is worth the effort despite rough edges in the first run.
Machine Learning & AIadvancedfreevideo lectures#probabilistic graphical models#bayesian networks#markov networks#inference#koller
Recommendations by year
20062026
I used to take a lot of courses on cousera and EdX. I still take some here and there, but not as many as before. Some of the courses are amazing and unbelievably rewarding, like Daphne Koller's Probabilistic Graphical Models, Robert Sedgewick's Analytical Combinatorics, and Gerald Sussman's course on system optimizations. I'm very grateful for such learning opportunities. Unfortunately over time, I also found that these courses had diminishing returns for the following reasons:
- Due to the nature of MOOC, the assignments are largely either multiple-choice questions or programming assignments that merely asked students to fill in some blanks in some functions (there are a few exceptions of course). What a descent US university does really well is challenging students with tough yet insightful and inspiring assignments. That's how students learn deeply and retain the knowledge, at least for me. Merely listening to lectures and ticking off a few ABCDs hardly helps real learning.
- Lack of feedback. A university course assigns TAs, gives tutorials and office hours, grades assignments with detailed feedbacks, and it is so much easier to form study groups and have high bandwidth discussions. MOOCs try their best to offer such help, but they don't work as well or at least not as conveniently.
- Many courses are watered down. For instance, Andrew Ng's ML course on Coursera is far less rigorous than that (229? I forgot) offered in Stanford? The course is great for students to gain some intuition, but I'm not sure if it's good enough for one to build solid ML foundations.
The Georgia Tech online masters?
Depending on what stats you want to do, there are some pretty decent MOOCs. No one is going to claim that Daphne Koller's PGM course is weak in anyway for example[1].
[1] https://www.coursera.org/learn/probabilistic-graphical-model...
There's an excellent course on PGM by Koller on Coursera. My friend took it and now he's a PGM evangelist. If you are wondering where PGM lies in the spectrum of machine learning, you should research the difference between generative and discriminate modeling. We have been driven to PGM to solve our ML problem that was hard to frame as A NN. Mainly because we had some priors we needed to encode to make the problem tractable. It reminds me a little of heuristics in search.
The person I'm talking to: an early ML student.
PGMs are a great topic to explore (and Koller+Friedman is a great book), but, as a word of caution to anyone interested: implementation of any of these more advanced models remains a major challenge. For anyone building production facing models, even if your problem is a pretty good match for the more interesting PGMs, the engineering requirements alone are a good reason not to go too far down that path.
The PGM book is also structured very clearly for researchers in PGMs. The book is laid out in 3 major section: the models, inference techniques (the bulk of the book), and learning. Which means, if you follow the logic of the book, you basically have to work through 1000+ pages of content before you can actually start running even toy versions of these models. If you do need to get into the nitty-gritty of particular inference algorithms, I don't believe there is another textbook with nearly the level of scope and detail.
Bishop's section on PGMs from Pattern Recognition and Machine Learning is probably a better place to start learning about these more advanced models, and if you become very interested then Koller+Friedman will be an invaluable text.
It's worth noting that the PGM course taught by Koller was one of the initial, and still very excellent, Coursera courses. I'm not sure if it's still free, but it was a nice way to get a deep dive into the topic in a reasonably short time frame (I do remember those homeworks as brutal though!)[0].
0. https://www.coursera.org/specializations/probabilistic-graph...
But I just got an email from her as a registered coursera user that she will soon rerun the Propabilistic Graphical Models course which is now split to three separate courses. I hope that her new position won't change that, her course is very good.
I disagree re: ESL and RMS, but Harrell's book is superb. It's really more aimed at biostatisticians and clinicians, though.
If the math in ESL gives you trouble, you might prefer http://www-bcf.usc.edu/~gareth/ISL/ISLR%20First%20Printing.p... (and I'm not just saying this because the first author was one of my advisers, although I do think that he and Daniela are particularly gifted teachers).
If the math in ESL is too trivial for you, there's always https://web.stanford.edu/~hastie/StatLearnSparsity_files/SLS... , which covers some graphical modeling strategies in later chapters and even kicks the tires of the autoencoder (imho perhaps the greatest recent advance in neural networks for practitioners) along the way.
Koller's course and Ng's course are also good.
Ultimately I feel like you have to get the math right or you'll never acquire the intuition that helps you design your own approaches. But you also have to put in the work.
That reminds me, tibshirani's Stanford course (accompanies ISL and ESL) is terrific. Better than those other two, actually. I wish Harrell would offer one.
Do what the author did: take Koller's course and follow the research that interests you. In this case, the propagation through factor graphs is also a good analogy for when you start looking at back propagation in NNs and the autoencoder.
Thanks for this. Incidentally, the second paper you link to is co-auhored, among others, by Daphne Koller, who teaches [this great course on probabilistic graphical models](https://www.coursera.org/course/pgm) and Andrew Ng, who teaches [the best-known intro MOOC in machine learning](https://www.coursera.org/learn/machine-learning/home/welcome).
I love textbooks and spend more of my childrens' inheritance on them than I should.
But what MOOCs give me is the exercises. I often think I understand a problem, but it's only after getting 2.1/10 on a Coursera quiz that I realize I've missed a key step or concept.
Many textbooks have exercises but few have solutions. I've been working through Barto and Sutton's Reinforcement Learning for example (again), and although I do the exercises and programming questions, I never know if I've gotten it right. My experience with MOOCs shows I probably haven't in a large number of cases.
The best of both worlds is when I can follow a MOOC with the textbook to gain more depth, for example with the PGM course on Coursera.
Sure. Have you looked at the PGM class on coursera for CRFs? The slides are a great condensed version of koller's book on PGMs. NLP is a really easy context they are used in. With that, I would recommend the stanford NLP coursera class (the first one from a few years back) that covers viterbi, CRFS, and general sequential models.I know I'm recommending a lot of MOOC content here, but I would add here that I think they provide a great overview from which to break in to the papers/other literature on the topic.
For SVMs, outside of the typical research you'd do on wikipedia, I unfortunately haven't had much specific experience with SVMs. I used them quite a bit a few years back for relation extraction and other algorithms, but I'm mainly from an NLP background in that context.
Great explanation. I would like to add to this, that held-out data is often used in Bayesian learning too - for example, in cases when you intentionally over-specify the model (adding more parameters than might be needed) because you don't really know what the best model might be. The inference goes until the likelihood on held-out data keeps increasing. Example, gesture recognition in Kinekt. If someone finds this info useful, I also recommend Coursera course on Probabilistic Graphical Models.
Also, if you're interested in doing this computationally, you should check out Probabilistic Graphical Models. There's a great Coursera course.
It has much more than MCMC, but the "Probabilistic Graphical Models" textbook by Daphne Koller (of Coursera) and Nir Friedman is a thorough introduction to the subject. It includes a discussion of MCMC that will leave you with a deeper understanding than the typical shallow treatment.
As an alternative, you can try watching about 90 minutes of lecture starting here:
https://class.coursera.org/pgm-003/lecture/71
but it will drop you right into the fray without much context.
Here's some free book lists
http://www.kaggle.com/wiki/Tutorials
http://metaoptimize.com/qa/questions/186/good-freely-availab...
I think the best single resource is Kevin Murphy ML text, but there's lots of relevant stuff on linear algebra, Bayesian analysis etc
______________
Here's a "curriculum"
http://blog.zipfianacademy.com/post/46864003608/a-practical-...
And look at the coursera courses by Koller, Andrew Ng, NLP by Collins etc
I would also recommend the Coursera course on Machine Learning by Andrew Ng and Probabilistic Graphical Models by Daphne Koller. I would also go over some basics probability and statistics review. Maybe some linear algebra too. Python is a great language to do data analysis in. I recommend the scikit-learn and pandas packages and using ipython notebooks. Another book is the Elements of Statistical Learning (http://www-stat.stanford.edu/~tibs/ElemStatLearn/). There are also Kaggle contests for testing your chops.
But online courses actually give you more options as far as personalizing education. At most universities, unless they're very large or very specialized, you're likely to have one professor from a specific subject in a field. With online lectures, even from a reasonably small number of sources you can mix and match. Two personal examples I can think of is supplementing my algorithms class with MIT's algorithms class on OCW, and watching Tom Mitchell's lecture on kernel functions to add depth to the topic as covered in Ng's Ml-class.
Even at great universities there are professors who are better researchers than teaches that still may be a students only option. If you took the top 10, or even 5 teachers on a given subject you could give students the freedom to mix and match as suited their learning style.
Personally I actually find the lectures a poor substitute for a text. I find the Probabilistic Graphical Models textbook a great addition to the pgm class on coursera. Even though the lectures for pgm-class are long and detail, there's still only so much information you can get into a few hours. The information density of a textbook is hard to beat.
Most fun: Pat Pattison, Songwriting, Coursera. Very good lectures, very good material, very well presented. Teaches a lot about writing song lyrics in just 6 weeks, breaks it nicely down to steps and recipes. I used to think that the best feature of MOOCs is the automatic grading and feedback from programming homework, but in this course, for the homework songwriting you gave and got feedback from 3-5 random people in the course, and it was not only useful but this feeling of togetherness with strangers was even better than getting instantaneous feedback from a bot for programming homework. Shows that teaching art scales to MOOCs as well.
Nicest: Andrew Ng, Machine Learning, Coursera. Interesting topic, well-planned material, very well avoids going into the mathy details, but still conveys a feeling of understanding of the topic, so accessible to a wide audience. (Martin Odersky, Functional Programming Principles in Scala, Coursera, was almost equally nice, but had some rough edges in the first run.)
Most interesting: Probabilistic Graphical Models, Daphne Koller, Coursera. Very interesting topic. I took the first run of the course and it had lots of rough edges. Needs a lot of work to apply the lectures to the homework. I haven't seen such a demanding course since I took quantum mechanics at university.
Best organized: Jennifer Widom, Databases, Stanford. This is not the flashiest of a topic, but oh boy was it well organized. Runs like a clockwork. Everything in the lectures is relevant, everything from the lectures is applied and tested in the homework, there is lots of homework (but still not enough to make you remember SQL,XPath,XQuery,XSLT for the rest of your life if you don't keep using them), weekly homework has a nice progression from simpler things to medium difficult things, and the web environment is well designed, and gives wonderful feedback and guides you to get your queries correct.