#24 overallClassic
6.824 Distributed Systems
MIT OpenCourseWare
16 recommenders · 15 threads · recommended in 9 years, 2013–2025 · score 25.0
MIT's graduate-level distributed systems course, with published lecture notes, assigned readings of classic papers, and Go labs that build pieces such as MapReduce, Raft consensus, and a sharded key-value store. Lectures and materials are openly available on MIT OpenCourseWare and YouTube.
Commenters call it an excellent, hands-on introduction to distributed systems, valuable because the labs force you to implement and debug real designs.
CS Fundamentals & Systemsadvancedfreemixed#distributed systems#raft#consensus#mapreduce#golang#robert morris
Recommendations by year
20062026
Prof. Robert Morris' distributed systems course is amazing, wish I had some recordings.
https://pdos.csail.mit.edu/6.824/
Most of the real world distributed systems would have some videos in some conferences: Spanner: https://www.youtube.com/watch?v=IfsTINNCooY
Chubby: https://www.youtube.com/watch?v=PqItueBaiRg
Anything Leslie Lamport: https://www.youtube.com/watch?v=-4Yp3j_jk8Q
https://www.youtube.com/watch?v=kYX6UrY_ooA
Also the Distributed system course at MIT is fantastic. The labs are also great way to get your hands dirty with distributed system. The link to the class schedule is here: https://pdos.csail.mit.edu/6.824/schedule.html
This is not quite what you asked for, but I've been going through this distributed systems course and I highly recommend it: http://nil.csail.mit.edu/6.824/2017/schedule.html
It's mostly a collection of papers on different distributed protocols and systems, with good lecture notes and FAQs for each one. It also has programming assignments if you like hands-on practice.
For me this shows the difference between theoretical setting and what you would want to do in practice. I have been following 6.824 (where this is sourced from), to learn something about distributed systems programming and it was great fun to shed a lot of figurative sweat to convert those 26 (actually) lines into working "production" code. Hundreds lines of code, because in real-life we have packet loss, network partitions, etc. But the pseudo-code in the link itself is correct, however, it doesn't tell the whole story.
Now I am repeating that experience, as Akka project contributor ( http://akka.io/news/2017/03/17/akka-2.5.0-RC1-released.html ) on getting delta-CRDTs into Akka. And again - what was a few lines of pseudo-code in the original paper, or even tens of lines of real code but in some ideal setting ( https://github.com/CBaquero/delta-enabled-crdts ) is becoming literally thousands lines of "production grade" code.
Finally - I wholeheartedly recommend the 6.824 course to anyone interested in distributed systems. Even if you don't like strong consistency, you'll learn a lot about testing and debugging distributed systems, the knowledge you can re-use later in your career.
I've recently started going through the Raft lab for 6.824 (and still working on it), and I /highly/ recommend doing the labs in this course (I am not an MIT student, nor in college). This course is fairly practical in nature, and will expose you to the real-world aspects of distributed systems!
Even the first assignment, which is just implementing aspects of mapreduce, will make you exercise some nice concurrency concepts. They have tests and everything to check that you have a working implementation!
At the very least, checkout the recommended readings!
Distributed Systems: Principles and Paradigms by Tannenbaum is a classic textbook. It's considered pretty dry though.
I recommend strongly MIT's Distributed Systems course, both for the lectures and the labs. See http://pdos.csail.mit.edu/6.824
As a bonus, the labs (in Go) give you invaluable hands-on knowledge.
The MIT course for Raft (with the lectures on YouTube) was great to learn for me: http://nil.lcs.mit.edu/6.824/2020/labs/lab-raft.html
I’d recommend checking out MIT’s Distributed Systems course. All its videos and assignments are available online and teach you everything you would need to get into these systems and go in depth in them.
If your goal is distributed systems then take it, lectures are all on YouTube https://pdos.csail.mit.edu/6.824/schedule.html
This is site lays out a good plan https://teachyourselfcs.com/
Courses on there can be substituted for equivalents from MIT, eg. https://pdos.csail.mit.edu/6.828/2020/ for the Berkeley OS course. I just worked through that MIT course and the 6.824 Distributed systems (https://pdos.csail.mit.edu/6.824/schedule.html), also listed on that site [[personally, I recommend both of the forementioned classes taught by Robert Morris (also a cofounder of ycombinator)]] Both courses were fully available (lectures,assignments,projects) from the current course sites.
The MIT sites hosted there are incredibly open in general - https://courses.csail.mit.edu/, absolutely amazing institution.
MIT online trainig has this great YouTube playlist on distributed systems, with one of the lectures being on RAFT [1,2,3]. The entire playlist [4] is probably worth your time, though.
[1] Intro https://www.youtube.com/watch?v=UzzcUS2OHqo
[2] Part I: https://www.youtube.com/watch?v=64Zp3tzNbpE
[3] Part II https://www.youtube.com/watch?v=4r8Mz3MMivY
[4] Playlist for the course: https://www.youtube.com/watch?v=cQP8WApzIQQ&list=PLrw6a1wE39...
Stanford is a treasure trove of advanced courses in ML/AI.
Reddit has also this pretty great list of advanced courses in ML: https://www.reddit.com/r/MachineLearning/comments/fdw0ax/d_a...
There is this from MIT (Graph Analysis) https://people.csail.mit.edu/jshun/6886-s18/
I like the distributed systems class too (The assignments are in Golang) : https://pdos.csail.mit.edu/6.824/
My "loose" definition of advanced: "If you follow the class and do the work you will be able to implement something not trivial"
I really agree with this. I didn't take the distributed systems course when I was at university, but later I still managed to work at a few FAANGs. Flicking through the material in the MIT 6.824 course, I really wish I learnt this before I needed it, rather than scrambling to learn it piecemeal as things came up. I would've been promoted faster, and things would've been less stressful.
It is possible to learn this on the job. But with hindsight, my advice would be to just take the easy way out and learn it beforehand.
I just finished the distributed systems section of teachyourselfcs and I now have a solid understanding of a lot of concepts I previously only shallowly understood.
When I started learning, the recommended textbook was Distributed systems: Principles and Paradigms (the new recommendation is Designing Data-Intensive Applications) and the recommended course was MIT 6.824.
6.824 is the first online course that I have completed fully and it was well worth it. I read, took notes, and summarized all required course readings (around 20 papers) and completed the labs, which involved creating a raft-based key-value store. The labs were especially useful because they forced me to really understand the details of the topics I had learned (e.g., MapReduce, Raft, Raft’s log compaction) in order for my code to pass all the tests.
I’m very happy with the results of following the course and I now plan to put the same amount of effort into other topics.
Beside Martin Kleppmann's wonderful book, I'd also recommend MIT 6.824 distributed system course, you can find lots of resources on the course website.
Here some ideas that worked for me:
- online classes. Especially those from actual universities rather than Coursera. Coursera is great but often they target a larger audience and the content may be simplified. Personally, I learned a lot from MIT classes (especially distributed systems and operating systems). They have projects that are challenging and really interesting.
- Competitive programming. I found competition in general to be a great way to improve. It's stimulating and there are many resources available. The downside is that it's hard and can be depressing. For instance, it takes me 10 hours to solve the problems in an easy code jam round when some 18 year kids solve them in an hour. Depending on what you want to learn, there are different type of competition. For instance, code jam / codeforces for shorter type of problems. Google hashcode for slightly larger optimization problems. Kaggle for machine learning. I'm sure there are many others.
Actually, learning stuff and improving is the easy part (you just need free time). There are so many resources. The question is more deciding what to learn, and how can it be useful.
This looks great. However, without a truly comprehensive test suite it’s guaranteed to have a great many subtle bugs :). I recommend trying to hook it up with the tests from MIT’s distributed systems course: https://pdos.csail.mit.edu/6.824/labs/lab-raft.html.
It looks as though it would take only a minor refactoring at least for the leader election and log replication.
Cool post. Wonder if it would have helped to take a look at the MIT distributed systems course on the web and YouTube - one of the projects is exactly this (Go Raft implementation)
https://pdos.csail.mit.edu/6.824/labs/lab-shard.html
Check out MIT 6.824 which contains lecture notes and coding labs on Raft. (They used to teach Paxos but phased out a few years ago)
If you already know partial derivatives, gradient descent algorithms, and some linear algebra, it's easy. The assignments are not challenging either, even compared to other coursera classes. But for people that don't have this background, it can be hard.
I think the problem with Coursera is that they try to accommodate as many students as possible, which means the classes aren't very deep. I don't think this should be the role of a ML class to teach about partial derivatives or matrix factorization for instance. It's great that the class is accessible to many people, but sometimes you feel that you just scratch the surface, and that you're not at the level of university students that went through more challenging classes.
As a comparison, I did the labs of 6.828 and 6.824 (OS and distributed systems) on MIT opencourseware and it was much harder and much more rewarding than any coursera class. And I did only the labs, there is much more material to cover to do it seriously (quizzes, lot of articles to read...). Kudos to MIT students! (I wonder how many such classes they take during one semester). Too bad opencourseware classes aren't always properly set up for outside students. There may be some material missing, no forum to ask questions, video quality can be very bad and so on...
https://pdos.csail.mit.edu/6.828/2014/schedule.html The mit website pretty much has all the information open, which you can use to go through this course at your own pace. The lectures(most if not all) are on youtube, search for 6.824
There's a cloud computing class[0] launching on Coursera taught by Prof. Indranil Gupta which will cover all concepts such as fault tolerance, consensus etc. I'd suggest you can give that a shot. If you can't wait, you can checkout the course page[1] and get started right away.
You can take a look at other DS courses here - https://github.com/prakhar1989/awesome-courses/#systems
Lastly, you also have the option of working on assignments in MIT's DS class[2] (git clone git://g.csail.mit.edu/6.824-golabs-2014 6.824)
[0] - https://www.coursera.org/course/cloudcomputing
[1] - https://courses.engr.illinois.edu/cs425/
[2] - http://css.csail.mit.edu/6.824/2014/index.html