#14 overallClassic
6.006 Introduction to Algorithms
MIT OpenCourseWare
27 recommenders · 29 threads · recommended in 14 years, 2009–2024 · score 45.2
MIT's undergraduate introduction to algorithms and data structures, covering asymptotic analysis, sorting, hashing, graph search, and dynamic programming, with lecture videos and materials on MIT OpenCourseWare. It follows the CLRS textbook.
Commenters call it a good intro to data structures and algorithms that helped with complexity intuition and technical interviews, and say it pairs well with CLRS.
Algorithms & Data Structuresintermediatefreevideo lectures#algorithms#data structures#complexity#clrs#mit ocw
Recommendations by year
20062026
This.
That was my journey as a self-taught programmer who at one point realized had to learn the timeless foundations.
And then MIT's 6.005 [1], where you will apply all this with a realistic language (Java, although the concepts carry to any language), and learn how to design, code and test programs that "have no bugs, are easy to understand and ready for change".
And also learn a bit about algorithms, I don't think that one can design and understand properly without. And what is O(2^n) today will still be that in 100 years. MIT's 6.006 is amazing, both professor and TA [2].
I've seen Knuth's TAOCP recommended around here. Don't even consider that, do a course like 6.006 first, the Everest shouldn't be the first mountain you climb. Likewise favour HtDP over SICP at first, I've been there.
If after all this you are still interested in LLMs, I recommend EdX's "Large Language Models: Application through Production" [3]
[1] https://ocw.mit.edu/courses/6-005-software-construction-spri...
[2] https://ocw.mit.edu/courses/6-006-introduction-to-algorithms...
[3] https://www.edx.org/course/large-language-models-application...
I highly recommend MIT's 6.006 for a good intro to data structures and algorithms. My college's course wasn't great, but watching the MIT lectures and working through CLRS helped me gain a really solid intuitive understanding of complexity and approaches to different classes of problems. At the very least it helped a lot with interviews!
My experience with technical questions during interviews is that they did not need a deep understanding of any particular technology. Instead, they mostly rely on some knowledge of algorithms and data structures. Almost all of my coding questions came from the following classes I took at MIT: Intro to Algorithms (6.006) Design and Analysis of Algorithms (6.046) Mathematics for Computer Science (6.042)
Take some version of those classes (maybe OCW, edX, or Coursera) and you will be good to go — at least for technical interviews. If you like reading then Introduction to Algorithms by Cormen, Leiserson, Rivest, and Stein is the book that these courses cover.
Data structures and algorithms. I don't have a formal education either but after 25 years of programming, IMO everything boils down to data structures and algorithms - the former more so than the latter. I think everything else comes from experience gained from on the job training because you need to train the pattern matching part of your brain. A good foundation in data structures and algorithms simplifies that process a lot.
Data structures are how programmers describe reality in the Wittgenstein/Sapir-Whorf sense and they shape our understanding as much as we use them to encode it. The right data structures drastically simplify problems and the wrong data structures turn them into nightmares of bugs and spaghetti code. Database schemas are the natural extension of this at the application, rather than module, level.
I don't have a specific recommendation because data structures is such a broad field, but an MIT OCW data structures and algorithms course is a good place to start. Oh and the book Designing Data Intensive Applications for a higher level overview of how data structures and algorithms are used in the field.
Also, check out all the free MIT OpenCourseWare CS classes. I listened/watched through a bunch of them a few times and then did some online exercises (leetcode, advent of code, hackerrank, et al) and it really helped me understand algorithms better and I also became a much better programmer for it.
I'd also suggest going through Network+ and Security+ study guides too. IDK if the certs are worth it because I haven't actually taken the tests, but the books alone taught me a ton of the important concepts at a high level.
Erik's lectures on youtube are incredible and I've learned so much about Data Structures and Algorithms from him. Highly recommend if you are in the market for that knowledge.
Based on the description you give, you probably want a Data Structures/Algorithms intro.
The canonical Algorithms textbook is Introduction to Algorithms by Cormen et al. The MIT OpenCourseware course on Algorithms -- which includes videos and assignments -- follows along with that book: https://ocw.mit.edu/courses/electrical-engineering-and-compu...
Note: In your case, I'll recommend against SICP as a first resource, which is mentioned in some other comments. It's more of an introduction to functional programming. From your description, I'm not sure you need an intro to programming. And functional programming in particular is not really on the critical path of knowledge you'll need to muck about in VR land. It's not irrelevant, but it's not the critical path. An algorithms course probably is indispensable for that endeavor, though.
By all means, read SICP if you'd like. It's a great resource. But if you're trying to get from "I can program" to "hardcore hacking on VR stuff", step 1 is probably a decent algorithms course taught in the imperative style.
I think in your case there is an incredibly specific and practical way to solve this problem.
Books and online courses.
You already have great hands on skills. If you have the attention span for it you can improve your overall model of the system by reading books like Designing Data Intensive Applications. A good place to start for a reading list would be https://teachyourselfcs.com/.
When you have an aptitude for something there are sort of two natural directions you can go. It's possible to settle into a sort of "expert beginner" stage where you stop at a local maximum. The other option is to take the leverage that you already have and lever up again, hard. Those obscure and really useful programming tricks that you like, read some books on those too. See if you can find a conference talk where someone describes one. You probably know everything they're going to say anyway...except maybe one more trick that you can glean from that talk.
This is the path to Mastery.
You can also dig into some of the more academic content on youtube like MIT's algorithm courses stuff like that. But really I'd recommend just scratching your own itch.
In a lot of cases the answer to imposter's syndrome is "hey yeah, we all get it. Perfectly normal." In this instance though I suspect doing some reading will go a long way towards making you more comfortable.
Have a look at proper courses. For example https://ocw.mit.edu/courses/find-by-topic/#cat=engineering&s... should have it. Check the "Introduction to Algorithms" ones.
Cormen - Introduction to Algorithms (CLRS) is probably the most recommended.
Depending on where you are starting from, MIT opencourseware lectures might be helpful too.
This is a really interesting class, and Erik Demaine is a top notch lecturer. If you're interested in algorithms I can easily recommend this class.
This isn't intro material though, and most of it is fairly niche in terms of usage. Your average programmer probably won't use any of this knowledge.
The most generally useful bits I remember were on the different types of persistence in data structures, which is legitimately cool and occasionally useful, especially in purely fuctional programming.
I also found the link-cut trees to be about the most interesting data structure I know of, they're cool as hell. You start with a pretty simple DS (splay trees), add some interesting analysis that's also related to heavy-light decomposition, and extend them to do something impressive. It's just a fun mix of analysis and implementation towards a goal. The moment it clicked for me was one of those once-a-year satisfaction events.
If any of that somehow talked you into doing this class, I'd recommend the Intro to Algorithms class also in MIT OCW first, at least one of the years is partially taught by Erik Demaine as well.
This example is directly mentioned in MIT Introduction to Algorithms, I would recommend it - https://ocw.mit.edu/courses/electrical-engineering-and-compu...
In practice, I think you'd be hard put to find a university CS degree that didn't involve a fair bit of programming. There was often more distinction in the past, especially at schools where CS came out of the Math department for historical reasons. But look at something like the Intro to Algorithms and CS class at MIT--there's a two-part MOOC on edX--and there's definitely programming. (Although it's clearly designed for people who already have some programming background. The assumption at top-tier schools tends to be that incoming students who want to major in CS already know programming to some degree, which has it's own set of problems but that's another topic.)
My daughter's first class in programming used this book. She really liked the class and still has a fondness for Scheme, although her subsequent programming classes have used Java.
I took a look at the online second edition (and I happen to have a hard copy of the first edition in my library). I like the book and believe that it is one of the best books I've seen for teaching a beginner to program. The subjects covered are non-trivial, but the book gets the students there in a well paced presentation. The book emphasizes the importance of breaking down problems into well defined parts, defining the interfaces, and implementing each part as a function that satisfies the desired contract.
If you are an experienced programmer, I believe that your time would be better spent on a book that is less oriented around teaching the basic steps that you already know. Books that I can recommend to experienced programmers interested in advanced subjects in programming:
1) Structure and Interpretation of Computer Programs, 2ed., by Harold Abelson, Gerald Jay Sussman, Julie Sussman. This book will teach you how to program in Scheme, and you will learn much more than just the syntax. I would consider this to be a very demanding book for a complete beginner. Available online: https://mitpress.mit.edu/sicp/full-text/book/book.html
2) The Science of Programming (1981), by David Gries. This book and the smaller, but equally difficult monograph The Discipline of Programming (1976), by Dijkstra are great books but too hard and mathematical for many readers. They focus on verifying the correctness of programs and introduce a formalism that shows how the logical effects of each program statement can be combined to prove that the program satisfies its specification. There are practical limitations not addressed by these early expositions on program correctness, but the techniques covered can be applied by hand and I've often used them on especially troubling passages of code that I wanted to make sure were correct. If you are up for it these books will introduce new ways of thinking about your code. I'd advise taking a peek at the books before buying to make sure it will be worth the investment; it definitely was for me.
3) Learn You a Haskell for Great Good!, by Miran Lipovaca. Like learning Scheme, learning Haskell will teach you about a new way of thinking about programming. This is a nice introduction to the language. Even though I'm still trying to learn Haskell, this book was my favorite of all my books on Haskell (I've got all of the ones written in English). Learn Haskell to understand where researchers are exploring the possible futures of programming. This book is also available online: http://learnyouahaskell.com
4) Algorithms + Data Structures = Programs (1976), by Niklaus Wirth. This book is far easier on the reader than the ones I've listed above; I've included this book because it is just a fun read for any programmer. It's old and uses Pascal, but it still a great book on programming that introduces a number of algorithms and data structures. It's available used for reasonable prices: https://www.amazon.com/Algorithms-Structures-Prentice-Hall-A.... For a more advanced introduction to data structures I suggest MIT's Open Courseware videos taught by Professor Erik Demaine, https://ocw.mit.edu/courses/electrical-engineering-and-compu...
Check out [1] for videos of Erik Demaine's lectures on folding. Also, I highly recommend the videos for the algorithms and data structures courses he's taught/co-taught ([2] for example)
[1] https://ocw.mit.edu/courses/electrical-engineering-and-compu...
[2] https://ocw.mit.edu/courses/electrical-engineering-and-compu...
Algorithms/data structures classes are a good start.
I hear that the Coursera algorithms classes by Prof Sedgwick (Princeton) and Prof Roughgarden (Stanford) are very good. I think the Princeton ones might not be available after the Coursera switchover so would recommend downloading the course content if you can. There were some threads on HN in the last week about it.
Also MIT OpenCourseware has Intro to Algorithms which I hear is good too.
C is not the place to look for good core concepts. I recommend http://ocw.mit.edu/courses/electrical-engineering-and-comput... instead.
I think these courses are awesome as well:
6006 Introduction To Algorithms from MIT
http://ocw.mit.edu/courses/electrical-engineering-and-comput...
Machine Learning from Stanford: Learn about the most effective machine learning techniques, and gain practice implementing them and getting them to work for yourself. https://www.coursera.org/course/ml
You should also try going through MIT's online course on algorithms based on CLR: http://ocw.mit.edu/courses/electrical-engineering-and-comput...
Doing a BS in CS would be the best, otherwise you will need a ton of self-discipline to learn even a half of what you would learn in a degree program. If you decide to self study, and have enough "mathematical maturity", you will find everything you need on MIT OpenCourseWare, they have all the core courses available online with video lectures, solved homeworks etc. I would start with going through those two first:
Structure And Interpretation Of Computer Programs:
http://ocw.mit.edu/courses/electrical-engineering-and-comput...
Mathematics For Computer Science:
http://ocw.mit.edu/courses/electrical-engineering-and-comput...
The first one is pretty much a master class in software engineering, the other prepares you for studying algorithms, I would consider those the two main more theoretical facets of programming at a professional level. With that background you can try to work on their algorithms course:
http://ocw.mit.edu/courses/electrical-engineering-and-comput...
It would also be very valuable to eventually take all the core mathematics subjects from OCW: Single+multivariable calculus, linear algebra and probability theory - the profit is less immediate but those things do come very useful. All those six courses on the MIT OCW are top notch.
I am a 29 year old self-taught developer. Like you, my experience lies primarily with ruby and javascript (though I have developed a habit of doing project euler problems in clojure).
1) Practice. Glassdoor is an indispensable tool for this. For example, if you're interviewing at Facebook, you can basically compile a list of the algorithms that have been asked and practice that set, until you have it completely down. Rehearse in front of peers, friends, pets, inanimate objects, whatever. You can likely find a working solution to any of these problems on stack.
2) Yeah, they are pretty important, but probably not to the degree that they are emphasized in the interview process. For example, a common theme in algorithm problems is finding a solution which works within a certain time complexity (linear, log, etc). In practice this often reduces to the following situation: You are doing some calculation which involves a bunch of nested for-loops. Eliminate one of the inner for-loops.. or something to that effect. So yeah, it's important practically. It becomes more important when you're dealing with millions of records and terabytes of data, and an extra for-loop could mean that your program effectively runs a month longer than it should.
3. The only online course I took for CS stuff was a MIT open-courseware class on Algorithms. Additionally, I highly recommend the book "Algorithms" by Robert Sedgewick and Kevin Wayne.
Try the video course that goes with the Cormen book on Algorithms. You can start it right now, its not a scheduled course like Coursera but more a repository like Khan.
http://ocw.mit.edu/courses/electrical-engineering-and-comput...
Unfortunately, at least in my experience, most textbooks are rather unfriendly for self-teaching (including the ones very frequently recommended), especially in the more theoretical fields like algorithms, theory of computation etc. (no solutions to exercises, not enough worked out examples, general strong reliance on classroom usage), it takes a long time to find good ones and even then a very long time to get to a decent speed in a new field, surely much longer then when taking a course. Thus I would recommend relying more on some online courses at least initially:
http://ocw.mit.edu/courses/electrical-engineering-and-comput... - Very good course covering the most crucial theoretical things every programmer should now
http://aduni.org/courses/ - Good discrete mathematics, algorithms, theory of computation courses (I recommend "taking" them in this order) without assuming any prerequisites
http://ocw.mit.edu/courses/electrical-engineering-and-comput... - More sophisticated "CS 101" kind of course
I think almost everyone with some programming experience should be able to work through them in the sequence listed in one year perhaps, at a satisfying level of detail. At the same time, many people programming professionally lack mastery of this basic material.
But online courses actually give you more options as far as personalizing education. At most universities, unless they're very large or very specialized, you're likely to have one professor from a specific subject in a field. With online lectures, even from a reasonably small number of sources you can mix and match. Two personal examples I can think of is supplementing my algorithms class with MIT's algorithms class on OCW, and watching Tom Mitchell's lecture on kernel functions to add depth to the topic as covered in Ng's Ml-class.
Even at great universities there are professors who are better researchers than teaches that still may be a students only option. If you took the top 10, or even 5 teachers on a given subject you could give students the freedom to mix and match as suited their learning style.
Personally I actually find the lectures a poor substitute for a text. I find the Probabilistic Graphical Models textbook a great addition to the pgm class on coursera. Even though the lectures for pgm-class are long and detail, there's still only so much information you can get into a few hours. The information density of a textbook is hard to beat.
I haven't taken the class, but have helped many friends through the course if they needed it.
The class is more of an introduction to programming using Python than a class about learning Python the language. They learned some things like sorting, searching, testing, basic hash tables, etc. However, the primary reasons the course exists are to see let students explore Computer Science or because their major requires that they take either 6.00 (This course) or 1.00 (Same type of course, but in Java.)
If you want to learn Python and you're an experienced programmer, this class will take you maybe a night or two to breeze through. I would recommend taking other courses on OCW instead though with a more specific focus. If you want basic-intermediate algorithms, take 6.006. If you want low-level, 6.004 is a great introduction. (If you want advanced algorithms, buy CLRS and take 6.046).
tl;dr: 6.00 is more for people that have never programmed anything before, not to learn Python.
For those of you just getting into this book, there is also a great course with video lectures up on MIT's OCW site: http://ocw.mit.edu/courses/electrical-engineering-and-comput...
MIT hosts a lot of their course materials online at http://ocw.mit.edu/index.htm . Dig through some of their classes.
I'm not sure at what level you mean by 'beginner', but my recommendation is to run through 6.001 (Structure and Interpretation of Computer Programs). MIT has a new Python-based intro class that has since replaced 6.001, but the old class is still a very well-baked and well-respected curriculum.
After that, if you want, dig into a couple data structures and algorithms classes (like 6.006) to build up your theoretical background.
Next look into a software engineering course (theirs is 6.005). This should get you a bit of feel for diving into a large system without getting lost, or organising one of your own.
After that, just start poking around at things that look interesting. Build something shiny. (That's the great thing about our industry---you can get your tools for free, and build something useful without making a huge initial investment. That would not happen in a lot of other fields.)
I really can't stand it when people complain about computer science education.
News flash - programming != computer science. It doesn't require a PhD to learn (or teach) Python or AJAX. In fact there's nothing inherently scientific about the day-to-day uses of that language. You want to learn to program? Go to ITT Tech or work on open source. Hell, you could land an interview pretty much anywhere if you spent 12 weeks watching the MIT Intro to Algos lecture and The C Programming Language. WHY DO YOU THINK IT'S AN INTRO COURSE?
If you want to learn how programming languages (syntax, grammar, compiler, interpreter), an operating system, automata, logic/circuits, Turing machines, AI, HCI, graphics, databases, MATH works and the theory and provenance behind all of it...you want a computer science degree.
Sorry your professor isn't teaching you the difficult inner-workings of jQuery.
Frankly, CLRS goes over a lot of the basic knowledge you need to get through the book. I don't think a discrete math textbook will help you if your main goal is to get through CLRS; the math in that book isn't that dense. Like all things, it just takes practice. Spend a lot of time on sections that are hard to understand, and future sections will become a lot faster and easier.
Try the MIT algorithms video lectures: http://ocw.mit.edu/OcwWeb/Electrical-Engineering-and-Compute...
According to an older ocw version of this course [1], you need "General ease with algorithms, elementary number theory and discrete probability". You can learn these from MIT's own ocw's excellent "Mathematics for Computer Science" [2] and "Introduction to Algorithms" [3].
For a gentler intro to crypto, I am planning to do Cristof Paar's course first, given that he takes his time to carefully explain all the math; there is a youtube list with all the lectures [4] and a course webpage [5]. In case any further encouragement is needed, according to a blog posting "Matasano Crypto Challenges, Set 5" [6]: "This set was surprisingly easy, actually. The book Understanding Cryptography by Paar & Pelzl is an excellent intro to the basic maths needed for crypto — namely, the group theory and number theory necessary for RSA and Diffie-Hellman."
Now that I am at it, here are some HN links you may find helpful if planning to learn crypto, in no particular order:
Rolling Your Own Crypto (DON'T!) [7]
Telegram’s Cryptanalysis Contest [8]
The most dangerous code in the world [9]
BearSSL – Smaller SSL/TLS [10]
NaCl: DJB’s new crypto library [11]
JavaScript Cryptography Considered Harmful [12]
Cryptographic Right Answers [13]
Ask HN: Current Crypto Best Practices [14]
Mathematical Cryptology [15]
Ask HN: What are the best books for learning information security? [16]
Of course, there are also the courses Bohne's CS255, Rivest's 6.857 and many others, and the books Cryptography Engineering, Introduction to Modern Cryptography etc ... Harvard's [pdf] https://people.seas.harvard.edu/~salil/cs127/fall13/conclusi... is a good guide.
Good times for learning!
[1] https://ocw.mit.edu/courses/electrical-engineering-and-compu...
[2] https://ocw.mit.edu/courses/electrical-engineering-and-compu...
[3] https://ocw.mit.edu/courses/electrical-engineering-and-compu...
[4] https://www.youtube.com/playlist?list=PL2jrku-ebl3H50FiEPr4e...
[5] http://www.crypto-textbook.com/
[6] https://raywang.tech/2017/04/21/set5-writeup/
[7] https://news.ycombinator.com/item?id=13221923
[8] https://news.ycombinator.com/item?id=6940665
[9] https://news.ycombinator.com/item?id=4695350
[10] https://news.ycombinator.com/item?id=12863699
[11] https://news.ycombinator.com/item?id=704689
[12] https://news.ycombinator.com/item?id=7903720
[13] https://news.ycombinator.com/item?id=16748400
[14] https://news.ycombinator.com/item?id=14041827
[15] https://news.ycombinator.com/item?id=7795958
[16] https://news.ycombinator.com/item?id=11710028
Coursera has a couple of courses on algorithms that would be acceptable for beginners but also very challenging (in a good way).
MIT OCW's Intro to Computer Science and Programming (in Python) covers a lot of ground related to programming (more than just Algos), but it covers some essential sorting algorithms and eventually gets into dynamic programming. I don't think anything much simpler than this would be worth your time (not to say this is the best), but the readings themselves (especially those from wikipedia) can actually be a little dense: http://ocw.mit.edu/courses/electrical-engineering-and-comput...
MIT OCW also has a full course on algorithms.
I don't think anybody doing your project should discourage you. It's usually in the process of trying to do something that has been done before that you get an insight or think about a better way of doing it. Plus, doing things that have been done before is a very good way to learn. Thats how in the old days apprentices became masters.
As for the alg/ds. In my opinion, there are some fundamental things that you need to know about DS/alg that make you a better developer. What people usually get wrong is assuming that it is the applicability of the algorithms or datastructure that matters ( who cares if you know about a Red/Black Tree ! ) . I think its more about the patterns and the way they allow you to think.
Understanding how tree traversal is done may not be directly useful in your work but helps you understand recursive thinking which is super important. Knowing how QuickSort/MergeSort work doesn't have any application in real life but makes you think about problems with a divide/conquer approach in mind which is again super practical. You will never write a Hash/Dictionary but knowing what is the difference between one and an array has immense applicability on your work.
My advice to you: stay away from memorizing DS/Alg. Throw out your "cracking the coding interview" book if you have one and pick a algorithm and datastructure book and try to understand it. Then you can move on to solving problems yourself and you will realize that you are a better developer for it. Here is what I would suggest:
http://ocw.mit.edu/courses/electrical-engineering-and-comput... http://interactivepython.org/runestone/static/pythonds/index...
Of course, you don't need to know all of the stuff here :)
Open source courses would be cool. Not open source like mit's ocw or coursera, but a platform where everyone collectively decides on the material. So, for an algorithms course, maybe MIT's hashing lectures are second to none, but some guy on YouTube has better explanations for divide and conquer algorithms. The idea would be to find the best material for each lecture and compile it into a class
Take it easy, there are lot of options available online if you are not enjoying what is being taught in college but you have interest in particular subject. For e.g.: When I was in college I felt my profs. are boring , so I always used to take online courses , like algorithms , operating system through ocw.mit.edu or stanford.edu or coursera , It helped me a lot to maintain interest in subject , and at same time participate in discussions online , there is always a big community somewhere which will be happy to help you .
For reference, MIT has two undergrad courses - 6.006 and 6.046 - which use CLRS, covering about half the book each. An undergrad course at MIT is expected to take up about 140 hours, including lectures, reading, and assignments. So, working for 10 hours per day, one could theoretically complete each course in two weeks.
There are a few other things to consider. One is that it's probably not necessary to go through all of the material to be able to manage interview-type questions. (It would obviously be best to undertand CLRS from cover to cover for the purpose of becoming a better engineer, but that's not what's under discussion.)
Second, it's really questionable whether someone can effectively work for 10 hours per day on this sort of material. My guess is that people fall into two categories - those who are not used to algorithmic thinking, and those who are. People from the first category would not be able to absorb that much new information that quickly, so for them, what I said is inaccurate. However, people in the second category would not only be able to process CLRS in large batches, but would need much less than 140 hours to work through half the book; someone familiar with the underlying concepts (basic programming, probability, etc.) and with a background in proofwriting could very well get through enough information to answer interview questions twice as fast or faster, so in about a week or less.
http://www.catonmat.net/blog/mit-introduction-to-algorithms-... is a very good site, with videos of MIT lectures.
http://www.cs.sunysb.edu/~algorith/video-lectures/ is another great source for video lectures.
http://en.wikipedia.org/wiki/List_of_algorithms has a veritable zoo of algorithms. Best part: contribute to the pages and you'll learn it better than just reading. For a true zoo of algorithms to start off a search, http://www.itl.nist.gov/div897/sqg/dads/ is perhaps the best.
Stack Overflow's got many good threads on Big O and comparative complexities, for instance http://stackoverflow.com/questions/133008/what-is-big-o-nota...