#89 overall
EE364A Convex Optimization
Stanford University
5 recommenders · 5 threads · recommended in 4 years, 2010–2023 · score 5.0
EE364A is Stanford's introductory course on convex optimization, covering convex sets and functions, duality, and practical methods for solving convex problems. Video lectures are publicly available alongside the course materials.
Commenters call it one of the best courses for building a mathematical foundation behind machine learning and AI, and praise its video lectures.
Mathematicsintermediatefreevideo lectures#convex optimization#optimization#duality#linear algebra#boyd#stanford
Recommendations by year
20062026
And what about after that? In my experience at least, I can highly recommend Stanford's EE364a Convex Optimization. http://web.stanford.edu/class/ee364a/
When I was an undergrad at Berkeley, one particularly well known research lab would only look at your resume if you took that course online. Warning: that class is not for the faint of heart. And be good at linear algebra.
Why is Convex Optimization (EE364a) not included?
https://stanford.edu/class/ee364a/
https://www.youtube.com/playlist?list=PL3940DD956CDF0622
It's one of the best courses to take if you want to obtain some fundamental understanding of the mathematical concepts behind AI.
Yes there's much more to it than NNs/transformers/'Attention is all you need' paper/whatever else is trendy right now. No, don't expect to do important, as in employable, work if you won't be spending some time truly understanding the mathematical foundations.
I would recommend a first-principles approach, if you're really interested in the field, building a career around it, and not just jumping on something because it's "hot".
To that end, start with Convex Optimization [1]. You'll develop an incredibly versatile - but not esoteric - mathematical background. You'll link the mathematics to solving real engineering problems fairly quickly. You'll tackle the basics of machine learning as well.
After this, you'll have a pretty strong background to get into more traditional machine learning and deep learning. Regarding the former, Andrew Ng's notes are pretty solid [2], and for DL, Karpathy's Stanford course is great [3].
Self-studying all this material could take up to a year (part time, assuming you do it while having a job), but I don't know of a better way to gain the skills and get into the field. This approach balances your learning of fundamentals, engineering applications, real software, numerical computing, and the more fun "new" stuff.
Hope it works out well for someone else.
[1] - http://stanford.edu/class/ee364a/ Great video lectures available too.
[2] - http://cs229.stanford.edu/materials.html Also has great video lectures on Youtube.
[3] - http://cs231n.stanford.edu/index.html Ditto on the lecture vids.
For further reading there is an excellent resource here http://web.mit.edu/15.053/www/ with a leitmotive example and both theoretical and practical approches. It also explains in detail the theory behind the real world implementation of simplex method (aka revised simplex) with inversed matrix factorization methods, etc. This is a quite old book but as far as I know even modern linear solvers use those techniques (at least open-source ones for sure).
This is a very specific and little subset of convex optimization and yet very powerful (and exciting).
Convex optimization methods can also be used in many non-convex situations, for example when you have quasi-convex or log-convex functions as objective or constraints. I recommend Boyd's excellent course https://lagunita.stanford.edu/courses/Engineering/CVX101/Win... and the online free book https://web.stanford.edu/~boyd/cvxbook/bv_cvxbook.pdf (warning : very addictive)
If you're looking for something less pricey... I highly recommend Stephen Boyd's book (free here: http://www.stanford.edu/~boyd/cvxbook/), as well as his cvx matlab toolbox. There are also some video lectures which might be less mind-numbing (http://www.stanford.edu/class/ee364a/index.html).
Sure, a couple things.
(I'm assuming you're comfortable with multivariable calculus.)
Andrew Ng's coursera course is good.
PRML (pattern recognition and machine learning) by bishop is good, and has a useful introduction to probability theory.
You also want a good grounding in linear algebra. Strang is basically the authority on linear: http://ocw.mit.edu/courses/mathematics/18-06-linear-algebra-...
You want a strong grounding in probability theory and statistics. (This is the basic language and intuition of the entire field.) I don't have as many preferences here (although its the most important); someone in this thread pointed to a course on statistical learning @ stanford that's good.
A good understanding of optimization is helpful. Here's a link that leads to a useful MOOC for that: http://stanford.edu/~boyd/cvxbook/
there's a lot of other stuff (markov decision processes, gaussian processes, monte carlo methods come to mind) that is useful that I'm not pointing to, but if you've hit the other stuff here then you'll probably be able to find out those things.
If you're into it, https://www.coursera.org/course/pgm is good but not vital.
You may want to know about reinforcement learning. This answer does better than I can: https://www.quora.com/What-are-the-best-books-about-reinforc...
Deep learning seems popular these days :) (http://www.deeplearningbook.org/)
Otherwise, it depends on the domain.
For NLP, there's a great stanford course on deep learning + NLP (http://cs224d.stanford.edu/syllabus.html), but there's a ton of domain knowledge for most NLP work (and a lot of it really centers around data preparation).
For speech, theoretical computer science matters (weighted finite state transducers, formal languages, etc.)
For vision, again, stanford: (http://cs231n.stanford.edu/syllabus.html)
For other applications, well, ask someone else? :)
Also:
arxiv.org/list/cs.CL/recent arxiv.org/list/cs.NE/recent arxiv.org/list/cs.LG/recent arxiv.org/list/cs.AI/recent
EDIT: unfortunately, there's also a lot of practitioner's dark art; I picked a lot up as a research assistant, and then my first year in industry felt like being strapped to a rocket.