#13 overallClassic
CS231n: Convolutional Neural Networks for Visual Recognition
Stanford University
39 recommenders · 41 threads · recommended in 10 years, 2015–2024 · score 49.3
Stanford's CS231n covers convolutional neural networks and deep learning for visual recognition, including backpropagation, training networks, CNNs, and related architectures, with course notes, assignments, and lectures. It is taught with a hands-on, assignment-driven approach.
Commenters repeatedly call it an excellent, hands-on introduction to deep learning and neural networks, praising the course notes, assignments, and Karpathy's teaching.
Machine Learning & AIintermediatefreemixed#convolutional neural networks#deep learning#computer vision#backpropagation#karpathy#stanford
Recommendations by year
20062026
I am learning game development in Godot, specifically with the intention of making an Oculus Quest VR game. I just finished the initial tutorial yesterday: https://docs.godotengine.org/en/3.2/getting_started/step_by_...
Alongside that, I am also watching Disney's Imagineering-in-a-Box, which describes how they develop lands and rides for their theme parks: https://disneyparks.disney.go.com/blog/2020/03/enjoy-a-one-o...
I recently finished Stanford's CS231N Computer Vision course from 2017 (watching YouTube + 3 Jupyter Notebook assignments). Also highly recommended. http://cs231n.stanford.edu/syllabus.html
Personally I recommend Stanford CSI 231n http://cs231n.stanford.edu/
Its specifically geared towards visual recognition, but it starts with the basics of machine learning and moves on to feed forward nets and covnets and covers RNNs and attention towards the end.
The assignments are a great set of jupyter notebooks that really get your hands on the material and you can find a number of peoples complete assignments on github just by searching.
The lectures are available online as well https://www.youtube.com/playlist?list=PL3FW7Lu3i5JvHM8ljYj-z...
I've done hinton's and Ngs courses and as someone who already has a non-ai development background I found this to be the best introduction. Its really an extension of Andrej Karpathy's Neural Nets for Hackers (http://karpathy.github.io/neuralnets/)
For deep learning or chips for deep learning? With an EE/Chem/Math background you should be set to go :)
For deep learning, I highly recommend the cs231n course materials (available for free online) and the Deep Learning Book by Goodfellow et al.
For chip design for deep learning specifically, it's a fairly new field, and a lot of commercial interests, hence why a lot of it isn't available in the form of an "Intro to Deep Learning Chip Design" course. But, the basic point is that deep learning can both train and perform inference in astonishingly low precision.
For training: https://arxiv.org/abs/1502.02551
For inference, there are so many papers confirming this fact that there really isn't one key paper to point to. This article is a good introduction however: https://petewarden.com/2015/05/23/why-are-eight-bits-enough-...
Other than that, stripping out the cache hierarchy, HBM memory, etc. are the obvious steps to take in targeting deep learning. Note that it's my opinion that deep learning chip startups (such as ours) need another "secret sauce" beyond simply lowering precision, we're not like Intel which can just say "okay, here's $100mil, make me an 8-bit GPU, Go."
Why do you have to learn to calculate integrals and derivatives in school, or how compilers work internally? Same answer. But seriously, the CS231 class is excellent, and Andrej is an excellent teacher. You can follow along at home (which is what I am doing.) The syllabus (at http://cs231n.stanford.edu/syllabus.html) has the course notes and the assignments. The assignments are self grading, you know when you have it programmed correctly. The lectures are here: https://www.youtube.com/playlist?list=PLlJy-eBtNFt6EuMxFYRiN...
After taking Andrew Ngs course I highly recommend you take cs231n afterwards.
Assuming you know a little bit of calculus you'll be able to create and train fully connected networks, convolution networks, recurrent networks, and long short term memory networks using only numpy. You will feel very comfortable with deep learning after taking this class.
http://cs231n.stanford.edu/syllabus.html
The lecture videos for the course got taken down due to complaints, but you can recover them via google
A month ago I was in the same place- I would start reading a short blog posting on RNNs/ConvNets/etc., and within 2-3 paragraphs my eyes would glaze over from the math and other foreign terminology. Frustrating. To try and fix this I am "auditing" the Stanford course on ConvNets: http://cs231n.stanford.edu/syllabus.html
I'm about 2/3 done with the homeworks, and I understand this stuff now. I'll never be a data scientist, but I know enough to implement these networks on my own, and to understand blog posts like this. It's a lot of work for one course, much more than I remember from my own undergrad years. I had to revisit Calculus & Linear Algebra too. But if you're genuinely interested in this stuff you can pick it up.
Andrej is also a great lecturer; his CS231n class in the winter was both the most enjoyable and educational I've taken all year. All of the materials are available at cs231n.stanford.edu, although I can't seem to find the lecture videos online. It may not have been recorded.
As a bonus, there's an ongoing class on deep learning architectures for NLP which covers Recurrent (and Recursive) Neural nets in depth (as well as LSTM's and GRU's). Check out cs224d.stanford.edu for lecture notes and materials. The lectures are definitely being recorded, but I don't think they're publicly available yet.
Karpathy is one of my favourite authors - not only is he deeply involved in technical work (audit the CS231n course for more[1]!), he spends much of his time demystifying the field itself, which is a brilliant way to encourage others to explore it :)
If you enjoyed his blog posts, I highly recommend watching his talk on "Automated Image Captioning with ConvNets and Recurrent Nets"[2]. In it he raises many interesting points that he hasn't had a chance to get around to fully in his articles.
He humbly says that his captioning work is just stacking image recognition (CNN) on to sentence generation (RNN), with the gradients effectively influencing the two to work together. Given that we've powerful enough machines now, I think we'll be seeing a lot of stacking of previously separate models, either to improve performance or to perform multi-task learning[3]. A very simple concept but one that can still be applied to many other fields of interest.
[1]: http://cs231n.stanford.edu/
[2]: https://www.youtube.com/watch?v=xKt21ucdBY0
[3]: One of the earliest - "Parsing Natural Scenes and Natural Language with Recursive Neural Networks" http://nlp.stanford.edu/pubs/SocherLinNgManning_ICML2011.pdf
I personally preferred Andrej Karpathy's CS231n taught by him and his private videos about neural nets in general and transformers in particular. He has a youtube vid where he builds one from scratch in Python!
3BlueOneBrown videos are a great complement to Karpathy's lectures to aid in visualising what is going on.
I really like Andrej Karpathys content
yeah its youtube i know... but, its hand on too
for gpt/llm/ml in general https://karpathy.ai/zero-to-hero.html
it starts with writing back prop from scratch and take you through writing everything you need and training a gpt2 equivalent model in the end
I also thought his lectures at standford, on youtube cs231n 2016, were really good. they cover GAN for generation, but I think after that you can read the papers that were the source for diffusion models dalle and midjouney use
He is a master educator. While at Stanford he developed their undergrad machine learning intro course named cs231n which immediately became legendary. It's somewhat out of date on some details but it's still well worth watching especially as delivered by Andrej. You can find all 11 lectures on YouTube.
It's difficult because there are hundreds of really good courses and I don't keep track. It's almost a full time job to just keep track of this stuff given how much there is. Search for one of these "awesome Deep Learning lectures" Github repos and you will find them.
The Stanford courses (224, 231, and the Transformer one) are a safe bet. In general, I would recommend university courses where the lectures are online. There are a lot of them. Fast.ai is supposed to be good as well from what I've heard, but I don't have experience with that myself.
For me that was cs231n delivered by Andrej Karpathy. I consider it the best “from first principles” lecture series on neural nets that I was able to follow despite my somewhat rusty calculus and linear algebra.
I would recommend auditing Stanford courses in following order:
1. CS231n Machine Vision https://www.youtube.com/playlist?list=PLkt2uSq6rBVctENoVBg1T...
2. CS234 Reinforcement Learning https://www.youtube.com/watch?v=FgzM3zpZ55o&list=PLoROMvodv4...
3. CS330 Meta Learning https://www.youtube.com/watch?v=0rZtSwNOTQo&list=PLoROMvodv4...
Those will get you on track with general concepts about reasoning, AI engineering and concepts of learning itself
Language models for me a bit of headache because there're in different domain on intersection with linguistics and humanities but here's a good course
https://www.youtube.com/watch?v=rha64cQRLs8&list=PLoROMvodv4...
Those are all free and high-quality but require a lot of brain power
It was definitely not a casual course. I went in hoping for a graduate level course and that's what it was. I had to spend a lot of time on it. I didn't find the math too advanced but understanding the lectures and doing the exercises took a while.
Karpathy's CS231n was indeed another great course. We're really spoiled for choice these days.
No one suggested standford cs231n: http://cs231n.github.io/. I'd recommend the winter 2016 lectures (by FeiFei Li, Karpathy and Johnson). For getting started with convnets / deeplearning, I think this is one of the best hands on ressources out there.
My go-to when I started in Deep Learning is Stanford's CS 231n course http://cs231n.github.io/, the lecture notes are amazing. It was instrumental when I first dove deep into Deep Learning and helped me understand all the components needed to make Convolutional Neural Networks(CNN) and Neural Networks(NN) work. I first read this and watched the lecture videos. This course was a great primer for me to understand the content and theory in Goodfellow's Deep Learning book.
For deep learning / neural networks, I think these are the best balance of theory and practice (not to mention great visualizations): http://cs231n.github.io
More on the intuition/theory side for NNs, but Michael Nielsen’s book is highly recommended: http://neuralnetworksanddeeplearning.com
For NLP, you can try this extremely slick interactive course on spaCy (highly recommended Swiss Army knife library for NLP) from package author Ines Montani: https://course.spacy.io
The Keras tutorials are solid, here’s one on seq2seq models: https://blog.keras.io/a-ten-minute-introduction-to-sequence-...
For more cutting edge NLP, it looks like the fast.ai course covers the Transformer model (the basis of BERT, GPT-2, et al. This is a great overview of that architecture: http://nlp.seas.harvard.edu/2018/04/03/attention.html
And PyTorch implementations of Transformer models can be found here: https://github.com/huggingface/pytorch-pretrained-BERT/blob/...
LMK if anyone wants more, I have a learning resource hoarding problem.
I highly suggest the Stanford course https://cs231n.github.io. All the notes are in the link. I found it a very good introduction to backpropagation
Try the CS231n course from Stanford. Assignments, solutions, slides and lectures are available. Proper grad level deep learning course with excellent notes. The assignments are IMO, the best way to get good and in-depth knowledge of deep learning.
For books, Ian Goodfellow's deep learning is the best theory (as theoretical as Deep Learning gets ) oriented book.
Our vision is to be the tool that starts with great settings for beginners but lets you graduate into the internals as you become more expert - at the lowest level you can interactively create computation graphs and see their results as you change settings, sort of like eager mode for ml frameworks on steroids (or other visual computation graph programs that designers use like Origami/Quartz Composer).
The lobes in the UI are all essentially functions that you double click into to see the graph they use, all the way down to the theory/math.
If you want more comprehensive ways to learn the theory, I highly recommend Stanford's 231n course (http://cs231n.stanford.edu/) and the Goodfellow/Bengio/Courville Deep Learning book (https://www.amazon.com/Deep-Learning-Adaptive-Computation-Ma...)
Check out online courses. Coursera, EDX, etc. Andrew Ng's intro to machine learning course is a nice way to get up to speed with some basic concepts without diving too far into math. It doesn't cover any state of the art machine learning concepts, though.
Some other resources I have bookmarked:
- Convolutional Neural Networks for Visual Recognition Youtube playlist [1]
- Deep Learning for Self-Driving Cars [2]
- Natural Language Processing with Deep Learning [3]
[1]: https://www.youtube.com/playlist?list=PL3FW7Lu3i5JvHM8ljYj-z...
[2]: https://selfdrivingcars.mit.edu/
[3]: https://www.youtube.com/playlist?list=PL3FW7Lu3i5Jsnh1rnUwq_...
This is kind of a masters degree course i created for myself to get knowledge of Machine Learning from bottoms up
First, you need a strong mathematical base. Otherwise, you can copy paste an algorithm or use an API but you will not get any idea of what is happening inside Following concepts are very essential
1) Linear Algebra (MIT https://ocw.mit.edu/courses/mathematics/18-06-linear-algebra... ) 2) Probability (Harvard https://www.youtube.com/watch?v=KbB0FjPg0mw )
Get some basic grasp of machine learning. Get a good intuition of basic concepts
1) Andrew Ng coursera course (https://www.coursera.org/learn/machine-learning)
2) Tom Mitchell book (https://www.amazon.com/Machine-Learning-Tom-M-Mitchell/dp/00...)
Both the above course and book are super easy to follow. You will get a good idea of basic concepts but they lack in depth. Now you should move to more intense books and courses
You can get more in-depth knowledge of Machine learning from following sources
1)Nando machine learning course ( https://www.youtube.com/watch?v=w2OtwL5T1ow)
2)Bishops book (https://www.amazon.in/Pattern-Recognition-Learning-Informati...)
Especially Bishops book is really deep and covers almost all basic concepts.
Now for recent advances in Deep learning. I will suggest two brilliant courses from Stanford
1) Vision ( https://www.youtube.com/watch?v=NfnWJUyUJYU )
2) NLP ( https://www.youtube.com/watch?v=OQQ-W_63UgQ)
The Vision course by Karparthy can be a very good introduction to Deep learning. Also, the mother book for deep learning ( http://www.deeplearningbook.org/ )is good
From my personal experience I would recommend Andrew NG's courses. (I am on the second one right now). I have done fast.ai's first course which is low on math and more practical (you directly jump to coding, and then try to make some sense of things.) CS231n is definitely a great option, however I chose coursera because I needed deadlines/reminders of Coursera to keep me motivated and finish something. If you are looking for more options, there are lot more courses listed here in Machine Learning category : https://github.com/prakhar1989/awesome-courses however I don't know if someone has reviewed them.
1. Start with the fast.ai courses. It's an applied deep learning course using state of the art techniques. 2. For classical machine learning (regression, etc...), Andrew Ng's course on Coursera is widely considered "the basics" 3. As you progress, check out CS231 and CS224 from Stanford for state of the art image processing and natural language processing techniques. The lecture videos are on YouTube and the course assignments are available online. The third course I recommend is Geoffrey Hinton's neural networks course on Coursera (he is one of the most important researchers in the field). 4. If you're an application engineer, focus on using existing tooling to build cool projects. Keras and scikit-learn are great out of the box tools. 5. If you are more research oriented, you can start reading papers. In Silicon Valley, there's a meet up group that reads papers every Monday and tries to implement the algorithms in the paper. It takes a while to get to this level, but try not to get overwhelmed. Experts spend 7 years studying this stuff full time to get a PhD. 6. You really don't need much math to get started with ML. A high school understanding of calculus and some basic understanding of numerical optimization are the two main concepts you need to know. If you want to get into the research, there'll come a time when you will need more advanced math, but in my experience you can pick that up as you go along if you are curious.
Maybe you could start an AI study group online? The Silicon Valley study group was great, but I was just visiting.
Geoff Hinton's Coursera course was what got me into it. It's not for the faint of heart. I might recommend Andrej Karpathy's cs231n as a more up to date source today.
Probably you could look at # Watches, Stars, forks etc to gauge that, but if you want to look at vs theano, torch, pyTorch, Caffe2, start with cs231N lecture: http://cs231n.stanford.edu/slides/2017/cs231n_2017_lecture8....
(I think there's vid on youtube)
" The facts speak loudly for themselves; in most cases, deep learning-based solutions lack mathematical elegance and offer very little interpretability of the found solution or understanding of the underlying phenomena."
I don't agree with this statement, a simple look at cs231n lecture series will show you how much math is involved. A lot of articles/people claim its a black box, but while writing a small network architecture you realise it's not. Methods like stride, padding, activations, learning rate, optimisations, drop outs etc. give you "aha" moment which is followed by a mathematical explanation. One should study the topic thoroughly before criticising it.
Get your feet wet:
* http://cs231n.stanford.edu/ (the course notes are excellent)
* http://neuralnetworksanddeeplearning.com/
* http://www.deeplearningbook.org/
Use Tensorflow to train a few small neural nets. Move on to CNNs and RNNs. Make sure you actually do this. By this point you'll have read a lot, and retain none of it if you don't put it to use. Look at reinforcement learning. Use the book by Sutton and Barto, the new edition: https://webdocs.cs.ualberta.ca/~sutton/book/the-book-2nd.htm... Read the first 4-5 chapters, then go online and read about Deep Q learning, policy gradients, DDPG, etc. Then try to solve some problems on OpenAI Gym.
Once you have an idea of the kinds of problems you can solve, and have a couple you're interested in, go back and learn the foundational math, and start reading research papers.
In general, start with modern books that mention deep learning. With older books or high-level-overview books, you'll get frustrated when you see something cool on /r/machinelearning and can't find any mention of it in the book.
Agreed, cs231n is a great course. For neural net information less specific to computer vision I would also recommend looking up video lectures from Nando de Freitas and/or Hugo Larochelle.
I also have to put in a plug for Geoff Hinton's Coursera course. It's not for the faint of heart. It's essentially a graduate level college course with the commensurate required time investment. It's getting a little old now but it was my introduction to neural nets and if you want to learn from a pioneer in the field it's still a good option in my opinion: https://www.coursera.org/learn/neural-networks
I would love to recommend this Stanford course,very beneficial.
http://cs231n.github.io/
For general introductory material in this style from Stanford, CS231n (fairly general but specialization in vision) and CS224d (specialization in DL for NLP) are great. The material for both of these are online for free and the video lectures (taken down due to legal challenges regarding accessibility) are available if you look hard enough ;)
If you're particularly after unsupervised deep learning, I'd recommend you do one or both of the above (or equivalent) and then read relevant recent papers.
http://cs231n.stanford.edu
http://cs224d.stanford.edu
++ this. The branches of maths that are relevant to ML are all pretty extensive, and you can do a huge amount of applied ML and understand the underlying theory while understanding only a fairly small subset of (for e.g.) probability theory.
Starting by learning the maths will mean you learn a lot of stuff which isn't directly relevant. Not the worst thing that could happen, but you'll be a hell of a lot more directed (even if you want to learn the theory - and I would recommend learning at least some) if you pick a decent ML course and learn the maths you bump into as you go.
http://cs231n.github.io/ is one of the best general hands-on introductions I've found. The TF tutorials are pretty good too if you just want to try some things out, but I predict that once you've worked your way through the TF tuts you'll still not really understand what's going on and will feel a bit like you just learned the magic words that made the black box dance some particular dances.
As for notation - https://en.wikipedia.org/wiki/List_of_mathematical_symbols is really good for when you stumble on something unfamiliar.
Good luck.
As long as you know a lil bit of math and coding, you're good to go. If you're watching the cs231n videos and don't understand something, you can always check out their amazing class notes.
The cs231n course covers a lot of material, including some (or most) of the material from the Andrew Ng's Coursera course. You can totally do them in parallel though.
I recommend this course: CS231n Winter 2016 Stanford (Lecturer is Andrej Karpathy) https://www.youtube.com/playlist?list=PLLvH2FwAQhnpj1WEB-jHm...
check out cs224d and cs231n (both stanford) , there's another course by university of waterloo https://uwaterloo.ca/data-science/deep-learning , you will find lecture videos for all on youtube
Andrew Ng's machine learning course on coursera is great for beginners. It'll show you some theory backed by intuition, and if you do the assignments you'll implement ML algorithms yourself (in Matlab).
After that, I recommend Stanford's CS231n. It's focused on computer vision tasks, but you can learn a lot about neural nets in general from it. The videos were recorded around a year ago I think, so it's up to date with the latest deep learning techniques and libraries. For some reason the videos were taken down from the CS231n site but you can watch them here: https://www.youtube.com/watch?v=g-PvXUjD6qg&list=PLlJy-eBtNF...
Great writeup from Stanford CS231 course: http://cs231n.github.io/convolutional-networks/
I would recommend a first-principles approach, if you're really interested in the field, building a career around it, and not just jumping on something because it's "hot".
To that end, start with Convex Optimization [1]. You'll develop an incredibly versatile - but not esoteric - mathematical background. You'll link the mathematics to solving real engineering problems fairly quickly. You'll tackle the basics of machine learning as well.
After this, you'll have a pretty strong background to get into more traditional machine learning and deep learning. Regarding the former, Andrew Ng's notes are pretty solid [2], and for DL, Karpathy's Stanford course is great [3].
Self-studying all this material could take up to a year (part time, assuming you do it while having a job), but I don't know of a better way to gain the skills and get into the field. This approach balances your learning of fundamentals, engineering applications, real software, numerical computing, and the more fun "new" stuff.
Hope it works out well for someone else.
[1] - http://stanford.edu/class/ee364a/ Great video lectures available too.
[2] - http://cs229.stanford.edu/materials.html Also has great video lectures on Youtube.
[3] - http://cs231n.stanford.edu/index.html Ditto on the lecture vids.
Greg Brockman (Founder of Open AI) has written this amazing answer:
If you want to read one main resource... the Goodfellow, Bengio, Courville book (available for free from http://www.deeplearningbook.org/) is an extremely comprehensive survey of the field. It contains essentially all the concepts and intuition needed for deep learning engineering (except reinforcement learning).
If you'd like to take courses... Pieter Abbeel and Wojciech Zaremba suggest the following course sequence:
- Linear Algebra — Stephen Boyd’s EE263 (Stanford) - Neural Networks for Machine Learning — Geoff Hinton (Coursera) - Neural Nets — Andrej Karpathy’s CS231N (Stanford) - Advanced Robotics (the MDP / optimal control lectures) — Pieter Abbeel’s CS287 (Berkeley) - Deep RL — John Schulman’s CS294-112 (Berkeley)
(Pieter also recommends the Cover & Thomas information theory and Nocedal & Wright nonlinear optimization books).
If you'd like to get your hands dirty... Ilya Sutskever recommends implementing simple MNIST classifiers, small convnets, reimplementing char-rnn, and then playing with a big convnet. Personally, I started out by picking Kaggle competitions (especially the "Knowledge" ones) and using those as a source of problems. Implementing agents for OpenAI Gym (or algorithms for the set of research problems we’ll be releasing soon) could also be a good starting place.
Quora link: https://www.quora.com/What-are-the-best-ways-to-pick-up-Deep...
Excellent list! Can anyone with personal experience with these tell me which ones are most worthwhile?
Also I'd like to suggest cs231n over the udacity course for deep learning
For people interested, Stanford has an excellent online course on deep-learning with an emphasis on convolutional networks. [1]
It comes with video, notes, all the math, cool ipython notebooks and will let you implement a deepish network from scratch. That includes doing backprop through the svn, softmax, max-pool, conv and ReLU layers.
After that you should be more than capable to build a 'real' net using your favourite lib (Tensorflow, theano etc).
[1]: http://cs231n.stanford.edu/
Two courses I've found are very good for Neural Networks & Deep Learning are Karpathy's CS231n from Stanford[0] and Nando de Freitas's Deep Learning class from Oxford[1][2].
Been through parts of both and they seem really good. That said, I'd bet that both of these classes would be a lot more meaningful after either Hastie's class (listed here) or Andrew Ng's course.
[0] http://cs231n.github.io/
[1] https://www.youtube.com/playlist?list=PLE6Wd9FR--EfW8dtjAuPo...
[2] http://www.cs.ox.ac.uk/teaching/courses/2014-2015/ml/
Thanks! I'm really enjoying seeing the huge variety of domains people have applied char-rnn to. I haven't anticipated response at this scale, to both the original blog post and the code release. (I try to keep a somewhat up to date list of responses on the bottom of the blog - cooking recipes, music, obama speeches, magic cards...)
In terms of learning, I would also encourage people to try the materials from our CS231n class - this is a class I taught at Stanford last quarter with my adviser. It's technically about Convolutional Networks, but most of the class material is building up generic Neural Networks, backprop, and so on. You can also try our IPython Notebook assignments.
Course notes: http://cs231n.github.io/ Syllabus with slides too: http://cs231n.stanford.edu/syllabus.html
Another good pointer is Andrew Ng's Coursera class - that's a thorough introduction as well.
Ya, you sound a bit grumpy. I think Kaparthy wrote this blogpost to a) play with JavaScript and b) to reach a new audience.
If you want more rigor, check out the coursed he co-created: http://vision.stanford.edu/teaching/cs231n/
To just play with something : https://huggingface.co/spaces/nateraw/yolov6 (There's an images tab, and some samples below).
If you go to the associated code, you'll see that it needs a 'backbone', 'neck' etc. What is a backbone? Questions that arise directly from the code will lead you towards good blog articles, etc. https://huggingface.co/spaces/nateraw/yolov6/blob/main/yolov...
OTOH, you could go and have a look at (for instance) the Stanford vision courses for a more 'theoretical' approach. But the code itself is often solid guide to what's going on (the frameworks used for Deep Learning map well onto what's being discussed in blogs/lectures/papers).
You are confusing platforms with individual models. A platform, such as Pytorch or Tensorflow, does not care what shape is your input. You design and train a model for the whatever inputs you want. On the other hand, if someone trained a specific model architecture (e.g. YOLOv3 or Resnet-50) on some dataset of square images, then yes, this particular pre-trained model will expect the same input size and shape as what it was trained on. Does it make sense? If you take a beginner level course on deep learning (e.g. Stanford CS231n or FastAI course) you will immediately realize there's nothing (in any sense of the word) that prevents you from using any input shape to train your model.
If you don't want to learn the tools you use, you will need to find someone who will train a model on your images. If you're willing to pay for it you will find plenty of help. However, if you want to neither learn nor pay, then what exactly are you complaining about?
With all due respect, this is a quite random reading list, and appears more a result of the 'if "deep learning" in post.title: post.upvote()' trend on Hacker News ...
I will pick on two under the "Classics" section simply as I know the author and have used their work, so am not in any way saying the work isn't useful (it certainly can be in the right spot!), but it isn't "classic" or what I'd recommend for early readers at all. "Uncertainty in Deep Learning" and "Dropout as a Bayesian Approximation" were both published within the last year and a half and is a PhD thesis + paper on interpretations of neural networks in a Bayesian fashion. "Classic" for a paper + thesis less than two years old is quite a stretch even for the fast moving field of deep learning. The same holds true for many of the other papers in "Classics" such as batch norm which is (a) recent and (b) certainly not the only of such techniques (see layer norm, recurrent batch norm, ...) and (c) has complications in implementation[1].
As the simplest example, why is the original dropout paper[7] not under classics? It's an elegant paper, fundamentally important for current neural networks, and is more classic than "Dropout as a Bayesian Approximation" or "Dropout Rademacher Complexity of Deep Neural Networks" which are both listed.
I'm also highly dubious of the noted neuroscience connection - most deep learning researchers use very little from neuroscience.
Again, this list may be helpful to the creator of the repo and tailored toward their specific research direction but it is not useful for readers from Hacker News or those aiming to get their start in deep learning. Why so many upvotes? Zero comments? Zero discussion?
If you want a book, check out the Deep Learning book[2]. If you want a course for RNNs, check out CS224d[3]. If you want a course for CNNs, check out CS231n[4]. If you want to get down and dirty in a practical software engineering way, check out Fast AI[5]. If you want summaries of select recent deep learning papers in GitHub format, check out Denny Britz's notes[8]. There are many other starting points but those are my default suggestions.
If you really want to start learning, this isn't the right list for you and I'd really like to suggest a more sane and potentially tailored path. Seriously. If you reply with what you want, I'll do my best to suggest a starting point.
Background: I'm a deep learning researcher who publishes papers and articles[6].
[1]: http://www.alexirpan.com/2017/04/26/perils-batch-norm.html
[2]: http://www.deeplearningbook.org/
[3]: http://cs224d.stanford.edu/
[4]: http://cs231n.github.io/
[5]: http://course.fast.ai/
[6]: http://smerity.com/articles/2016/google_nmt_arch.html
[7]: https://www.cs.toronto.edu/~hinton/absps/JMLRdropout.pdf
[8]: https://github.com/dennybritz/deeplearning-papernotes