A Cookbook of Self-Supervised Learning Self-supervised learning(SSL) has significantly advanced AI. The world contains a massive amount of unlabelled data and it is almost impossible to label everything. Recent AI systems have been able to learn from such massive unlabelled
@jeande_d
-
Appreciation for Comprehensive Deep Learning Handbook PDF
By
–
Really structured well and concisely, ~120 pages!! Thanks for this amazing deep learning handbook, @francoisfleuret
! The AI community benefits a lot from having publicly available resources like this. Book link[PDF]: https://
fleuret.org/public/lbdl.pdf -
The Little Book of Deep Learning: Free Comprehensive Guide
By
–
The Little Book of Deep Learning A very concise/brief book on deep learning. Covers almost any topic you'd want to know today from foundations, efficient computation, model architectures, training models, synthesis(generative AI), etc… And it's free: https://
fleuret.org/public/lbdl.pdf -
Guide to Training Your Own Large Language Models
By
–
How to train your own Large Language Models This is a great article on the mechanics of training (your own) large language models. Discusses the importance of training your own LLM, data pipelines, model training, evaluation, and deployment. https://
blog.replit.com/llm-training -
Transformer Math 101: Understanding Computation and Memory Usage
By
–
Transformer Math 101 An excellent blog post about basic math related to computation and memory usage for transformers. Nicely explained!! https://
blog.eleuther.ai/transformer-ma
th/
… -
Pen and Paper Exercises in Machine Learning: Foundational Topics
By
–
Pen and Paper Exercises in Machine Learning A handy collection of pen-and-paper exercises in machine learning. Covers foundational topics related to machine learning such as linear algebra, optimization, graphical models and etc… This is a relevant resource for anyone who
-
Emergence and Reasoning in Large Language Models Talk
By
–
Emergence and reasoning in large language models – A talk This is a great talk by @_jasonwei on the emergent abilities of large language models(LLMs), chain-of-thought prompting, and other related topics. Video(YT): https://
youtube.com/watch?v=0Z1ZwY
2K2-M
… Slides: https://
self-supervised.cs.jhu.edu/fa2022/files/j
asonwei-CoT-emergence-talk-JHU.pdf
… -
Transformers Remain Fundamentally Unchanged Despite Recent Extensions
By
–
There have been some incredible works around extending transformers to other tasks(ex: ViT) and efficiency(etc.., ex: flash attention.) but deep down, it's the same transformer, with some minor changes here and there.
-
Transformers: Six Years of Universal Neural Network Architecture
By
–
The "Attention is all you need paper" that introduced Transformer neural network architecture has been around for roughly six years. It's by far the first architecture to maintain its universality for a long time, not just for a single modality but for other modalities as well.
-
CNNs at Scale: How AlexNet Changed Machine Learning History
By
–
Many people were clearly using CNNs but history says Alex and his colleagues did it on large scale and the results transcended the field.