The Ultra-Scale Playbook: Training LLMs on GPU Clusters
@jeande_d
-
Distributed Training Playbook: LLM Infrastructure Fundamentals and Benchmarks
By
–
Excellent playbook on the intricacies of distributed training. Covers basics of LLM training infra from the very ground-up, practicals/code, and efficiency benchmarks grounded on 4100 distributed experiments. Topics covered include:
– Data parallelism
– Tensor parallelism
– More -
LLMs Psychology and Chip Architecture Optimization Deep Dives
By
–
Last week we got Deep Dive into LLMs and all their pyschology. This week we get "how to squeeze out every bit in chips & architectures" from Jeff and Naom! Good times 🙂 https://t.co/bTlxmpOCm2
— Jean de Dieu Nyandwi (@Jeande_d) 12 février 2025Last week we got Deep Dive into LLMs and all their pyschology. This week we get "how to squeeze out every bit in chips & architectures" from Jeff and Naom! Good times 🙂
-
Andrej Karpathy Releases New Introductory LLM Video Series
By
–
A new video in Karpathy "Intro to LLMs for general audience" series dropped. This is a world gift. Thank you for all high quality videos, Andrej!!!
-
Fundamental Skills Remain Valuable in the AI Era
By
–
One interesting(and positive) thing about current AI landscape is “fundamental skills” will always be valuable. What you get off generative AI tools is a limit of what you can ask, and your ask largely depends on your base knowledge. Some problems will not require anything extra
-
The Hundred-Page Language Models Book: A Concise Introduction
By
–
Remember "The Hundred-Page Machine Learning Book"? There is also "The Hundred-Page Language Models Book". Very concise and released with the same idea as previous books: read first, buy later.
-
Tribute to Felix Hill’s Influential Deep Learning NLP Lectures
By
–
I have not personally met @FelixHill84
, but this is incredibly sad to hear. I have watched his lectures on deep learning for NLP several times, they were my reference materials for long time, and still valuable today. May his family, friends, and colleagues stay strong -
Essential Book Recommendation for Understanding Generative AI and LLMs
By
–
If you're into generative AI and LLMs and would like to understand the underlying tech/tools/models, get this book. Go into debt if you have to.
-
2024 Achievements: Master’s Degree, NeurIPS Paper, Research Associate
By
–
2024 wrapped – Completed master of science in engineering AI at Carnegie Mellon
– Contributed to two remarkable papers in multimodal evals & multilingual multimodal LLMs – 1 accepted to NeurIPS 2024(D&B track) and other under review
– Started a research visit/associate at -
AI Enhances Developer Value Rather Than Replacing Them
By
–
There has never been a good time to learn coding. Software developers constitute less than 1% of global population. AI will help developers to build better software applications and faster. The value of developers will not be zero because AI systems can do the work. In fact, the