TorchInductor supports dynamic kernel generation, and is likely the state of the art at symbolic Tensor type systems. long gone are the days of static shape tensor compilers
MACHINE LEARNING
-
Choose Your Own Adventure: Building LLMs with JAX
By
–
You probably could choose your own adventure! . Eg @_nerdai_ coded the first book in Jax https://
github.com/nerdai/llms-fr
om-scratch-rs
…). -
2025 LLM Roadmap: From Fundamentals to Production Applications
By
–
Here’s the 2025 LLM roadmap 1. Code and train your own LLM to really understand the fundamentals
2. Train models more conveniently using production-ready libraries
3. Learn about the big-picture considerations for real-world LLM/AI apps -
Embedding Fine-Tuning Guide for RAG Systems
By
–
RAG Embedding Fine-Tuning Guide A practical guide demonstrating embedding model fine-tuning for RAG systems, featuring LangSmith monitoring and RAGAS metrics for performance evaluation. Check out the complete implementation https://
github.com/apatti/AIEBoot
camp/blob/main/09_Finetuning_Embeddings/Fine_tuning_Embedding_Models_for_RAG_using_RAGAS.ipynb
… -
PyTorch Democratizes Fast Kernel Writing Challenge
By
–
we've been working on democratizing fast kernel writing on the @PyTorch team. try the challenge, either you or your AI!
-
The Danger of Overthinking in Large Reasoning Models
By
–
9). The Danger of Overthinking – This paper investigates overthinking in Large Reasoning Models (LRMs)—a phenomenon where models prioritize extended internal reasoning over interacting with their environment.
-
Inner Thinking Transformers: Scaling Reasoning in Small LLMs
By
–
10). Inner Thinking Transformers – A new method that enhances reasoning efficiency in small-scale LLMs via dynamic depth scaling. ITT aims to mitigate parameter bottlenecks in LLMs, providing scalable reasoning efficiency without expanding model size.
-
Open-Reasoner-Zero: Efficient RL Framework Outperforms DeepSeek-R1
By
–
7). Open-Reasoner-Zero – an open-source large-scale minimalist RL framework to enhance reasoning. Achieves significant scalability requiring only 1/30th of the training steps of DeepSeek-R1-Zero-Qwen-32B to outperform it on GPQA Diamond.
-
MoBA: New Attention Mechanism for Efficient Long-Context LLMs
By
–
8). MoBA – A new attention mechanism that enhances efficiency in handling long-context sequences for LLMs while maintaining strong performance.
-
Is Artificial Intelligence Replacing Human Learning?
By
–
Is Artificial Intelligence Turning Human Learning a Thing of the Past?
#AI #AIio #BigData #ML #NLU #Futureofwork @lexfridman @sama @kaifulee @ID_AA_Carmack @karpathy @2morrowknight @ylecun