LS Paper Club: LLM Post-Training: A Deep Dive into Reasoning Large Language Models
MACHINE LEARNING
-
Llama Models Power India’s First Open Source Audio Language Model
By
–
Llama models were used to develop India's first open source audio language model — Shuka v1 https://
go.fb.me/4bbs7t -
Pareto Analysis of High-Capability AI Models at Competitive Pricing
By
–
Love me a good Pareto graph: our focus on highly capable, efficient models at a great price looks pretty good on these (note the logarithmic X axis).
-
Benchmarks and Competitions Drive Scientific Progress
By
–
Benchmarks and competitions are an effective way to drive scientific progress.
-
AI Won’t Automate Programming: Why Learning Code Still Matters
By
–
Some people today are discouraging others from learning programming on the grounds AI will automate it. This advice will be seen as some of the worst career advice ever given. I disagree with the Turing Award and Nobel prize winner who wrote, “It is far more likely that the
-
Auditing AI Models Through Interpretability and Sparse Autoencoders
By
–
We audited this model using training data analysis, black-box interrogation, and interpretability with sparse autoencoders. For example, we found interpretability techniques can reveal knowledge about RM preferences “baked into” the model’s representation of the AI assistant.
-
RAG Techniques: Improving LLM Retrieval Systems with Embeddings
By
–
This episode of Data Brew goes in-depth into RAG, tips and tricks, and how the technique address limitations of LLMs. Databricks Research Scientist Andrew Drozdov chats about:
– Improving retrieval systems with embeddings and fine-tuning techniques
– Enhancing search results -
Gemma 3 Analysis: Large Vocabulary Size and Model Insights
By
–
Just read through the Gemma 3 report and toyed around with the models a bit, and there are a bunch of interesting tidbits: 1. Vocab size. They again use a very large vocab: 262k token (in contrast, Llama 3 has ~1/2 the vocab size), which should make the model more friendly for
-
Negative Results in ML Research Should Be Encouraged and Published
By
–
I strongly disagree with you, Damien. Publishing negative results in ML research should be encouraged, not frowned upon. Negative results are not "cheap, a dime a dozen" but are important results to help advance the field. https://
sakana.ai/ai-scientist-f
irst-publication/#fn:3
… -
MIT Ranked #1 in Computer Science, Data Science and AI
By
–
QS ranks MIT #1 in computer science, data science and AI, electrical and electronic engineering, & more: https://
bit.ly/4kNlbCT