Mildly obsessed with what the "highest grade" pretraining data stream looks like for LLM training, if 100% of the focus was on quality, putting aside any quantity considerations. Guessing something textbook-like content, in markdown? Or possibly samples from a really giant model?
RESEARCH
-
Reasoning Models Revolution: New Scaling Laws Transform LLM Capabilities
By
–
"How many R’s are in strawberry?"
A year ago, ChatGPT would have guessed—or hallucinated. Today, it thinks. I just released a new video diving into the "revolution" of reasoning models and the new axis of scaling laws that's transforming how large language models operate. -

Master Contrastive Prompting for AI Success
By
–
I’ve mastered contrastive prompting.
It amplifies my AI prompt success.
• Compare prompts side-by-side
• Use real examples
• Optimize your approach Click below to read more: https://
godofprompt.ai/blog/contrasti
ve-prompting
… -

Neural Conversational Model Paper Turns 10 Years Old
By
–
"A Neural Conversational Model" is 10 years old, w/ @quocleix . TL;DR you can train a chatbot with a large neural network (~500M params!). Samples This paper was received with mixed reviews, but I'm glad all the critics are now riding the LLM wave https://
arxiv.org/abs/1506.05869 -
Property Inference Advances in Machine Learning Research
By
–
yeah, agree that that's a nice relaxation of this problem. this work focuses on property inference:
-
New Research Reveals Hidden Human Costs of ChatGPT Usage
By
–
But, now, read the new research on the hidden human cost of using ChatGPT:
-

Expert Knowledge Transformation Into AI Data Innovation
By
–
We turn expert knowledge into AI data. Meet Jake Shaver, who helps experts find new paths into AI and drives innovation behind the scenes. #AI #SnorkelAI
-
WSDM 2009 Talk Slides on Google Search System Evolution
By
–
A set of slides from a talk I gave at WSDM 2009 describes some aspects of our search system evolution up to then: https://
static.googleusercontent.com/media/research
.google.com/en//people/jeff/WSDM09-keynote.pdf
… of talk: -

Mistral Small 3.2 minor upgrade released
By
–
Mistral AI released Mistral Small 3.2 with a minor upgrade for instruction following, repetition errors and function calling!