If you measure downstream performance on HellaSwag rather than speedrun-equivalent loss, then different tokenizer approaches come out on top… The first run I did was much better on common-sense downstream, trained in equivalent time or better.
LLMS
-
Tokenizer Training and Data Filtering Compliance Standards
By
–
Well, the tokenizer I used was trained on large quantities of data — I filtered the tokens based on yet more data from FineWeb. Question is if that's acceptable according to your rules…
-
Gemini 3 Preview Spotted on Vertex AI Platform
By
–
BREAKING : gemini-3-pro-preview-11-2025 has been spotted on Vertex AI (Not available yet). What we know so far – A preview version of Gemini 3 will be coming in November with 1M context window support. https://
x.com/legit_api/stat
/legit_api/status/1986066955191665043
… -
Celebrating shift from JSON tool calls to code blobs
By
–
Finally! Congrats, a step in a great direction!
We've been stuck in this local optimum of "writing actions as JSON blobs with individual tool calls in them" for much too long. Just let models write tool calls in code blobs! -
Retokenization and Language Knowledge in Model Training
By
–
The biggest question is whether you allow re-tokenization, and whether that should be done with the same data as the training itself. Right now there is knowledge about the language in existing tokens built-in and changing that is against the rules and/or unfavorable.
-
Rediscovery of CodeAct and smolagents a year later
By
–
I think they're finally going to rediscover CodeAct and smolagents, 1 year later!
-
ThinkMorph: Emergent Properties in Multimodal Chain-of-Thought
By
–
ThinkMorph: Emergent Properties in Multimodal Interleaved Chain-of-Thought Reasoning Gu et al.: https://
arxiv.org/abs/2510.27492 #ArtificialIntelligence #DeepLearning #MachineLearning -
Google NotebookLM to add custom styles for video overviews
By
–
BREAKING : Google is working on Custom Styles for Overviews on NotebookLM. This option will let users set a custom prompt defining a visual style of the video. This also potentially means that Overviews will be powered by Nano Banana 2 very soon!
-
Building Efficient AI Agents with Model Context Protocol
By
–
New on the Anthropic Engineering blog: tips on how to build more efficient agents that handle more tools while using fewer tokens. Code execution with the Model Context Protocol (MCP):
-
Codex CLI Pull Requests Growth Compared to Gemini CLI
By
–
Counting PRs is a little silly, but there's a huge shift in the Codex CLI PRs since August – though still behind Gemini CLI PRs, which I did not expect. New Codex model would be pretty cool to have.