Very early on in the project I did a small run with/without QK norm and found that it helped. Same for the embedding weight sharing. I'll retry! I'm not tied to any details of the model and they weren't chosen any more carefully than a single run, I spent most of the time just
GENERATIVE AI
-
Advanced Llama Architecture: Rotary Embeddings and ReLU² MLP
By
–
> llama-like architecture
> dense transformer > rotary only (no positional embeddings) > qk norm > untied embedding/unembedding > norm after token embedding > relu² mlp > no biases in linears > no learnable rmsnorm params > mqa > logit softcap > optimizer = -

Karpathy releases new LLM training project under 8000 lines
By
–
> karpathy just released a new project > in it, and in under 8000 lines of code you get to: > train the tokenizer using a new rust implementation
> pretrain a transformer llm on fineweb, evaluate core score across a number of metrics > midtrain on user-assistant -
Humanize Feature: Transform AI Writing Into Natural Human Sound
By
–
Try our Humanize feature and turn AI writing into something that actually sounds human. pic.twitter.com/uMLLJub36c
— Abacus.AI (@abacusai) 13 octobre 2025Try our Humanize feature and turn AI writing into something that actually sounds human.
-

Instant4D: Fast 4D Gaussian Splatting from Sora Videos
By
–
Instant4D
— AK (@_akhaliq) 13 octobre 2025
4D Gaussian Splatting in Minutes
videos display the reconstuction results from the sora generated videos. Each video is 9:16 aspect ratio with 5 seconds duration. The reconstruction process for each video takes around 5 minutes including preprocessing time pic.twitter.com/41yfckgJptInstant4D 4D Gaussian Splatting in Minutes videos display the reconstuction results from the sora generated videos. Each video is 9:16 aspect ratio with 5 seconds duration. The reconstruction process for each video takes around 5 minutes including preprocessing time
-

StreamingVLM: Real-Time Video Understanding for Infinite Streams
By
–
StreamingVLM
— AK (@_akhaliq) 13 octobre 2025
Real-Time Understanding for Infinite Video Streams pic.twitter.com/Sj6Uh80yR2StreamingVLM Real-Time Understanding for Infinite Streams
-

Preparing Physicians for the Generative AI Era
By
–
A concise viewpoint on how physicians can prepare for the generative A.I. era https://
jamanetwork.com/journals/jamai
nternalmedicine/fullarticle/2840192
… @JAMAInternalMed -
ChatKit Evals Agent Builder Beta Release and RFT Expansion
By
–
Where it stands today: ChatKit & Evals are broadly available. Agent Builder & Connector Registry are in beta / limited release, especially for enterprise / internal users. Reinforcement fine-tuning (RFT) for reasoning & custom tool use is being expanded.
-
AI Agents Drive Growth: Klarna, Clay, OpenAI Platform
By
–
Use cases & early signals: Klarna built a support agent handling ~2/3 of ticket volume with an earlier agentic workflow. Clay reportedly saw 10× growth using a sales agent. ChatGPT said: The move makes OpenAI a central AI agent platform — an “app store for agents.”
-

OpenAI AgentKit Empowers Developers Building Agentic Workflows
By
–
AI agents just got a major power-up.
Meet AgentKit — OpenAI’s new unified toolkit built to help developers build, deploy, and optimize agentic workflows faster and more reliably.