Very clever. And matches what I would expect: models are weak on vision relative to everything else, so visual steps are where errors accumulate most in workflows.
MACHINE LEARNING
-
Model distillation: the most powerful are American, interesting examples
By
–
The reality is that everyone distills from everyone, usually from the most powerful, and the most powerful right now are American models. Here are some interesting examples.
-
Stanford HAI director Fei-Fei Li featured on FastCompany cover explaining world models
By
–
HAI Founding Director @drfeifei is featured on @FastCompany
's cover, explaining "world models" – AI that understand physical space and real-world dynamics. Rooted in human-centered philosophy, she explains what makes it different and what's at stake: -
Distillation with logits: obsolete technique, generate synthetic data
By
–
But thinking about 'distillation' only for the strict case of having access to logits is very old-school, a problem from the time when OpenAI cut off access to logits from the API. I am not up to date on what techniques exist and their effectiveness, but generate synthetic data,
-
Train LLM from scratch and AI Engineering book resources
By
–
Train LLM from scratch: https://
github.com/FareedKhan-dev
/train-llm-from-scratch
…
—
AI Engineering book: https://
dailydoseofds.github.io/ai-engg-book/ -
Build a GPT-style transformer from scratch without high-level libraries
By
–
Train your own LLM from scratch. This repo builds a GPT-style transformer from the ground up, without using any high-level libraries. You see exactly how attention, multi-head attention, the feed-forward block, embeddings, residuals, and layer norm fit together. And it doesn't
-
AI solves 7/10 hard math problems but still criticized
By
–
Weird headline – I am not sure solving 7 out of 10 novel very hard problems meant AI "did not live up to the task," when 15 months ago LLMs couldn't do math. But the actual study is interesting and illuminates flaws & successes of AIs in math. https://
1stproof.org/assets/docs/re
port.pdf
… -
Model neutrality surpasses cloud neutrality as an offensive tool
By
–
Why model neutrality is more important than cloud neutrality was In the cloud era, companies opted for cloud neutrality as a defense mechanism Model neutrality can be an **offensive** mechanism. It is more
-
Deleted tweet: API users misunderstand frontier model power in native harnesses
By
–
Deleted a tweet on the fact that API users don't understand how much more powerful the frontier models are in native harnesses since I didn't differentiate in the post (limited characters!) between folks carefully evaluating other harnesses for tasks & those just using naked API.
-
W-Flow: Single-step image generation via Wasserstein gradient flow
By
–
What if you could generate high-quality images in one step instead of hundreds? Stanford and ByteDance introduce W-Flow: a single-step generator that turns random noise into target data by following a Wasserstein gradient flow. It compresses a full diffusion-like evolution into