LLMS

Transformers have dominated LLM text generation, and generate tokens sequentially. This is a cool attempt to explore diffusion models as an alternative, by generating the entire text at the same time using a coarse-to-fine process. Congrats @StefanoErmon & team! https://t.co/HUL9Lb9buX

— Andrew Ng (@AndrewYNg) 27 février 2025

Transformers have dominated LLM text generation, and generate tokens sequentially. This is a cool attempt to explore diffusion models as an alternative, by generating the entire text at the same time using a coarse-to-fine process. Congrats @StefanoErmon & team!

→ View original post on X — @andrewyng

This is interesting as a first large diffusion-based LLM.

Most of the LLMs you've been seeing are ~clones as far as the core modeling approach goes. They're all trained "autoregressively", i.e. predicting tokens from left to right. Diffusion is different – it doesn't go left to… https://t.co/I0gnRKkh9k

— Andrej Karpathy (@karpathy) 27 février 2025

This is interesting as a first large diffusion-based LLM. Most of the LLMs you've been seeing are ~clones as far as the core modeling approach goes. They're all trained "autoregressively", i.e. predicting tokens from left to right. Diffusion is different – it doesn't go left to

→ View original post on X — @karpathy