One more argument: We could easily replace the sampling of the multinomial by a dynamical system, eg by herding https://
icml.cc/Conferences/20
09/papers/447.pdf
… I’m not sure randomness is needed. More on the opinion side of things: I don’t think humans are very far from being parrots. We may yet
@nandodf
-
Herding Algorithms and Human Intelligence in AI Systems
By
–
-
Language as Tool for Communication and Self-Reflection
By
–
Digesting these arguments with great interest. Thanks everyone. Here are some quick sketchy reflections to expand our debate: 1. Language is a tool for communication but it’s more than that. We turned the tool on ourselves to self reflect and think. Of course we still need
-
State of the Art Evaluation in Code Generation
By
–
A pretty accurate eval of the state of the art in code generation I think we’ve all been there
-
Next Word Prediction and Loss Functions for Model Understanding
By
–
This is a brilliant explanation of why next word prediction is a good loss function to achieve understanding. In addition to predicting the next bit, one can also use masking losses and diffusion to achieve understanding. The general intuition is to minimise the difference… https://t.co/tUDUQWP4NJ
— Nando de Freitas (@NandoDF) 25 mai 2025This is a brilliant explanation of why next word prediction is a good loss function to achieve understanding. In addition to predicting the next bit, one can also use masking losses and diffusion to achieve understanding. The general intuition is to minimise the difference
-
AI Scaling Laws and the Critical Value of Data Discussion
By
–
This is a wonderful thread on AI scaling laws and the value of data
-
Veo 3 Generative AI Video Creation Tool Impresses Community
By
–
Congrats on another amazing delivery @avdnoord – Veo 3 is super impressive
-
Diffusion Models: Generation, Guidance, and Flow Matching Techniques
By
–
I’ve been away this week, but next week we re-start the diffusion lectures! We’ll be doing generation, classifier (free) guidance, flow matching with ODEs, and some pretty cool math What an amazing week for diffusion: text diffusion, jaw dropping video models, high fidelity
-
Microsoft AI Team Ships Multimodal Educational Platform
By
–
After 6 months at
@MicrosoftAI
our small multimodal AI team has shipped! We’re having fun, but also thinking deeply about how to empower users with fun educational material -
Paradox Checker: Tool Use and the Halting Problem
By
–
One of my favourite examples of tool use: def paradox_checker(): if halts(paradox_checker): while True: pass else: return "Halted" I also like to ask my daughters to grab pi. They know it’s tricky to pick certain kinds of numbers – they
-
ChatGPT Video Tutorial: Watch and Share the Prompt
By
–
ChatGPT dude! https://
youtu.be/QGKq8NHbPAY?si
=LtfGRtygohAd7M9l
… Share the prompt too