This is rather profound Large language models shows signs of being capable of self recursively improving their code. This is both exciting and poses somewhat of a larger risk. What a time to be alive.
RESEARCH
-
Hinton’s Coursera Course Introduces RMSprop Algorithm
By
–
This was covered in Hinton's marvellous Coursera course btw. (Which also introduced rmsprop in passing)
-
World Models Progress: GAIA-1 Generative Model for Autonomous Driving
By
–
World Models have come a long way in 5 years! Generative world models like the recent GAIA-1 might eventually become one of the key ingredients for autonomous vehicles.
— hardmaru (@hardmaru) 7 octobre 2023
See: GAIA-1: A Generative World Model for Autonomous Drivinghttps://t.co/BnrhNnhL8K
👇https://t.co/Xe5aZvkqzrWorld Models have come a long way in 5 years! Generative world models like the recent GAIA-1 might eventually become one of the key ingredients for autonomous vehicles. See: GAIA-1: A Generative World Model for Autonomous Driving https://
arxiv.org/abs/2309.17080 -
Factuality in LLMs: Benchmarks and Knowledge Assessment
By
–
Nice paper by Tu Vu on factuality in LLMs: http://
arxiv.org/abs/2310.03214, enjoyed contributing in a minor role to it while I was at Google. The main takeaway for me is that most factuality benchmarks for LLMs don't really take into account the fact that many types of knowledge -
Google Gemini Surpasses GPT-4 by 5X in Performance
By
–
Google Gemini Eats The World – Gemini Smashes GPT-4 By 5X, The GPU-Poors https://
bit.ly/3PcLUtK #AI #MachineLearning #DeepLearning #LLMs #DataScience -
AnimateDiff ControlNet AI VFX Experiment Shows Incredible Results
By
–
Wtf 😱🤯
— Linus ✦ Ekenstam (@LinusEkenstam) 7 octobre 2023
AI VFX experiment: AnimateDiff + ControlNet
This is some incredible work by @CitizenPlain
Check thread 🧵
pic.twitter.com/NEKyXH8iIMWtf AI VFX experiment: AnimateDiff + ControlNet This is some incredible work by @CitizenPlain Check thread
-
Memento Mori: Deep Existential Topics in AI and Technology
By
–
Memento Mori — we could start with deep existential topics…
-
LoRA vs Full Fine-tuning: Combining Techniques for Model Optimization
By
–
I've been experimenting with that recently. It seems pretty similar. You can do both BTW – LORA for initial layers and full f/t for final
-
FreshLLMs: Improving LLM Factuality with Real-Time Knowledge
By
–
Excited to share FreshLLMs, our work that gives a “fresh” look of LLMs in the context of factuality! Our newly-curated FreshQA benchmark showed that LLMs still struggle on real-time knowledge and false-premise statements. More importantly, our technique FreshPrompt to incorporate
-
Freezing Layers and Discriminative Learning Rates in Fine-Tuning
By
–
Very happy to see more folks supporting what I've been saying for years — it really is a good idea to freeze layers when fine tuning. (And it's also a good idea to use discriminative learning rates.) See our ULMFiT paper for details (which is from 2018, but is still correct.)