This combination of formal language types and informal types via LLMs is a really interesting and thought provoking programming framework. Nice work
@AdaptiveAgents
@nandodf
-
Formal and Informal Language Types in LLM Programming Framework
By
–
-
Discrete Tokens, Replication, and Entropy in Life and Language
By
–
Agree, I think discrete tokens win when robust (self-)replication is necessary to fight entropy. That seems to be the case with life (DNA) and language.
-
PAQ8 bit sequences language models efficiency 2010
By
–
I agree, but it’s tricky for models to learn this. It would make a good PhD thesis Curiously, the old PAQ8 language models back in 2010 used sequences of bits, as you suggest, because we had to rely on binary C operations to make them efficient enough. See
-
Continuous Vectors vs Large-Scale Quantisation in AI Models
By
–
I strongly agree with all your comments. Thanks for sharing. One question: if researchers figure out how to do continuous vectors, do you think that will be better than large-scale (or residual) quantisation? Why?
-
Gato Architecture: The Future of Multimodal AI Models?
By
–
Meta Chameleon and Gemini have also adopted the Gato ( https://
arxiv.org/pdf/2205.06175 ) architecture. Is this going to be the ultimate approach for MIMO (multimodal input multimodal output models) or is there something else we should be trying? There’s been great progress in scaling -
Micromanagement vs Engineering Detail in AI Leadership
By
–
I loved this podcast with @elonmusk
. He’s just about right. I especially liked his comment about micromanagement. I too have been told this, but I feel micromanagement is often confused with paying attention to engineering detail. In AI and tech, VPs should be able to sit with -
Evolution of Ideas in Groundbreaking AI Research Papers
By
–
Hi Neil, I learned about this nice paper from reading your *truly* groundbreaking papers. I wasn’t sure who introduced the idea from reading the related work sections. Without intending to put you on the spot, could you please provide us with a short note on the evolution of
-
AlphaGo Tuning with Bayesian Optimization Improved Win Rate
By
–
I agree with this thread of @avt_im
. Prior to the match with Lee Sedol, we tuned the latest AlphaGo agent with Bayesian Optimization and this improved its win-rate from 50% to 66.5% in self-play games. This tuned version was deployed in the final match. See -
Residual Quantisation: VQ-VAE Improvement for Data Compression
By
–
This is a great K-AI paper on residual quantisation. It is an improvement over VQ-VAE. In short, after quantising the data (images, video, audio) one also quantizes the resulting error to get a finer aggregate description, which is then passed to the decoder. @kchonyc I’m
-
OpenAI’s Inclusive Approach to Technology Presentation and Innovation
By
–
One thing I love about @OpenAI’s presentation of their latest technology is that they include researchers and engineers. They also welcome failure and live demos, which are essential to make progress. Well done on inclusion and growth mentality. https://t.co/QRJQTyToyK
— Nando de Freitas (@NandoDF) 15 mai 2024One thing I love about @OpenAI
’s presentation of their latest technology is that they include researchers and engineers. They also welcome failure and live demos, which are essential to make progress. Well done on inclusion and growth mentality.