http://
rodneybrooks.com/blog post covering some of my opening keynote on Nov 28th for MIT's Generative AI Week is now up. This is one of the topics I talked about–the impact of the existence of LLMs on three long running debates in AI & linguistics. I have this handle at bsky social.
LLMS
-
LLMs Impact on AI and Linguistics Debates
By
–
-
Mixture of Experts Routing History in Language Models
By
–
The idea of MoE/routing for LMs has a long history (e.g. Fig 4 of this 2010 multimodal PAQ LM https://
arxiv.org/pdf/1108.3298.
pdf
…) but executions and not just the idea matter a lot, and we should welcome every advance. Matt Mahoney attributes the idea to @SchmidhuberAI -
LLMCompiler: Open Source Framework for Parallel LLM Agent Task Execution
By
–
New Paper Alert -LLMCompiler If you enjoyed @karpathy
's video on an "LLM OS", you're going to want to check out this paper (complete with a full open source repo!) This system compiles an effective plan for executing multiple tasks in parallel, letting LLM agents work -
Better Coding With ChatGPT: Your AI Pair Programming Partner
By
–
Better Coding With ChatGPT
— Louis-François Bouchard 🎥🤖 (@Whats_AI) 9 décembre 2023
Talk with it and communicate your problems, ask questions, brainstorm, and debug. Use it as a friend coding with you. ChatGPT is the evolution of rubber duck debugging.
With @OfficialLoganK, Developer Relations @OpenAI. pic.twitter.com/c6mgwiakJkBetter Coding With ChatGPT Talk with it and communicate your problems, ask questions, brainstorm, and debug. Use it as a friend coding with you. ChatGPT is the evolution of rubber duck debugging. With @OfficialLoganK
, Developer Relations @OpenAI
. -
Seed Averaging: Essential Technique for Deployed LLM Models
By
–
This is why I still believe that seed averaging is a MUST for even the deployed models. It helps with accuracy AND reliability. Of course, it also requires more compute, but that might be the only way to overcome the current limitations of the LLM-based products.
-
Test of Grok, Elon Musk’s AI, compared to ChatGPT
By
–
#Grok, Elon Musk's #AI is AVAILABLE So what is it worth? Well, I do the full test in my latest video →
https://youtu.be/yy9AaZx32W4 Program:
– How to access it?
– How does it work?
– Tests and comparisons with #chatGPT Poke @ElonMusk: the 1st test of -
AI Self-Talk: The Evolution of RLHF and Instruction Following
By
–
"Let's think step by step" is the best-known instance of this, but people forget the phrase isn't instruction, it's self-talk — before RLHF the distinction mattered:
-
Prompt Engineering Techniques for Steering AI Model Behavior
By
–
Exhibit A in why prompt engineering is more fun with old-school completion vs. newfangled ChatML APIs — if you put some care into avoiding tokenization gotchas, "speaking for the model" is the most reliable way to steer it:
-
Universal Self-Adaptive Prompting for Zero-Shot LLM Learning
By
–
Visit the #EMNLP2023 Google booth today at 3:30 PM to learn about Universal Self-Adaptive Prompting, an automatic method tailored for zero-shot learning (while compatible with few-shot) that uses a small amount of unlabelled data & an inference-only LLM. pic.twitter.com/m8tRtXt1pL
— Google AI (@GoogleAI) 9 décembre 2023Visit the #EMNLP2023 Google booth today at 3:30 PM to learn about Universal Self-Adaptive Prompting, an automatic method tailored for zero-shot learning (while compatible with few-shot) that uses a small amount of unlabelled data & an inference-only LLM.
-
Sequential Data Generation as Machine Learning’s Form of Dreaming
By
–
“Generating sequential data is the closest computers get to dreaming.” Seems to be the consensus among ML researchers for the past decade