The battle is who will be the first to get a language model to alphabetically sort this list of words: ['apple', 'ant', 'air', 'album', 'alligator', 'anaconda', 'asteroid', 'astronaut', 'anchor', 'arch'].
@_jasonwei
-
LLMs and Human Language: A Profound Insight from Stanford Research
By
–
Clever @google
/
@stanford paper on LLMs from my brother @jerryweiAI
. Performance boosts are great, but there is a more profound insight in this paper that was not explicitly stated: LLMs are trained on human language, but due to the nature of how language was developed (first -
AI-Powered Immersive Virtual Worlds for Entertainment and Learning
By
–
Within the next couple of years, I expect we will see the emergence of highly immersive multiplayer virtual worlds that provide opportunities for entertainment, social connection, learning. They will become a powerful new medium for interactive storytelling, as AI gets
-
Dating Preferences for Advanced Language Models with Large Context Windows
By
–
i only date language models with six-figure context window
-
Compute scaling requires meaningful tokens, not arbitrary generation
By
–
It's not just about compute though, the tokens have some meaning to them. It doesn't work to just get the model to generate arbitrary tokens to increase compute. I would be skeptical of this working even if you trained the model on such examples.
-
Study on Chain-of-Thought Faithfulness and Model Bias
By
–
Really cool paper studying the faithfulness of chain-of-thought (CoT): http://
arxiv.org/abs/2305.04388 The paper uses a biased prompt to try to mislead the model. For example, all few-shot exemplars could be answer (A), or they could add a suffix such as "I think the answer is but -
Fragmentation of humanity through personalized AI linguistic spheres
By
–
Thought-provoking blog post: https://
ribbonfarm.com/2023/05/04/lif
e-after-language/
… It's not implausible that, in the coming decades, humanity could fragment into eight billion discrete linguistic spheres, each person conversing with a tailored AI in a private idiom. As a side effect, our inner monologues -
Ancient Chinese Knowledge and Modern Neural Networks Connection
By
–
Turns out the ancient Chinese knew a lot about modern neural networks
-
Chain-of-Thought Prompting: Does Incorrect CoT Hurt Performance?
By
–
A question I get asked a lot about chain-of-thought (CoT) prompting is whether having incorrect CoT will hurt performance. This implicit research question here is basically whether the model is really learning how to reason from the CoT example, or if CoT exemplars elicit
-
Google’s non-competitive research environment with collaborative multi-author papers
By
–
FWIW, Google isn't a pressure cooker. There are no incentives for # papers or conference acceptances.
The reason for this is our papers have many co-authors. This joke was well-received internally, but I lacked the awareness to keep my mouth shut publicly (again, apologies).