So it's extra grating to read that we "likened AI to “a hyper-intelligent deep-sea octopus.”" >>
RESEARCH
-
Octopus Intelligence and Language Understanding Beyond Form
By
–
2. The octopus is posited to be hyperintelligent precisely because we wanted to show that no entity could learn to "understand" language based on form alone; it's doesn't matter what's inside the box. >>
-
Language Models Are Not AI: Clarifying the Terminology
By
–
1. We were NOT talking about "AI". That's a paper about language models, and even though other people are (now) talking about LLMs as if they were AI, we weren't and never would. >>
-
Octopus Thought Experiment: Critical Analysis of AI Understanding
By
–
This is a decent summary of the octopus thought experiment from Bender & @alkoller 2020, with two glaring exceptions, right at the start: https://
techcrunch.com/2024/06/01/wtf
-is-ai/
… >> -
SimPO: Reference-Free Preference Optimization for Language Models
By
–
10/ SimPO – a simpler and more effective approach for preference optimization with a reference-free reward; uses the average log probability of a sequence as an implicit reward (i.e., no reference model required) which makes it more compute and memory efficient.
-
Aya23: Multilingual Language Model Excelling in 23 Languages
By
–
7/ Aya23 – a family of multilingual language models that can serve up to 23 languages; it intentionally focuses on fewer languages and allocates more capacity to these languages; shows that it can outperform other massive multimodal models on those specific languages.
-
Are Long-LLMs Really Necessary for Long-Context Tasks?
By
–
8/ Are Long-LLMs A Necessity For Long-Context Tasks? – claims that long-LLMs are not a necessity to solve long-context tasks…
-
GNN-RAG: Combining LLMs with Graph Neural Networks
By
–
5/ GNN-RAG – combines the language understanding abilities of LLMs with the reasoning abilities of GNNs in a RAG style; the GNN extracts useful and relevant graph information while the LLM takes the information and leverages its capabilities to perform question answering over
-
Attention as RNN: Parallel Training with Constant Memory Inference
By
–
6/ Attention as an RNN – presents a new attention mechanism that can be trained in parallel (like Transformers) and be updated efficiently with new tokens requiring constant memory usage for inferences (like RNNs).
-
Vision-Language Modeling: Introduction and Effective Training Methods
By
–
4/ Introduction to Vision-Language Modeling – presents an introduction to vision-language models along with key details of how they work and how to effectively train these models.