8/ Mixture of Memory Experts – proposes an approach to significantly reduce hallucination (10x) by tuning millions of expert adapters (e.g., LoRAs) to learn exact facts and retrieve them from an index at inference time.
@dair_ai
-
Visual Chain of Thought: Sketching Framework for Multimodal LLM Reasoning
By
–
7/ Sketching as a Visual Chain of Thought – a framework that enables a multimodal LLM to access a visual sketchpad and tools to draw on the sketchpad; it can equip a model like GPT-4 with the capability to generate intermediate sketches to reason over complex tasks.
-
Transformers Meet Neural Algorithmic Reasoners Hybrid Architecture
By
–
5/ Transformers Meet Neural Algorithmic Reasoners – a new hybrid architecture that enables tokens in the LLM to cross-attend to node embeddings from a GNN-based neural algorithmic reasoner (NAR).
-
Self-Tuning LLMs Improve Knowledge Acquisition from Raw Documents
By
–
6/ Self-Tuning with LLMs – improves an LLM’s ability to effectively acquire new knowledge from raw documents through self-teaching.
-
Mixture-of-Agents: Multi-LLM Collaborative Layer Architecture
By
–
4/ Mixture-of-Agents – an approach that leverages the collective strengths of multiple LLMs through a Mixture-of-Agents methodology; layers are designed with multiple LLM agents and each agent builds on the outputs of other agents in the previous layers.
-
SelfGoal: LLM Agent Framework for Hierarchical Goal Achievement
By
–
3/ SelfGoal – a framework to enhance an LLM-based agent's capabilities to achieve high-level goals; adaptively breaks down a high-level goal into a tree structure of practical subgoals during interaction with the environment.
-
LLM-Driven Discovery of Preference Optimization Algorithms
By
–
2/ Discovering Preference Optimization Algorithms with LLMs – proposes LLM-driven objective discovery of state-of-the-art preference optimization.https://t.co/tXYiVCzW9S
— DAIR.AI (@dair_ai) 16 juin 20242/ Discovering Preference Optimization Algorithms with LLMs – proposes LLM-driven objective discovery of state-of-the-art preference optimization.
-
SimPO: Reference-Free Preference Optimization for Language Models
By
–
10/ SimPO – a simpler and more effective approach for preference optimization with a reference-free reward; uses the average log probability of a sequence as an implicit reward (i.e., no reference model required) which makes it more compute and memory efficient.
-
LLMs for Financial Analysis and Profitable Trading Strategies
By
–
9/ Financial Statement Analysis with LLMs – claims that LLMs can generate useful insights from its analysis of trends and financial ratios; shows that GPT-4 performs on par with narrowly specialized models; and achieves a profitable trading strategy based on GPT’s predictions.
-
Are Long-LLMs Really Necessary for Long-Context Tasks?
By
–
8/ Are Long-LLMs A Necessity For Long-Context Tasks? – claims that long-LLMs are not a necessity to solve long-context tasks…