4). A Survey on Data Synthesis and Augmentation for LLMs – provides a comprehensive summary of data generation techniques in the lifecycle of LLMs; includes discussions on data preparation, pre-training, fine-tuning, instruction-tuning, preference alignment, and applications.
@dair_ai
-
LongRAG Enhances RAG Understanding Long-Context Knowledge
By
–
5). LongRAG – enhances RAG's understanding of long-context knowledge which includes global information and factual details; consists of a hybrid retriever, an LLM-augmented information extractor, a CoT-guided filter, and an LLM-augmented generator…
-
Agentic Information Retrieval: LLM Agent Applications
By
–
1). Agentic Information Retrieval – provides an introduction to agentic information retrieval, which is shaped by the capabilities of LLM agents; discusses different types of cutting-edge applications of agentic information retrieval and challenges.
-
Aya Expanse: Multilingual Foundation Models 8B and 32B Parameters
By
–
2). Aya Expanse – a family of open-weight foundation models for multilingual capabilities; releases an 8B and 32B parameter model, including one of the largest multilingual dataset collections to date, with 513 million examples… https://t.co/LWH49PJKZ0
— DAIR.AI (@dair_ai) 27 octobre 20242). Aya Expanse – a family of open-weight foundation models for multilingual capabilities; releases an 8B and 32B parameter model, including one of the largest multilingual dataset collections to date, with 513 million examples…
-
Top ML Papers of the Week: LongRAG, Granite 3.0, and Beyond
By
–
The Top ML Papers of the Week (Oct 21 – 27): – LongRAG
– Granite 3.0
– Data Synthesis Overview
– Agentic Information Retrieval
– Scalable Watermarking for LLMs
– A Theoretical Understanding of CoT Read on for more: -
OpenAI o1 Models Planning Abilities Self-Evaluation Constraints
By
–
9). On the Planning Abilities of OpenAI’s o1 Models – reports that o1-preview is particularly strong in self-evaluation and constraint-following; also mentions that these o1 models demonstrate bottlenecks in decision-making and memory management, which are more pronounced in
-
CoTracker3: Semi-Supervised Point Tracking Model with Pseudo-Labels
By
–
10). CoTracker3 – proposes a new point tracking model and a new semi-supervised training recipe; enables usage of real videos without annotations during training by generating pseudo-labels using off-the-shelf teachers.https://t.co/vCbJzcXSPj
— DAIR.AI (@dair_ai) 20 octobre 202410). CoTracker3 – proposes a new point tracking model and a new semi-supervised training recipe; enables usage of real videos without annotations during training by generating pseudo-labels using off-the-shelf teachers.
-
Model Kinship Strategy for Improved LLM Merging
By
–
8). Model Kinship for Merging LLMs – proposes model kinship to measure the degree of similarity between LLMs; model kinship is used to build a model merging strategy (Top-k Greedy Merging with Model Kinship) which yields better performance.
-
Agent S: Open Framework Enables Autonomous Computer Interaction
By
–
7). Agent S – a new open agentic framework that enables autonomous interaction with computers through a GUI; Agent S tackles challenges such as acquiring knowledge, planning over long-task horizons, and handling dynamic interfaces.
-
Inference Scaling Laws for Long-Context RAG Systems
By
–
6). Inference Scaling for Long-Context RAG – uses two strategies to investigate scaling laws for RAG: in-context learning and iterative prompting; RAG performance improves with the expansion of the effective context length under optimal configurations.