8/ Internet of Agents – a new framework to address several limitations in multi-agent frameworks such as integrating diverse third-party agents and adaptability to dynamic task requirements; introduces an agent integration protocol, instant messaging architecture design, and
@dair_ai
-
3DGen: Fast Text-to-3D Asset Generation Pipeline
By
–
9/ 3DGen – a new pipeline for end-to-end text-to-3D asset generation in under a minute; integrates state-of-the-art components like AssetGen and TextureGen to represent 3D objects in three ways, namely view space, in volumetric space, and in UV space; achieves a win rate of 68%
-
Mixture of Experts Survey: Technical Details and Applications
By
–
7/ A Survey on Mixture of Experts – a survey paper on Mixture of Experts (MoE), including the technical details of MoE, open-source implementations, evaluation techniques, and applications of MoE in practice.
-
RouteLLM: Efficient Router Models for Cost-Performance Balance
By
–
6/ RouteLLM – proposes efficient router models to dynamically select between stronger and weak LLMs during inference to achieve a balance between cost and performance; the training framework leverages human preference data and data augmentation techniques to boost performance;
-
Reducing Contextual Hallucinations in Large Language Models
By
–
5/ Contextual Hallucinations Mitigation in LLMs – proposes a new method that detects and significantly reduces contextual hallucinations in LLMs (e.g., reduces by 10% in the XSum summarization task)…
-
Mixture of Million Experts: Sparse Parameter-Efficient Expert Retrieval
By
–
3/ Mixture of A Million Experts – introduces a parameter-efficient expert retrieval mechanism that leverages the product key technique for sparse retrieval from a million tiny experts.
-
LLM Reasoning: Geometric Perspective and Expressive Capacity
By
–
4/ Reasoning in LLMs: A Geometric Perspective – explores the reasoning of LLMs from a geometrical perspective; reports that a higher intrinsic dimension implies greater expressive capacity of the LLM.
-
RankRAG: Instruction Fine-Tuning Framework for LLM RAG Enhancement
By
–
2/ RankRAG – introduces a new instruction fine-tuning framework to perform effective context ranking and answering generation to enhance an LLM’s RAG capabilities.
-
FlashAttention-3 Optimizes GPU Attention Operations for Modern Hardware
By
–
1/ FlashAttention-3 – proposes to adapt FlashAttention to take advantage of modern hardware; the techniques used to speed up attention on modern GPUs include producer-consumer asynchrony, interleaving block-wise matmul and softmax operations, and block quantization and incoherent
-
Top ML Papers of the Week: RankRAG, RouteLLM, FlashAttention-3
By
–
The Top ML Papers of the Week (July 8 – July 14): – RankRAG
– RouteLLM
– FlashAttention-3
– Internet of Agents
– Learning at Test Time
– Mixture of A Million Experts
…