5/ Contextual Hallucinations Mitigation in LLMs – proposes a new method that detects and significantly reduces contextual hallucinations in LLMs (e.g., reduces by 10% in the XSum summarization task)…
@dair_ai
-
Mixture of Million Experts: Sparse Parameter-Efficient Expert Retrieval
By
–
3/ Mixture of A Million Experts – introduces a parameter-efficient expert retrieval mechanism that leverages the product key technique for sparse retrieval from a million tiny experts.
-
LLM Reasoning: Geometric Perspective and Expressive Capacity
By
–
4/ Reasoning in LLMs: A Geometric Perspective – explores the reasoning of LLMs from a geometrical perspective; reports that a higher intrinsic dimension implies greater expressive capacity of the LLM.
-
FlashAttention-3 Optimizes GPU Attention Operations for Modern Hardware
By
–
1/ FlashAttention-3 – proposes to adapt FlashAttention to take advantage of modern hardware; the techniques used to speed up attention on modern GPUs include producer-consumer asynchrony, interleaving block-wise matmul and softmax operations, and block quantization and incoherent
-
RankRAG: Instruction Fine-Tuning Framework for LLM RAG Enhancement
By
–
2/ RankRAG – introduces a new instruction fine-tuning framework to perform effective context ranking and answering generation to enhance an LLM’s RAG capabilities.
-
Top ML Papers of the Week: RankRAG, RouteLLM, FlashAttention-3
By
–
The Top ML Papers of the Week (July 8 – July 14): – RankRAG
– RouteLLM
– FlashAttention-3
– Internet of Agents
– Learning at Test Time
– Mixture of A Million Experts
… -
AI Agents Evaluation Framework Optimizing Cost and Accuracy
By
–
10/ AI Agents That Matter – analyzes current agent evaluation practices and reveals shortcomings that potentially hinder real-world application; proposes an implementation that jointly optimizes cost and accuracy and a framework to avoid overfitting agents.
-
Ctrl-G Framework: Logical Control for LLM Generation
By
–
7/ Adaptable Logical Control for LLMs – presents the Ctrl-G framework to facilitate control of LLM generations that reliably follow logical constraints; it combines LLMs and Hidden Markow Models to enable following logical constraints (represented as deterministic finite
-
SummHay: New Benchmark for Haystack Summarization and Source Citation
By
–
9/ Summary of a Haystack – proposes a new task, SummHay, to test a model’s ability to process a Haystack and generate a summary that identifies the relevant insights and cites the source documents.
-
Synthetic Data Effects on LLM Bias and Internal Attributes
By
–
8/ LLM See, LLM Do – closely investigates the effects and effectiveness of synthetic data and how it shapes a model’s internal biases, calibration, attributes, and preferences; finds that LLMs are sensitive towards certain attributes even when the synthetic data prompts appear