4/ LLaMA Pro – proposes a post-pretraining method to improve an LLM’s knowledge without catastrophic forgetting; it achieves this by tuning expanded identity blocks using only new corpus while freezing the inherited blocks.
@dair_ai
-
Mobile ALOHA: Bimanual Mobile Manipulation via Low-Cost Teleoperation
By
–
1/ Mobile ALOHA – proposes a system that learns bimanual mobile manipulation with low-cost whole-body teleoperation; it first collects high-quality demonstrations and then performs supervised behavior cloning.https://t.co/Vf4REkMBuW
— DAIR.AI (@dair_ai) 7 janvier 20241/ Mobile ALOHA – proposes a system that learns bimanual mobile manipulation with low-cost whole-body teleoperation; it first collects high-quality demonstrations and then performs supervised behavior cloning.
-
32 Techniques to Mitigate Hallucination in Large Language Models
By
–
2/ Mitigating Hallucination in LLMs – summarizes 32 techniques to mitigate hallucination in LLMs; introduces a taxonomy categorizing methods like RAG, Knowledge Retrieval, CoVe, and more.
-
Top ML Papers of the Week: DocLLM, ALOHA, Fine-tuning
By
–
The Top ML Papers of the Week (Jan 1 – Jan 7): – DocLLM
– Mobile ALOHA
– Self-Play Fine-tuning
– Fast Inference of MoE
– LLM Augmented LLMs
– Mitigating Hallucination in LLMs
… -
LLaRA: Adapting LLMs for Dense Retrieval Tasks
By
–
9/ Making LLMs Better at Dense Retrieval – proposes LLaRA which adapts an LLM for dense retrieval; LLaMa-2-7B was improved on benchmarks like MSMARCO and BEIR.
-
Gemini vs GPT-4V: Vision-Language Models Comparison
By
–
10/ Gemini vs. GPT-4V – a comparison of vision-language models like Gemini & GPT-4V; finds that GPT-4V is precise and succinct in responses, while Gemini excels in providing detailed, expansive answers accompanied by relevant imagery and links.
-
26 Principles for Optimizing LLM Prompts and Instructions
By
–
7/ Principled Instructions Are All You Need – introduces 26 principles to streamline the process of querying and prompting LLMs; conducts extensive experiments on LLaMA-1/2 & GPT-3.5/4 to verify their effectiveness on instructions & prompts design.
-
Survey of Reasoning with Foundation Models: Latest Advancements
By
–
8/ A Survey of Reasoning with Foundation Models – provides a comprehensive survey of seminal foundational models for reasoning, highlighting the latest advancements in various reasoning tasks, methods, benchmarks, and potential future directions.
-
MLP Layers as Lookup Tables for Factual Recall in LLMs
By
–
5/ Fact Recalling in LLMs – studies how MLP layers implement a lookup table for factual recall; suggests that early MLP layers act as a lookup table and recommends thinking about the recall of factual knowledge in the model as multi-token embeddings.
-
Math-Centric Corpus for Generative AI Foundation Models
By
–
6/ Generative AI for Math – presents a diverse and high-quality math-centric corpus comprising of ~9.5 billion tokens to train foundation models.