1/ LeanDojo – an open-source Lean playground consisting of toolkits, data, models, and benchmarks for theorem proving; also develops ReProver, a retrieval augmented LLM-based prover for theorem solving using premises from a vast math library.
@dair_ai
-
Extending LLM Context Windows to 32K with Minimal Fine-tuning
By
–
2/ Extending Context Window of LLMs – extends the context window of LLMs like LLaMA to up to 32K with minimal fine-tuning (within 1000 steps); attains good performance on several tasks while being more efficient and cost-effective.
-
Top Machine Learning Papers: LeanDojo, DragDiffusion, LLM Context
By
–
Top ML Papers of the Week (June 26 – July 2): – LeanDojo
– DragDiffusion
– Scaling MLPs
– Visual Navigation Transformer
– GPT-4 for Programming Education
– Extending Context Window of LLMs
… -
Subscribe to Weekly Top Machine Learning Papers Newsletter
By
–
Subscribe here if you want to receive the list of top ML papers in your inbox every week:
-
Wanda: Efficient LLM Pruning Without Retraining
By
–
9/ Wanda – introduces a simple and effective pruning approach for LLMs; the approach requires no retraining or weight update and outperforms baselines of magnitude pruning.
-
AudioPaLM Multimodal LM Enables Zero-Shot Speech Translation
By
–
10/ AudioPaLM – fuses text-based & speech-based LMs, PaLM-2 and AudioLM, into a multimodal architecture; outperforms existing systems for speech translation tasks and has zero-shot speech-to-text translation capabilities.
-
MotionGPT: Multimodal Control Signals for Human Motion Generation
By
–
8/ MotionGPT – uses multimodal control signals for generating consecutive human motions; it quantizes multimodal control signals intro discrete codes which are converted to LLM instructions that generate motion answers.https://t.co/B2xAYdbuHc
— DAIR.AI (@dair_ai) 25 juin 20238/ MotionGPT – uses multimodal control signals for generating consecutive human motions; it quantizes multimodal control signals intro discrete codes which are converted to LLM instructions that generate motion answers.
-
LMFlow: Extensible Toolkit for Foundation Model Fine-tuning
By
–
7/ LMFlow – an extensible toolkit that simplifies finetuning and inference of general large foundation models; supports continuous pretraining, instruction tuning, parameter-efficient finetuning, alignment tuning, and large model inference.
-
SequenceMatch: Backtracking Text Generation with Error Mitigation
By
–
6/ SequenceMatch – incorporates backtracking into text generation through a backspace action; enables the model to mitigate compounding errors by reverting sample tokens that lead to sequence OOD.
-
LOMO: Memory-Efficient Optimizer for Full LLM Parameter Tuning
By
–
5/ LOMO – proposes a new memory-efficient optimizer that combines gradient computation and parameter update in one step; enables tuning the full parameters of an LLM with limited resources.