9/ Symbol Tuning – an approach to finetune LMs on in-context input-label pairs where natural language labels are replaced by arbitrary symbols; boosts performance on unseen in-context learning tasks and algorithmic reasoning tasks.
@dair_ai
-
TinyStories: Efficient Language Models for Story Generation
By
–
6/ TinyStories – uses a synthetic dataset of short stories to train and evaluate LMs that are much smaller than SoTA models but can produce fluent and consistent stories with several paragraphs, and demonstrate reasoning capabilities.
-
DoReMi: Domain-Weighted Resampling for Efficient Model Training
By
–
7/ DoReMi – trains a small proxy model over domains to produce domain weights without knowledge of downstream tasks; it resamples a dataset with the domain weights which allows using a 280M proxy model to train an 8B model (30x larger) more efficiently.
-
Drag Your GAN: Interactive Point-Based Image Control Method
By
–
1/ Drag Your GAN – an approach for controlling GANs that allows dragging points of the image to precisely reach target points in a user-interactive manner.https://t.co/Nbhtle5VRG
— DAIR.AI (@dair_ai) 21 mai 20231/ Drag Your GAN – an approach for controlling GANs that allows dragging points of the image to precisely reach target points in a user-interactive manner.
-
Top ML Papers of the Week: DragGAN, CodeT5+, Med-PaLM 2
By
–
Top ML Papers of the Week (May 15 – 21): – DragGAN
– CodeT5+
– StructGPT
– Med-PaLM 2
– Symbol Tuning
– Evidence of Meaning in LLMs
… -
MultiModal-GPT: Vision Language Model for Multi-Round Dialogue
By
–
10/ MultiModal-GPT – a vision and language model for multi-round dialogue with humans; the model is fine-tuned from OpenFlamingo, with LoRA added in the cross-attention and self-attention parts of the language model.
-
FrugalGPT: Reducing LLM Inference Costs While Improving Performance
By
–
8/ FrugalGPT – presents strategies to reduce the inference cost associated with using LLMs while improving performance.
-
StarCoder: Open-Source 15.5B Parameter Code Language Model
By
–
9/ StarCoder – an open-access 15.5B parameter LLM with 8K context length and is trained on large amounts of code spanning 80+ programming languages.
-
InstructBLIP Achieves State-of-the-Art Zero-Shot Visual-Language Performance
By
–
6/ InstructBLIP – explores visual-language instruction tuning based on the pre-trained BLIP-2 models; achieves state-of-the-art zero-shot performance on 13 held-out datasets, outperforming BLIP-2 and Flamingo.
-
Active Retrieval Augmented LLMs Advance Knowledge-Intensive Generation
By
–
7/ Active Retrieval Augmented LLMs – an approach that actively decides when and what to retrieve across the course of the LLM generation; demonstrates superior or competitive performance on long-form knowledge-intensive generation tasks.