RESEARCH

→ View original post on X — @sama

Compute spending will vastly outpace data investment

By

@sama

–

02 February 2023 2h03

do you really think? im pretty sure we will outspend on compute by a _huge_ margin, and at least for companies like openai data spend will go down as models get smarter. curious if i am missing something!

Safer E2E Conversational AI Through Value Sensitive Design

By

–

02 February 2023 2h03

(11/12) Guiding the Release of Safer E2E Conversational AI through Value Sensitive Design
Authors: A. Stevie Bergman, @gavin_does_nlp
, Shannon Spruit, @dirk_hovy, @em_dinan
, Y-Lan Boureau, Verena Rieser

Memory-Efficient NLLB-200: Language-Specific Expert Pruning

By

–

02 February 2023 2h02

(10/12) Memory-efficient NLLB-200: Language-specific Expert Pruning of a Massively Multilingual Machine Translation Model
Authors: Yeskendir Koishekenov, Vassilina Nikoulina, Alexandre Berard

Activation Compression for Model Parallel Training

By

–

02 February 2023 2h02

(9/12) Does compressing activations help model parallel training?
Authors: @bian_song
, @DachengLi177
, Hongyi Wang, @ericxing
, Shivaram Venkataraman

SparseGPT: One-Shot Pruning of Massive Language Models

By

–

02 February 2023 2h02

(8/12) SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shot
Authors: @elias_frantar
, @DAlistarh

Large Language Models as Reasoning Teachers

By

–

02 February 2023 2h02

(7/12) Large Language Models Are Reasoning Teachers
Authors: @itsnamgyu
, Laura Schmid, Se-Young Yun

Teaching Small Language Models to Reason

By

–

02 February 2023 2h02

(6/12) Teaching Small Language Models to Reason
Authors: Lucie Charlotte Magister, Jonathan Mallinson, Jakub Adamek, @ericmalmi
, Aliaksei Severyn

CiT: Data Curation Methods for Vision-Language Model Training

By

–

02 February 2023 2h02

(5/12) CiT: Curation in Training for Effective Vision-Language Data
Authors: Hu Xu, @sainingxie
, Po-Yao Huang, @Licheng_Yu
, Russell Howes, Gargi Ghosh, Luke Zettlemoyer, @cfeichtenhofer