Intriguing Properties of Compression on Multilingual Models. This paper finds that compression confers several interesting & prev. unknown generalization properties on mBERT NER models. Kelechi Ogueji, Orevaoghene Ahia, @lekeOnilude
, Sebastian Gehrmann, @SaraHookr
, @KreutzerJulia
LLMS
-
Compression Enhances Generalization in Multilingual NER Models
By
–
-
HELM: Holistic Evaluation Method for Language Models
By
–
Language Models are Changing AI: The Need for Holistic Evaluation. New benchmarking method, Holistic Evaluation of Language Models (HELM) has been developed at the Center for Research on Foundation Models to help provide transparency in LMs. @RishiBommasani
, @PercyLiang
, Tony Lee -
Teaching Algorithmic Reasoning to LLMs via In-context Learning
By
–
Teaching Algorithmic Reasoning via In-context Learning. A four-stage approach to teaching algorithmic reasoning to LLMs is identified and studied in this work by @oh_that_hat
, @Azade_Na
, @Hugo_Larochelle
, Aaron Courville, @BNeyshabur
, @HanieSedghi -
Fast DistilBERT Pipeline for CPU Inference Optimization
By
–
Fast DistilBERT on CPUs. @HaihaoShen
, @ZafrirOfir
, Bo Dong, Hengyu Meng, Xinyu Ye, Zhe Wang, Yi Ding, Hanwen Chang, Guy Boudoukh, @MosheWasserblat propose a new pipeline for creating and running Fast Transformer models on CPUs -
Intermediate Models Value Through Transfer Learning Fine-tuning
By
–
Where to start? Analyzing the potential value of intermediate models. By finetuning a model on a source dataset, one may have a better starting point when finetuning a target dataset. @LChoshen
, Elad Venezian, Shachar Don-Yehia, @NoamSlonim
, @YoavKatz -
LLMs Struggle Learning Long-Tail Knowledge From Training Data
By
–
Large Language Models Struggle to Learn Long-Tail Knowledge. @kandpal_nikhil
, @HaikangDeng
, Adam Roberts, @Eric_Wallace_
, @ColinRaffel examine the relationship between their pre-training datasets and their knowledge memorized. -
Top NLP Research Papers November 2022
By
–
If you work in #NLP, it's important to keep up to date with the latest research. In this post, we look at some of the top papers on NLP that were published in November 2022 https://
txt.cohere.ai/top-nlp-papers
-november-2022/
… -
Configurable AI Models and Safety Mode Control Options
By
–
i think it’s very reasonable for “default models” to be inoffensive, but within some pretty bounds i think people should be able to configure the models to behave how they’d like, eg like turning off safe mode
-
Fine-tuned Models Producing Poor Results: Data Quality Issues
By
–
but the results from the fine-tuned models were trash and the responses I was getting were really bad I am assuming it takes either significantly more refined prompt/completion data than what I gave it or just more data (both could be true)
-
Initial Fine-Tuning Approach: Breaking Books into Chunks and Generating Questions
By
–
initial approach: oh this seems not too bad, I read this doc https://
beta.openai.com/docs/guides/fi
ne-tuning
… and was like yeah I can just break up the authors' books into chunks and generate some simple questions for each chunk and then use that to fine-tune the model