Improving Policy Learning via Language Dynamics Distillation. @hllo_wrld
, @Jayelmnop
, @LukeZettlemoyer
, @EGrefen
, @_rockt propose Language Dynamics Distillation (LDD), which pretrains a model to predict environment dynamics given demonstrations with language descriptions
AI
-
Language Dynamics Distillation Improves Policy Learning
By
–
-
Language Abstractions Improve Intrinsic Exploration in AI
By
–
Improving Intrinsic Exploration with Language Abstractions. @Jayelmnop
, @hllo_wrld
, @RobertaRail
, @MinqiJiang
, Noah Goodman, @_rockt
, @EGrefen explore natural language as a general medium for highlighting relevant abstractions in an environment. -
Compression Enhances Generalization in Multilingual NER Models
By
–
Intriguing Properties of Compression on Multilingual Models. This paper finds that compression confers several interesting & prev. unknown generalization properties on mBERT NER models. Kelechi Ogueji, Orevaoghene Ahia, @lekeOnilude
, Sebastian Gehrmann, @SaraHookr
, @KreutzerJulia -
Exploration Integral to General Intelligence and Learning Systems
By
–
General Intelligence Requires Rethinking Exploration. @MinqiJiang
,
@_rockt
, @EGrefen argue that exploration is integral to all learning systems despite the fact that the study of exploration in AI has mostly focused on reinforcement learning. -
HELM: Holistic Evaluation Method for Language Models
By
–
Language Models are Changing AI: The Need for Holistic Evaluation. New benchmarking method, Holistic Evaluation of Language Models (HELM) has been developed at the Center for Research on Foundation Models to help provide transparency in LMs. @RishiBommasani
, @PercyLiang
, Tony Lee -
Explainable Artificial Intelligence XAI for Engineers
By
–
Explainable Artificial Intelligence (XAI) for AI & ML Engineers: This article was published as a part of the Data Science Blogathon. Introduction Hello AI&ML Engineers, as you all know, Artificial Intelligence (AI) and Machine Learning Engineering are… https://
analyticsvidhya.com/blog/2022/10/e
xplainable-artificial-intelligence-xai-for-ai-ml-engineers/?utm_source=dlvr.it&utm_medium=twitter
… -
Teaching Algorithmic Reasoning to LLMs via In-context Learning
By
–
Teaching Algorithmic Reasoning via In-context Learning. A four-stage approach to teaching algorithmic reasoning to LLMs is identified and studied in this work by @oh_that_hat
, @Azade_Na
, @Hugo_Larochelle
, Aaron Courville, @BNeyshabur
, @HanieSedghi -
Fast DistilBERT Pipeline for CPU Inference Optimization
By
–
Fast DistilBERT on CPUs. @HaihaoShen
, @ZafrirOfir
, Bo Dong, Hengyu Meng, Xinyu Ye, Zhe Wang, Yi Ding, Hanwen Chang, Guy Boudoukh, @MosheWasserblat propose a new pipeline for creating and running Fast Transformer models on CPUs -
Intermediate Models Value Through Transfer Learning Fine-tuning
By
–
Where to start? Analyzing the potential value of intermediate models. By finetuning a model on a source dataset, one may have a better starting point when finetuning a target dataset. @LChoshen
, Elad Venezian, Shachar Don-Yehia, @NoamSlonim
, @YoavKatz -
LLMs Struggle Learning Long-Tail Knowledge From Training Data
By
–
Large Language Models Struggle to Learn Long-Tail Knowledge. @kandpal_nikhil
, @HaikangDeng
, Adam Roberts, @Eric_Wallace_
, @ColinRaffel examine the relationship between their pre-training datasets and their knowledge memorized.