I think a very useful thing for AI critics to do would be to "red team" their own beliefs by spending some serious time and effort (maybe with help) to see if they can get AI to do the things they say it can't. If they are right, that is very valuable benchmarking information.
RESEARCH
-
Programmers vs Mathematicians: Variable Naming Philosophy
By
–
Programmers: Always use descriptive variable names
Mathematicians: Single letter variable names always, ideally from obscure/dead alphabets -
Aligning LLMs to Quote from Pre-Training Data
By
–
9/ Aligning LLMs to Quote from Pre-Training Data – proposes techniques to align LLMs to leverage memorized information quotes directly from pre-training data.
-
NLP Cross-Field Engagement Declined Significantly Since 1980
By
–
10/ The Influence Between NLP and Other Fields – aims to quantify the degree of influence between 23 fields of study and NLP; the cross-field engagement of NLP has declined from 0.58 in 1980 to 0.31 in 2022…
-
Iterative Self-Revision and Search for LLM Reasoning Tasks
By
–
6/ Reasoning with Intermediate Revision and Search – presents an approach for general reasoning and search on tasks that can be decomposed into components; incorporates iterative self-revision capabilities and allows an LLM to build an interwoven network of thoughts.
-
Multilingual LLMs Survey: Methods, Taxonomy and Research Frontiers
By
–
7/ Overview of Multilingual LLMs – a survey on multilingual LLMs including a thorough review of methods, a taxonomy, emerging frontiers, challenges, and resources to advance research.
-
Knowledge Capacity Scaling Laws in Language Models
By
–
8/ The Physics of Language Models – investigates knowledge capacity scaling laws where it evaluates a model’s capability via loss or benchmarks, to estimate the number of knowledge bits a model stores.
-
LM-Guided Chain-of-Thought Knowledge Distillation Technique
By
–
4/ LM-Guided Chain-of-Thought – applies knowledge distillation to a small LM with rationales generated by the large LM; the rationale is then generated by the lightweight LM and the answer prediction is done by the frozen large LM.
-
Google DeepMind Synthetic Data Research Best Practices Overview
By
–
5/ Best Practices and Lessons on Synthetic Data – an overview by Google DeepMind on synthetic data research, covering applications, challenges, and future directions
-
OpenEQA Benchmark Dataset for Embodied Question Answering
By
–
2/ OpenEQA – proposes an open-vocabulary benchmark dataset to measure the capabilities of AI models to perform embodied question answering (EQA); it contains 1600 human-generated questions composed from 180 real-world environments.https://t.co/8OYcRgcyjH
— DAIR.AI (@dair_ai) 14 avril 20242/ OpenEQA – proposes an open-vocabulary benchmark dataset to measure the capabilities of AI models to perform embodied question answering (EQA); it contains 1600 human-generated questions composed from 180 real-world environments.