Correction: NaturalBench was a fruitful collaboration between CMU and University of Washington.
RESEARCH
-
Why CEV Rather Than Something Else? Merits of the Proposal
By
–
Why CEV rather than something else? What is its merit as a proposal?
-
Pangea: Open Multilingual Multimodal LLM for 39 Languages
By
–
Pangea: A Fully Open Multilingual Multimodal LLM for 39 Languages
—————————–
Paper: https://
arxiv.org/abs/2410.16153
Project website: https://
neulab.github.io/Pangea/
Model: https://
huggingface.co/neulab/Pangea-
7B
…
Extended post by @xiangyue96 : -
CMU releases NaturalBench for multimodal LLM evaluation
By
–
This week, we(two amazing research groups at CMU) released two projects in multimodal evaluation and multilingual multimodal LLMs. – NaturalBench: A vision-centric VQA evaluation benchmark that challenges all leading multimodal LLMs while using natural images and simple
-
NaturalBench: Vision-Language Models Evaluation on Adversarial Samples
By
–
Here are links to the projects and related Twitter posts NaturalBench: Evaluating Vision-Language Models on Natural Adversarial Samples
—————————–
Paper: https://
arxiv.org/abs/2410.14669
Project website: https://
linzhiqiu.github.io/papers/natural
bench/
…
Extended post by @ZhiqiuLin
: -
OpenAI o1 Model Reasoning Patterns and Performance Analysis
By
–
10). Reasoning Patterns of OpenAI’s o1 Model – when compared with other test-time compute methods, o1 achieved the best performance across most datasets; the authors observe that the most commonly used reasoning patterns in o1 are divide and conquer and self-refinement.
-
LLMs Reflect Creator Ideology Across Languages
By
–
8). LLMs Reflect the Ideology of their Creators – finds that LLMs exhibit a diverse ideological stance which reflects the worldview of its creators; finds consistent normative differences between how the same LLM responds in Chinese compared to English.
-
SynthID-Text: Scalable Watermarking Scheme for LLMs
By
–
9). Scalable Watermarking for LLMs – proposes SynthID-Text, a text-watermarking scheme that can preserve text quality in LLMs, enable high detection accuracy, and minimize latency overhead…https://t.co/isvnHcP610
— DAIR.AI (@dair_ai) 27 octobre 20249). Scalable Watermarking for LLMs – proposes SynthID-Text, a text-watermarking scheme that can preserve text quality in LLMs, enable high detection accuracy, and minimize latency overhead…
-
Feature Steering in LLMs: Evaluating Social Bias Control
By
–
6). Evaluation Feature Steering in LLMs – evaluates featuring steering in LLMs using an experiment that artificially dials up and down various features to analyze changes in model outputs; it focused on 29 features related to social biases and study if feature steering can help
-
Data Synthesis and Augmentation Techniques for Large Language Models
By
–
4). A Survey on Data Synthesis and Augmentation for LLMs – provides a comprehensive summary of data generation techniques in the lifecycle of LLMs; includes discussions on data preparation, pre-training, fine-tuning, instruction-tuning, preference alignment, and applications.