9/ Managing AI Risks – a short paper outlining risks from upcoming and advanced AI systems, including an examination of social harms, malicious uses, and other potential societal issues emerging from the rapid adoption of autonomous AI systems.
@dair_ai
-
ConvNets Match Vision Transformers: Scaling Laws Analysis
By
–
7/ ConvNets Match Vision Transformers – evaluates a performant ConvNet architecture pretrained on JFT-4B at scale; observes a log-log scaling law between the held out loss and compute budget.
-
Detecting Pretraining Data from Large Language Models
By
–
6/ Detecting Pretraining Data from LLMs – explores the problem of pretraining data detection which aims to determine if a black box model was trained on a given text.
-
LLMs Struggle With New Knowledge Understanding and Association
By
–
5/ LLMs Meet New Knowledge – presents a benchmark to assess LLMs' abilities in knowledge understanding, differentiation, & association; benchmark results show (unsurprisingly) that an LLM’s performance when introduced to new knowledge is not satisfactory.
-
Matryoshka Diffusion Models for High-Resolution Image Video Synthesis
By
–
3/ Matryoshka Diffusion Models – introduces an end-to-end framework for high-resolution image and video synthesis; involves a diffusion process that denoises inputs at multiple resolutions jointly and uses a NestedUNet architecture…
-
Spectron: End-to-End Spoken Language Model Surpasses Existing Systems
By
–
4/ Spectron – a spoken language model trained end-to-end to directly process spectrograms; it can be fine-tuned to generate high-quality accurate spoken language; surpasses existing spoken language models in speaker preservation and semantic coherence.
-
Fact-Checking Capabilities of Large Language Models Compared
By
–
2/ Fact-checking with LLMs – investigates fact-checking capabilities of LLMs; shows the enhanced prowess of LLMs when equipped with contextual information; GPT4 outperforms GPT-3, but accuracy varies based on query language & claim veracity.
-
Top ML Papers: Ring Attention, LLMs Rules Learning, Healthcare
By
–
Top ML Papers of the Week (Oct 9 – Oct 15): – Instruct-Retro
– Ring Attention
– LLMs can Learn Rules
– A Survey of LLMs for Healthcare
– Meta Chain-of-Thought Prompting
– Toward Language Agent Fine-tuning
… —-
1/ Ring Attention – a memory-efficient approach that leverages -
LLMs Generate Synthetic Datasets Solving Data Labeling
By
–
Harnessing LLMs to Generate Synthetic Datasets In the context of machine learning, there's often a lack of labeled data or very little of it. This data collection and labeling process typically ends up taking a lot of effort and time. With the advent of LLMs, the paradigm
-
Top ML Papers: LLMs, Streaming, and Multimodal Advances
By
–
Top ML Papers of the Week (Oct 2 – Oct 8): – StreamingLLM – Analogical Prompting
– The Dawn of LMMs
– Neural Developmental Programs
– LLMs Represent Space and Time
– Retrieval meets Long Context LLMs
… —- 1/ LLMs Represent Space and Time – discovers that LLMs learn linear