3/ Risks and Opportunities of Open-Source Generative AI – analyzes the risks and opportunities of open-source generative AI models; argues that the overall benefits of open-source generative AI outweigh its risks.
@dair_ai
-
Agent Planning with World Knowledge Model for Advanced AI
By
–
2/ Agent Planning with World Knowledge Model – a parametric world knowledge model to facilitate agent planning; the agent model can self-synthesize knowledge from expert and sampled trajectories; this is used to train the world knowledge model.
-
IBM Releases Granite Code Models Across 116 Programming Languages
By
–
10). Granite Code Models – introduce Granite, a series of code models trained with code written in 116 programming languages; it consists of models ranging in size from 3 to 34 billion parameters, suitable for applications ranging from application modernization tasks to on-device
-
World Models Survey: Generative Video Synthesis Methodologies Overview
By
–
8). Survey of General World Models – presents an overview of generative methodologies in video generation, where world models facilitate the synthesis of highly realistic visual content.
-
MAmmoTH2: Harvesting Web Data to Enhance LLM Reasoning
By
–
9). MAmmoTH2 – harvest 10 million naturally existing instruction data from the pre-training web corpus to enhance LLM reasoning; the approach first recalls relevant documents, extracts instruction-response pairs, and then refines the extracted pairs using open-source LLMs;
-
Consistency LLMs: Parallel Decoders Reduce Inference Latency
By
–
6). Consistency LLMs – uses efficient parallel decoders that reduce inference latency by decoding n-token sequence per inference step; inspired by he human's ability to form complete sentences before articulating word by word…
-
Flash Attention Stability: Numeric Deviation Effects Analysis
By
–
7). Is Flash Attention Stable? – develops an approach to understanding the effects of numeric deviation and applies it to the widely-adopted Flash Attention optimization…
-
DrEureka Automates Sim-to-Real Robot Design Using LLMs
By
–
5). DrEureka – uses LLMs to automate and accelerate sim-to-real design; it requires the physics simulation for the target task and automatically constructs reward functions and domain randomization distributions to support real-world transfer…https://t.co/4k5bHESjbD
— DAIR.AI (@dair_ai) 12 mai 20245). DrEureka – uses LLMs to automate and accelerate sim-to-real design; it requires the physics simulation for the target task and automatically constructs reward functions and domain randomization distributions to support real-world transfer…
-
AlphaMath Zero: MCTS Enhances LLM Mathematical Reasoning
By
–
4). AlphaMath Almost Zero – enhances LLMs with Monte Carlo Tree Search (MCTS) to improve mathematical reasoning capabilities; the MCTS framework extends the LLM to achieve a more effective balance between exploration and exploitation…
-
DeepSeek-V2: 236B MoE Model with Efficient Latent Attention
By
–
3). DeepSeek-V2 – a strong MoE 236B parameter model, of which 21B are activated for each token; supports a context length of 128K tokens and uses Multi-head Latent Attention (MLA) for efficient inference by compressing the Key-Value (KV) cache into a latent vector…