LLMS

Pass@2 Scores Gap Below Pass@250 Levels Explained

By

–

01 July 2025 16h47

The pass@2 scores are actually reported in the blog post, but as mentioned in the parent tweet, the gap is ~ 10-15% below pass@250 levels. What I meant in my previous tweet is to narrow this gap in the future.

→ View original post on X — @clementdelangue

Meta’s Llama Strategy: Reshaping AI Frontier Impact

By

@clementdelangue

–

01 July 2025 16h04

In my opinion, Meta has been the most impactful big tech company in AI thanks to the open release of Llama that changed the field and helped them catch up to the frontier in record times! I suspect they’ll be able to 100x their impact thanks to the addition of @alexandr_wang

Scaling LLM Inference Compute: Adaptive Branching Tree Search

By

–

01 July 2025 15h23

Our paper: “Wider or Deeper? Scaling LLM Inference-Time Compute with Adaptive Branching Tree Search”

AB-MCTS Search Capability Evaluation with High Pass@k Metrics

By

–

01 July 2025 14h00

Thanks @fchollet
. Indeed, the experiments used a large Pass@k which allowed us to focus on evaluating the search capability of AB-MCTS, rather than the official evaluation criteria based on k=2. We also used tasks in the public eval. Hopefully we’ll get down to Pass@2 someday! 🙂

Multi-LLM AB-MCTS Combination Outperforms on ARC-AGI-2 Benchmark

By

–

01 July 2025 11h33

The Multi-LLM AB-MCTS combination of o4-mini + Gemini-2.5-Pro + DeepSeek-R1-0528, current frontier AI models, achieves strong performance on the ARC-AGI-2 benchmark, outperforming individual models by a large margin. Implementation of AB-MCTS on GitHub: https://
github.com/SakanaAI/treeq
uest
…

→ View original post on X — @rowancheung

Baidu Open-Sources 10 ERNIE 4.5 Model Versions, 424B Beats DeepSeek V3

By

@rowancheung

–

01 July 2025 8h59

Baidu open-sourced 10 versions of its ERNIE 4.5 model family. Key highlights: —Models from 300M to 424B params
—424B version beats DeepSeek V3 on 22/28 benchmarks
—“Heterogeneous” training architecture for increased efficiency
—Apache 2.0 license

→ View original post on X — @alexjc

AI Training Now Legally Classified Under Copyright Law

By

@alexjc

–

01 July 2025 8h55

Well, the overwhelming consensus is now that the activities around AI training fall under Copyright law. The "non-expressive use" folks always had a fringe opinion, but have lost. Not just caselaw but regulations codify it. There may be exceptions to the rule, but not important.

Multiple LLMs Solve Previously Unsolvable ARC-AGI-2 Examples Together

By

–

01 July 2025 7h25

Many ARC-AGI-2 examples that were unsolvable by any single LLM were solved by combining multiple LLMs. In some cases, an initially incorrect attempt by o4-mini is used by R1-0528 and Gemini-2.5-Pro as a hint to get to the correct solution. ARC-AGI-2 code: https://
github.com/SakanaAI/ab-mc
ts-arc2
…