I just ran DeepSeek R1 on smolagents benchmark.
It's an absolute beast Looking forward to run this beast on the full GAIA benchmark! (smolagents benchmark only tests a sample, with a basic CodeAgent setup)
@aymericroucher
-
Evaluating DeepSeek R1 on AI Benchmarks
By
–
-
MiniMax MoE LLM Reaches SOTA with 4M Token Context Length
By
–
Going back to general paper reads:
> MiniMax's new MoE LLM reaches SOTA performance with 4M tokens context length This work from Chinese startup @minimax_ai introduces a novel architecture that achieves state-of-the-art performance while handling context windows up to 4 -
Smolagents v1.3.0 Adds Logging for AI Agent Runs
By
–
We've just released smolagents v1.3.0 🚀
— m_ric (@AymericRoucher) 15 janvier 2025
And it comes with a major feature: you can now log agent runs using OpenTelemetry to inspect them afterwards! 📊
This interactive format comes very handy for inspecting big multi-step runs, instead of scrolling through endless console… pic.twitter.com/0MfCxhxAL9We've just released smolagents v1.3.0 And it comes with a major feature: you can now log agent runs using OpenTelemetry to inspect them afterwards! This interactive format comes very handy for inspecting big multi-step runs, instead of scrolling through endless console
-
Using agentic LLM setups to improve knowledge reliability
By
–
But using LLMs for knowledge is a thing most people already do with ChatGPT (although it's misled that they do this), so using an agentic setup to increase reliability for knowledge makes more sense IMO!
-
Agentic AI Setups Boost LLM Benchmark Performance by 40+ Points
By
–
Cliché: "Agents are just hype"
Reality: Agentic setups can easily bring >40 percentage point increase compared to vanilla LLMs on some benchmarks This crazy score increase makes sense: if I had to answer a SimpleQA question like "Which Dutch player scored an open-play goal -
Exploring GUI Agent Trajectory Generation from Exploration
By
–
-> OS-Genesis: why not to generate the GUI agent trajectories from exploration rather than from high-level tasks?
(spoilers: it works reaally well) The main bottleneck in building GUI agents it to find training data.
GUI Agent trajectories are not easy to get by. Crowdsourcing -
Smolagents 1.2.0 Released with HuggingChat Integration
By
–
New release of smolagents: 1.2.0! 🥳
— m_ric (@AymericRoucher) 10 janvier 2025
Amongst other cool things like halving all loading times, after calling Tool.push_to_hub on a tool to upload it as a Space, you can now use it in HuggingChat ✨ pic.twitter.com/YeIcABjhS2New release of smolagents: 1.2.0! Amongst other cool things like halving all loading times, after calling Tool.push_to_hub on a tool to upload it as a Space, you can now use it in HuggingChat
-
Discussion on AI Agents and AGI Progress via GAIA Benchmark
By
–
Also on your thought in the line "Agents are not solved yet, they'll come when AGI comes" in your blog post, 100% agree:
When any system reaches 90% on GAIA benchmark (hard general tasks up to 1hour long, cc @clefourrier @Thom_Wolf @ThomasScialom
), that means that we have really -
Huggingface’s Smolagents Library Gains Traction for AI Agents
By
–
Since I published it on GitHub a few days ago, @huggingface
's new agentic library smolagents has gathered nearly 4k stars But we are just getting started on agents: so we are hiring an ML Engineer to join me and double down on this effort! The plan is to build GUI agents: -
Code as Agentic Actions for LLMs
By
–
This is a really important part, thank you for highlighting this @sloppenheimer
! Code (indeed many other languages than python could work) is just the better, overlooked version of writing agentic actions for LLMs