We control difficulty with: Graph size: more nodes/edges = more elements to reason over Operator complexity: different ops stress different reasoning skills (counting, paths, optimization)
RESEARCH
-
Snorkel Graph Benchmark Shows Significant Room for Advancement
By
–
As the leaderboard shows, this benchmark is far from saturation ATM: https://
leaderboard.snorkel.ai/category/snork
elgraph
… We’re looking forward to seeing continued advances in this skill, particularly in open models. -

SnorkelGraph: Graph Reasoning Benchmark for LLMs
By
–
Operations in graph theory are fundamental to analyzing complex systems, and provide a multi-hop, long-context reasoning challenge for LLMs. That’s why we built SnorkelGraph: a benchmark for evaluating graph reasoning with verifiable ground truth. https://
leaderboard.snorkel.ai/category/snork
elgraph
… -
Evaluating LLM reasoning on graph-based natural language tasks
By
–
Each question asks the LLM to compute the outcome of a natural language question (operator) over graphs encoded via node + edge lists. Example: “Find the minimum-density subgraph…” Because these questions all come with verifiable answers, LLM responses are easy to evaluate.
-
Modular Models and Forgetting in Multi-Modal AI Systems
By
–
Q: "what about forgetting as you add modalities/languages" A: optimistic on modular models (me: like Apple i think but not sure what other research is relavant. see also
-
Francine Bennett Uses Data Science To Make AI More Responsible
By
–
Francine Bennett Uses Data Science To Make AI More Responsible | http://
ow.ly/SucM30sOQyJ
#AI #AIio #AIInnovation #ML #DataScience #Futureofwork @demishassabis @Ronald_vanLoon @TamaraMcCleary @geoffreyhinton @goodfellow_ian @jeffdean @erikbryn -
The Quest for Spatial and Embodied Intelligence
By
–
thank you for the kind offer. the true prize is priceless – the quest for spatial and embodied intelligence.
-
BEHAVIOR AI Robotics System Built on Nvidia Omniverse Platform
By
–
(14/N) We thank @SimovationInc for providing high-quality JoyLo teleoperation data in simulation, reflecting their deep expertise in simulation and data quality. BEHAVIOR is built upon @nvidia
’s Omniverse. We thank @nvidia for their continuous support. -
Long-Horizon Tasks and Embodied AI Scaling Laws
By
–
(10/N) Together, let’s explore:
How close are we to solving long-horizon, complex, human-centric tasks?
How to efficiently combine low-level control and high-level planning?
What are the generalization limits of current models? Are there scaling laws for embodied AI? -
Stanford AI Behavior Challenge: Submit Your Work by November
By
–
(9/N) Evaluation & Submission
Submission instructions and evaluation details are available on our website: https://
behavior.stanford.edu/challenge
Start experimenting today and get ready to compete!
Deadline: Nov. 15th
Winners announced: Dec. 1st
NeurIPS challenge: Dec. 6-7, San Diego, CA