Highlights from recent evaluations (insurance underwriting & more): Surprising error modes in complex reasoning Trade-offs between tool use & efficiency Beyond accuracy: deeper evaluation with Snorkel Evaluate Full leaderboards →
LLMS
-
Small Specialized AI Models Represent Future Direction
By
–
been saying this since October last year btw Small Specialized Models are the future
-
XGBoost 2: ML Confusion with LLM Terminology
By
–
I rediscovered this interesting post from early 2024 by @OliverMolander on LinkedIn about XGBoost-based machine learning on Tabular Data: https://
linkedin.com/posts/olivermo
lander_xgboost-artificialintelligence-generativeai-activity-7170725771185975296-DqHm?utm_source=share&utm_medium=member_ios
… This is Oliver’s post (not mine):
————————————— "What's your thoughts on XGBoost 2?" "I've never heard of that LLM" -
MoE GPU Communication Overhead Analysis Technical Deep Dive
By
–
Calling all Mathletes, this one is for you. We’ve been asked to show the math behind our MoE claims. So we did. Our analysis confirms: On GPUs, expert parallelism creates severe communication overheads that dwarf computation and make MoE training painfully slow. At
-
Claude Update Impact on Developer Understanding and Code Workflow
By
–
Felt it too since the last Claude update. My daily work (via Cursor) significantly changed… Hardest part these days is making the executive decision: do I even need to understand how it works? For throw-away scripts or tools with verifiable outputs it's increasingly tough!
-
Open-Ended Quality Diversity AI-Generating Algorithms Foundation Models
By
–
Excited to host @jeffclune this Friday in our AI4Science Community! Open-Ended, Quality Diversity, and AI-Generating Algorithms in the Era of Foundation Models Join in the link below!
-
Ambient Agents and Above the Line Approaches with LangChain and Harmonic AI
By
–
Join LangChain CEO Harrison Chase and @harmonic_ai CEO Max Ruderman in NYC as they explore ambient agents and "Above the Line" approaches—moving past manual processes to systems that understand intent and orchestrate intelligent outcomes. RSVP:
-
Meta FAIR Releases New Test-Time Compute Scaling Method for LLMs
By
–
Meta FAIR just released a brand new test-time compute scaling method! With their new LLM solution aggregator trained with RLVR, they showed that majority voting or reward model ranking aren’t actually the most efficient. This simple yet robust method outperforms RM baselines.
-
LangChain SummarizationMiddleware Prevents Context Window Overflow
By
–
Avoid overflowing context windows with LangChain's SummarizationMiddleware. This is especially important for long running conversations that have lots of messages and agent loops with lots of tool calls.
-
OpenAI study: 18.3% of ChatGPT queries are searches
By
–
So @OpenAI quietly released a study on how people use ChatGPT…. And the most searched topic is specific information search, with 18.3% of queries. Let's stop on this number. People are increasingly using models