SDK. I think it’s really cool, but the documentation is not great, and overall it could be simpler. Even something like passing in a previous response ID feels like extra code you need to write compared to just passing in a session ID (though prev response ID allows forking).
@mattshumer_
-
Gemini and OpenRouter Models with Extended Context Windows
By
–
Gemini models are good for this! There’s also a new model on OpenRouter that you can try w/ a 2M token context window… should be more than enough
-
AI-powered to-do app with automatic task tracking
By
–
I would pay for a to-do list app that watches my screen and automatically checks off items/moves them to in-flight as I work
-
Building AI Products: Insights from Bio and Twitter
By
–
Please read my bio and thousands of tweets on building AI products sir
-
Simple message storage approach without LangGraph framework
By
–
Uh I have been doing this for many years now. I have no need to run to langgraph. Step 1: place messages in db Step 2: pull messages from db There is no step 3 haha
-
Agent Evals Offline Tests Real World Success Correlation
By
–
Fair, it's all definitions 🙂 But evals in the sense that most product builders think of them (static offline envs/test sets you run an agent version against) tend not to correlate super well to real-world success
-
A/B Testing AI Prompts in Production: Iterative Optimization Strategy
By
–
Use A/B tests in prod against whatever user-driven metric you care about, start by rolling it out to 1-5% of your users, see if it beats current baseline. If it does, great, switch it. If not, iterate the prompt + try again. And if you don't have enough users to get to stat sig
-
User satisfaction matters more than AI evaluation benchmarks
By
–
Users don't give a shit how well you do on evals. They care how well it solves their problems. You can often get evals to *somewhat* correlate with user problems, but coverage is usually mid at best and it's a moving target. We've seen far better success w/ A/B tests against $.
-
Evals Matter for Frontier Models, Not Product Builders
By
–
Let’s settle the evals debate. If you’re a lab, building general-purpose frontier models, yes, evals matter. If you’re a product builder, evals mostly don’t matter. It’s that simple.