Please read my bio and thousands of tweets on building AI products sir
@mattshumer_
-
Simple message storage approach without LangGraph framework
By
–
Uh I have been doing this for many years now. I have no need to run to langgraph. Step 1: place messages in db Step 2: pull messages from db There is no step 3 haha
-
Agent Evals Offline Tests Real World Success Correlation
By
–
Fair, it's all definitions 🙂 But evals in the sense that most product builders think of them (static offline envs/test sets you run an agent version against) tend not to correlate super well to real-world success
-
A/B Testing AI Prompts in Production: Iterative Optimization Strategy
By
–
Use A/B tests in prod against whatever user-driven metric you care about, start by rolling it out to 1-5% of your users, see if it beats current baseline. If it does, great, switch it. If not, iterate the prompt + try again. And if you don't have enough users to get to stat sig
-
User satisfaction matters more than AI evaluation benchmarks
By
–
Users don't give a shit how well you do on evals. They care how well it solves their problems. You can often get evals to *somewhat* correlate with user problems, but coverage is usually mid at best and it's a moving target. We've seen far better success w/ A/B tests against $.
-
Evals Matter for Frontier Models, Not Product Builders
By
–
Let’s settle the evals debate. If you’re a lab, building general-purpose frontier models, yes, evals matter. If you’re a product builder, evals mostly don’t matter. It’s that simple.
-
AI design generation becoming indistinguishable from human-created work
By
–
I was just sent this video of this app that was vibe-coded in Rork
— Matt Shumer (@mattshumer_) 6 septembre 2025
There’s just no way anybody can tell that it was created by AI
The design feels so unique and custom
This is going to become the norm… over the next year, AI is going to get really good at design pic.twitter.com/hi3E9sc3ENI was just sent this video of this app that was vibe-coded in Rork There’s just no way anybody can tell that it was created by AI The design feels so unique and custom This is going to become the norm… over the next year, AI is going to get really good at design
-
AgentBase Platform Sparks Industry Conversation on Agent Approach
By
–
Hey Thomas, AgentBase looks great. I don't say this lightly (I am deeply opposed to the approach most companies in this space are taking). Would love to chat, DM me if interested!
-
Master LLM Prompting: Write Clear Instructions Without Ambiguity
By
–
My biggest tip for prompting LLMs: If a human reading the prompt for the first time couldn't do the task properly without asking clarifying questions, then a LLM will fail as well. Flow: write your v1 prompt -> ask yourself "what is still unclear?" -> fix the prompt -> repeat!
-
Building AI Agents: Raw APIs Beat Complex Frameworks
By
–
Too many people use frameworks to build agents, when really, all you need are the raw LLM APIs and a while loop. Don't overcomplicate what doesn't need to be overcomplicated.