I'm sure they can do great things. I just strongly believe they'd have done more at a real frontier lab.
RESEARCH
-
Emerging Labs vs Frontier AI: Meaningful Work Comparison
By
–
do very well compared to what? They're going to do more meaningful work than the frontier labs?
-

PyTorch Integrates Muon Optimizer Into Core
By
–
considering Muon is so popular and validated at scale, we've just decided to welcome a PR for it in PyTorch core by default.
If anyone wants to take a crack at it… https://
github.com/pytorch/pytorc
h/issues/148819#issuecomment-3070108227
… -

Zuckerberg’s Impact on AI Progress and Talent Retention
By
–
Zuckerberg is doing more to harm AI progress than anyone else IMO. There is no way Meta is going to catchup. Zuck isn’t Elon. He’s distracting a lot of the top talent that were actually doing meaningful work, and now those people are going to retire at Meta and become less
-
On-Policy Learning in Life: Moving Beyond Imitation to Find Your Path
By
–
Becoming an RL diehard in the past year and thinking about RL for most of my waking hours inadvertently taught me an important lesson about how to live my own life. One of the big concepts in RL is that you always want to be “on-policy”: instead of mimicking other people’s successful trajectories, you should take your own actions and learn from the reward given by the environment. Obviously imitation learning is useful to bootstrap to nonzero pass rate initially, but once you can take reasonable trajectories, we generally avoid imitation learning because the best way to leverage the model’s own strengths (which are different from humans) is to only learn from its own trajectories. A well-accepted instantiation of this is that RL is a better way to train language models to solve math word problems compared to simple supervised finetuning on human-written chains of thought. Similarly in life, we first bootstrap ourselves via imitation learning (school), which is very reasonable. But even after I graduated school, I had a habit of studying how other people found success and trying to imitate them. Sometimes it worked, but eventually I realized that I would never surpass the full ability of someone else because they were playing to their strengths which I didn’t have. It could be anything from a researcher doing yolo runs more successfully than me because they built the codebase themselves and I didn’t, or a non-AI example would be a soccer player keeping ball possession by leveraging strength that I didn’t have. The lesson of doing RL on policy is that beating the teacher requires walking your own path and taking risks and rewards from the environment. For example, two things I enjoy more than the average researcher are (1) reading a lot of data, and (2) doing ablations to understand the effect of individual components in a system. Once when collecting a dataset, I spent a few days reading data and giving each human annotator personalized feedback, and after that the data turned out great and I gained valuable insight into the task I was trying to solve. Earlier this year I spent a month going back and ablating each of the decisions that I previously yolo’ed while working on deep research. It was a sizable amount of time spent, but through those experiments I learned unique lessons about what type of RL works well. Not only was leaning into my own passions more fulfilling, but I now feel like I’m on a path to carving a stronger niche for myself and my research. In short, imitation is good and you have to do it initially. But once you’re bootstrapped enough, if you want to beat the teacher you must do on-policy RL and play to your own strengths and weaknesses 🙂
→ View original post on X — @_jasonwei, 2025-07-16 01:26 UTC
-
Meta’s Timeline for Frontier AI Model Development Competition
By
–
Curious how long Meta takes to bring its new team & considerable resources to bear and produce a new frontier model. X took a little under two years to go from start to catching up with Grok 3. Meta has an existing effort & compute, but more complex organizational dynamics.
-

Asymmetry of Verification and Verifier’s Law in AI Training
By
–
New blog post about asymmetry of verification and "verifier's law": jasonwei.net/blog/asymmetry-… Asymmetry of verification–the idea that some tasks are much easier to verify than to solve–is becoming an important idea as we have RL that finally works generally. Great examples of asymmetry of verification are things like sudoku puzzles, writing the code for a website like instagram, and BrowseComp problems (takes ~100 websites to find the answer, but easy to verify once you have the answer). Other tasks have near-symmetry of verification, like summing two 900-digit numbers or some data processing scripts. Yet other tasks are much easier to propose feasible solutions for than to verify them (e.g., fact-checking a long essay or stating a new diet like "only eat bison"). An important thing to understand about asymmetry of verification is that you can improve the asymmetry by doing some work beforehand. For example, if you have the answer key to a math problem or if you have test cases for a Leetcode problem. This greatly increases the set of problems with desirable verification asymmetry. "Verifier's law" states that the ease of training AI to solve a task is proportional to how verifiable the task is. All tasks that are possible to solve and easy to verify will be solved by AI. The ability to train AI to solve a task is proportional to whether the task has the following properties: 1. Objective truth: everyone agrees what good solutions are 2. Fast to verify: any given solution can be verified in a few seconds 3. Scalable to verify: many solutions can be verified simultaneously 4. Low noise: verification is as tightly correlated to the solution quality as possible 5. Continuous reward: it’s easy to rank the goodness of many solutions for a single problem One obvious instantiation of verifier's law is the fact that most benchmarks proposed in AI are easy to verify and so far have been solved. Notice that virtually all popular benchmarks in the past ten years fit criteria #1-4; benchmarks that don’t meet criteria #1-4 would struggle to become popular. Why is verifiability so important? The amount of learning in AI that occurs is maximized when the above criteria are satisfied; you can take a lot of gradient steps where each step has a lot of signal. Speed of iteration is critical—it’s the reason that progress in the digital world has been so much faster than progress in the physical world. AlphaEvolve from Google is one of the greatest examples of leveraging asymmetry of verification. It focuses on setups that fit all the above criteria, and has led to a number of advancements in mathematics and other fields. Different from what we've been doing in AI for the last two decades, it's a new paradigm in that all problems are optimized in a setting where the train set is equivalent to the test set. Asymmetry of verification is everywhere and it's exciting to consider a world of jagged intelligence where anything we can measure will be solved.
→ View original post on X — @_jasonwei, 2025-07-16 00:59 UTC
-
IIM Sambalpur Hosts International Metaverse Banking Conference
By
–
IIM Sambalpur hosts international conference on #Metaverse banking https://
mediabrief.com/iim-sambalpur-
hosts-international-conference-on-metaverse-banking/
… via @
@MediaBrief_ -
Hofstadter’s Metaphysics and Artificial Consciousness Debate
By
–
You seem to be possessed by a vision of Hofstadter hallucinating blue flames and having an imagined uncouth relationship to metaphysics
-
Digital Agents and Real-World Robots: Advancing AI Competence
By
–
In the near term, I hope my research can help make digital agents more competent and helpful (e.g. make an automated travel planning or personal AI assistant possible). In the longer term, I hope my work plays some part in enabling real-world robots that can cook, clean, and do