#PaperADay 10
LeJEPA: Provable and Scalable Self-Supervised Learning Without the Heuristics https://arxiv.org/pdf/2511.08544 The comments on #PaperADay 3 recommended this paper as the state of the art JEPA paper, and it does look much better! They acknowledge that much of the prior JEPA
@id_aa_carmack
-
LeJEPA: Provable and Scalable Self-Supervised Learning Without Heuristics
By
–
-
Flow Model Architecture: Exploring Layer Configurations and Training Efficiency
By
–
Did you try any different configurations of the flow model than 4 layers? I would generally expect a wider 2 layer to train faster, unless there is some character to the flow problem that needs more abstraction.
-
floq: Training Critics via Flow-Matching for Scaling Compute in Value-Based RL
By
–
#PaperADay 9
floq: Training Critics via Flow-Matching for Scaling Compute in Value-Based RL https://arxiv.org/pdf/2509.06863 In theory, value based reinforcement learning is a regression problem, which is most naturally addressed with an MSE loss. However, there are a bunch of subtle -
Deep Delta Learning: Beyond Standard Residual Blocks
By
–
#PaperADay 2
2026: Deep Delta Learning
https://arxiv.org/abs/2601.00417 The standard residual network blocks are limited to adding on top of the existing state, which limits the expressivity of each layer. It is still a universal approximator, but we can always hope for function blocks -
Reading Bookmarked Papers: Weekly Challenge for January
By
–
I like and bookmark so many interesting sounding papers here, and don’t get back to most of them. Time to start making a dent. I’m going to try to at least skim one of the papers in my bookmarks each weekday for the rest of the month.#PaperADay
— John Carmack (@ID_AA_Carmack) 8 janvier 2026
2025: Emergent temporal… https://t.co/WUu7EXVEDWI like and bookmark so many interesting sounding papers here, and don’t get back to most of them. Time to start making a dent. I’m going to try to at least skim one of the papers in my bookmarks each weekday for the rest of the month. #PaperADay 2025: Emergent temporal
-
Simulation Work Requires Understanding True Performance Gap
By
–
I still think most work should be done in simulation, but it is crucial to calibrate your understanding of the true gap!
-
Evaluating RL Performance: Challenges in Benchmarking Improvements
By
–
Thanks — I always half expect someone to pop up say “we just cloned this other repo and ran it for a 50% performance improvement”. With RL it is very hard to know what “good performance” is.
-
Hyperparameter Tuning and Agent Performance Limitations
By
–
I don’t want to take too much credit — hyperparameter tuning for any given algorithm is a big deal, and I’m sure the others can perform well with more effort. I also have much better performing agents, but they won’t run on a laptop.
-
RL Team Replicates Physical Atari Work, Compares Baseline Agent
By
–
Another RL team replicated our Physical Atari work and compared my baseline agent against several standard algorithms. https://
robo-atari.com/report -
PyTorch’s Signed 64-bit Index Standardization Decision
By
–
Pytorch made the right call standardizing on signed 64 bit indexes. I would probably still be rather pointlessly making case by case decisions to use int32 if it were an option. Some old habits linger.