A few months ago, I tried to make GPT play FIFA like this. Kinda works but doesn't. At broad direction, it's correct like 50% of the time, but it's not very good at magnitude.
But once you get to the more granular moves, it fails hard, just making up stuff.
@soumithchintala
-
GPT Playing FIFA: Broad Directions Work, Granular Moves Fail
By
–
-
torch.compile CUDAGraph overhead optimization reinforcement learning
By
–
there's more CPU overhead, but it practically doesn't matter IMO unless you are doing tiny reinforcement-like workloads — and even there `torch.compile` will CUDAGraph it to near-zero overhead.
-
Tesla Humanoids Impress with Teleoperated Hands
By
–
this is correct.
— Soumith Chintala (@soumithchintala) 17 octobre 2024
i find the Tesla humanoids extremely impressive.
teleoping hands is not a big deal for now. https://t.co/kzrvVtqgcRthis is correct. i find the Tesla humanoids extremely impressive.
teleoping hands is not a big deal for now. -
Building-Scale Robot Tasks: Cool Work in Progress
By
–
really cool work, building-scale robot tasks! https://t.co/fysJMGWuc4
— Soumith Chintala (@soumithchintala) 15 octobre 2024really cool work, building-scale robot tasks!
-
SpaceX organizational excellence drives engineering innovation without bureaucracy
By
–
SpaceX continues to be incredible.
— Soumith Chintala (@soumithchintala) 13 octobre 2024
The engineering feats are absolutely nuts.
What I find more incredible is SpaceX as an organization — can execute structured long-term research and engineering bets without bureaucracy and with high velocity.
99.999% of organizations at this… https://t.co/TtjxVHfX2TSpaceX continues to be incredible.
The engineering feats are absolutely nuts. What I find more incredible is SpaceX as an organization — can execute structured long-term research and engineering bets without bureaucracy and with high velocity. 99.999% of organizations at this -
PyTorch’s Functional Interface Design Evolution History
By
–
fun fact: nn.functional wasn't part of the first design, and not part of the first 4 pre-releases; but then @szagoruyko5 convinced us that the functional interface in twitter/torch-autograd (in lua) was really nice to use for certain kinds of research; and that we must have
-
Meta Releases MovieGen Scientific Paper for Video Generation
By
–
yes, Meta released a full scientific paper on MovieGen, with a lot of details that'll help the field move forward. https://t.co/pu8z0Ra6U5
— Soumith Chintala (@soumithchintala) 4 octobre 2024yes, Meta released a full scientific paper on MovieGen, with a lot of details that'll help the field move forward.
-
MovieGen Stack Upgrade Enables Personalized Video Generation
By
–
the MovieGen stack got a huge upgrade.
— Soumith Chintala (@soumithchintala) 4 octobre 2024
compared to others, the Editing and Personalization features are really on a different level!
producing personalized videos from a single photo of a person was quite mind-blowing to me. https://t.co/rdB0EulkYMthe MovieGen stack got a huge upgrade.
compared to others, the Editing and Personalization features are really on a different level! producing personalized videos from a single photo of a person was quite mind-blowing to me. -
Scaling Laws Enable Efficient Transformer Hyperparameter Optimization
By
–
Modern transformers are well-behaved according to well-behaved scaling laws (which are a function of (num. of tokens, num. of parameters).
This allows one to find the hyperparameters at a smaller scale, and then keep scaling up parameters and data according to some power law. -
Model Convergence Issues in Large Language Models
By
–
because they have convergence issues, especially when model size gets larger