Based on all the user-request videos that @sama
's been posting, it looks like sora is powered by a Game Engine, and generates artifacts and parameters for the Game Engine.
@soumithchintala
-
Sora Powered by Game Engine: Architecture Analysis
By
–
-
Testing Long Context AI Model with PyTorch Codebase Analysis
By
–
Would love to try this. Its a long enough context length that we can throw in the entire PyTorch codebase and ask questions.
Happy to document my experience and give feedback. -
Meta Explains AI Open-Source Strategy in Earnings Call
By
–
If you have questions about why Meta open-sources its AI, here's a clear answer in Meta's earnings call today from @finkd
-
Allen AI Releases Training Data Scripts and Code Openly
By
–
very cool to release training data scripts and training code!!!@allen_ai is pushing the limits of openness, great work. https://t.co/vY9thKyVGd
— Soumith Chintala (@soumithchintala) 1 février 2024very cool to release training data scripts and training code!!! @allen_ai is pushing the limits of openness, great work.
-
Teknium releases high-quality AI data filtering and curation
By
–
great release from @teknium
, doing a lot of high-quality data filtering and curating work, and opening it up. -
JAX vs PyTorch: TPU, GPU, CPU Performance Comparison
By
–
Knowing all the development history, I'd say — JAX+XLA will be better on TPUs and slightly better on bf16 compatibility.
PyTorch will be better on NVIDIA and AMD GPUs, server-class and desktop-class CPUs, fp16 compat, vastly better on dynamic-shaped workloads.
They'll probably -
JAX TPU Performance Limits Hardware Portability Claims
By
–
would you draw a wider conclusion such as "hardware portability"?
I read the paper and concluded "JAX + TPU is unbeatable". It's hard to conclude anything more general.
The paper doesn't cover any other set of hardware — AMD, Trainium, GraphCore, etc. -
CodeLlama 70B Release: New Open Source LLM Drop
By
–
new CodeLlama 70B drop from @syhw and team.
Offline benchmarks look pretty good, but try it out and see for yourself!