From there I started asking around if anyone in the lab knew game theory well enough to attempt an analysis of the algorithm I had intuited, and fortunately Jean was there
CODE
-
Simplified KL Divergence Proof Advances Machine Learning Theory
By
–
Another big step at this part of the story was moving from a more complicated proof based on functional derivatives to the simpler one based on convexity of KL divergences that we published. @dwf contributed this final step
-
GraphCast: DeepMind’s Open-Source AI Weather Forecasting Model
By
–
Check out the codebase here: https://
github.com/google-deepmin
d/graphcast
… -
DeepMind Open Sources GenCast Weather Forecasting Model
By
–
DeepMind is on fire today, fully open sourced GenCast – SOTA diffusion based weather forecasting (ensemble) model! can predict temperature 15 days ahead in just 8 minutes powered by TPU v5e Code and weights open source! (although in a google cloud bucket)
-
Copy OpenAI Docs to ChatGPT for Latest API Updates
By
–
A little touch to improve your coding workflow: copy any @OpenAIDevs docs page to bring the latest API updates into ChatGPT, Cursor, or your favorite IDE! 📄✨ https://t.co/gy19Ml66nB
— Romain Huet (@romainhuet) 4 décembre 2024A little touch to improve your coding workflow: copy any @OpenAIDevs docs page to bring the latest API updates into ChatGPT, Cursor, or your favorite IDE!
-
GPU Training Issues with Latest PyTorch Nightly DDP
By
–
Glad to see the level of interest. I figured tokenization was too boring for most! The results on 8xH100 will have to wait until tomorrow; with the latest nightly torchrun is crashing in DDP with exit code -11.
-
HellaSwag Evaluation Script for Language Models Released
By
–
You can also find a HellaSwag evaluation script that works with one large batch item, EOT separators, and document_mask here: https://
github.com/alexjc/nanogpt
-speedrun/blob/master/eval_gpt2.py
… It's setup to use TokenMonster too, so you can check the scores on downstream benchmarks easily hopefully. -
Optimizing Code with Padding Tokens for Efficiency
By
–
The code hasn't changed so there are padding tokens in there still… I could be using that padding though!
-
Model weights release on Hugging Face platform
By
–
Congrats on the release! Would be nice to have the weights on http://
hf.co happy to help with that! cc: @cyrilzakka too -
TokenMonster vocabulary files released for AI models
By
–
I also uploaded the binary files for the tokens to this repository, which you can use as drop-in replacement. The TokenMonster vocabulary file is alongside the tokens and on GitHub too: