I know some companies like to prototype in PyTorch then rewrite in JAX for performance. Did you know you could just… prototype in Keras + PyTorch, then switch to JAX *while keeping all of your model code*? At most you'll have to rewrite the train_step or the training loop.
MACHINE LEARNING
-
Essential Keras Tips for Debugging and Prototyping with Different Backends
By
–
Essential Keras tips: 1. Debug/prototype with eager execution. You can prototype with the PyTorch backend, or even with the NumPy backend if you're only looking at a layer or a forward pass (the NumPy doesn't support gradients/training). With JAX or TF, make sure to use
-
Active Inference as a Solution to Distribution Shift in Models
By
–
There are essentially two main options to remedy this: 1. Find ways to perform active inference, so that the model adapts its learned program in contact with a new data distribution at test time. Would likely lead to some meaningful progress, but it isn't the ultimate solution,
-
Neural Networks Struggle with First-Order Logic Learning
By
–
As pointed out by @sirbayes
, this paper https://
starai.cs.ucla.edu/papers/ZhangIJ
CAI23.pdf
… has a formal investigation into the observation from the first tweet — that "my models were completely unable to learn to perform actual first-order logic — despite the fact that this ability was definitely part of -
Find LLM leaderboards: Aymeric Roucher shares ultimate collection by C. Le Fourrier
By
–
Are you trying to find good leaderboards to compare LLMs? @clefourrier is building the ultimate collection here:
-
Kirk Borne Receives Updated 3rd Edition on Transformers and Generative AI
By
–
WOW! I just received (minutes ago) my copy of the updated 3rd Edition of this book: http://
amzn.to/3TkBknI
"Transformers for Natural Language Processing and Computer Vision: Explore #GenerativeAI and Large Language Models #LLMs with Hugging Face, ChatGPT, GPT-4V, and DALL-E 3" -
Yi Models Tech Report Launch: Base, Chat, and Vision-Language Innovations
By
–
Yi models TECH REPORT just went live! Sharing our humble explorations launching, improving, innovating our base, chat, and vision-language models. Kudos to @01AI_Yi team behind the scenes. Love to hear the feedback from the community!
-
Teaching LLMs to Reason with Reinforcement Learning
By
–
Teaching Large Language Models to Reason with Reinforcement Learning Havrilla et al.: https://
arxiv.org/abs/2403.04642 #ArtificialIntelligence #DeepLearning #MachineLearning -
Exploration-Based Trajectory Optimization for LLM Agents
By
–
Trial and Error: Exploration-Based Trajectory Optimization for LLM Agents Song et al.: https://
arxiv.org/abs/2403.02502 #Artificialintelligence #DeepLearning #MachineLearning -
MediSwift-XL Sparse Model Outperforms Dense Competitor
By
–
(4/n) At 75% sparsity, MediSwift-XL outperforms the dense MediSwift-Med, despite having the same non-embedding parameters. This highlights the advantages of training larger but sparse models over smaller, densely parameterized models.