And another recent one. That one shows that too much duplicated data (here, ~ training for many epochs) leads to overfitting:
RESEARCH
-
Pythia Paper Findings on Model Architecture Differences
By
–
In the Pythia paper, they found that it doesn’t really make a difference if I recall correctly: https://
arxiv.org/abs/2304.01373 @BlancheMinerva may have some additional insights -
LLM University: New Learning Resource for NLP
By
–
Introducing LLM University — Your Go-To Learning Resource for NLP https://
bit.ly/45eHSZ9 #AI #MachineLearning #DeepLearning #LLMs #DataScience -
New NoPE Paper Released on ArXiv
By
–
Not yet not yet! But you might like this new NoPE paper: https://
arxiv.org/abs/2305.19466 -
Trending AI News Stories and Papers
By
–
Trending AI news stories + papers https://
open.substack.com/pub/akhaliq/p/
trending-ai-news-stories-papers-1f4
… -
Neural Operators for PDE and SDE: Diffusion Models COVID-19
By
–
We demonstrated it here in Covid-19 research. https://
biorxiv.org/content/10.110
1/2021.10.09.463779v1
… Regarding PDE vs SDE, if you input noise as a variable @CristopherSalvi has FNO for SDE. If you want noise to be captured implicitly, you can use our diffusion neural operator https://
arxiv.org/abs/2302.07400 -
On-Device AI Training: Implications Beyond Inference
By
–
Think the big Q here is, if all this stuff stays on device while also personalizing, where it's actually trained/updated before loading into memory. That has much grander implications beyond just inferencing it.
-
LLMs Master Game Playing Through Reasoning and Robot Control
By
–
LLMs outperform RL at game play by studying papers and reasoning through chain-of-thought. So sick. LLMs could be the native language to teach robots. More soon!
-
LIV: Language-Image Representations and Rewards for Robotic Control
By
–
LIV: Language-Image Representations and Rewards for Robotic Control
— AK (@_akhaliq) 5 juin 2023
paper page: https://t.co/wYfPYkBWP6
Language-Image Value (LIV) is a unified pre-training, fine-tuning, and reward learning algorithm for language-conditioned visual manipulation. LIV can perform zero-shot… pic.twitter.com/IeKJiwVTADLIV: Language-Image Representations and Rewards for Robotic Control paper page: https://
huggingface.co/papers/2306.00
958
… Language-Image Value (LIV) is a unified pre-training, fine-tuning, and reward learning algorithm for language-conditioned visual manipulation. LIV can perform zero-shot -
Meta AI Introduces Hiera: Faster Hierarchical Vision Transformer Model
By
–
New research from Meta AI — Hiera is an extremely simple hierarchical vision transformer that's both more accurate than previous models + significantly faster at inference and during training. Paper https://
bit.ly/45K4TU2