Recently, I went on my first podcast hosted by @swyx
. It was a fun unfiltered 2 hour long conversation. Could have gone on longer but we got chased out of the studio.. Talked about a lot of stuff, i.e., reminiscing old stuff at @Google and newer stuff at @RekaAILabs .
@yitayml
-
First Podcast Experience: Google and Reka AI Insights
By
–
-
Cross-Document Attention Leakage in Sequence Packing Solutions
By
–
This paper tries to solve a non-issue actually . Their claim is that when you do packing (they call it concat and chunk lol) you get cross document attention leakage. The truth is that if your infra is decent you'll have segmentation masks that prevent this from happening in
-
PaLI-X advances robotics with real-world apple and banana sorting
By
–
This is cool! 🥳 Nice to see more impact from PaLI-X (ViT22B + 32B UL2) on robotics.
— Yi Tay (@YiTayML) 29 juillet 2023
Gotta admit the ability to move apples and bananas in the real world just hits different from any emergent language/vision tasks. https://t.co/izPwjiVEoMThis is cool! Nice to see more impact from PaLI-X (ViT22B + 32B UL2) on robotics. Gotta admit the ability to move apples and bananas in the real world just hits different from any emergent language/vision tasks.
-
Frontier Models vs Foundation Models: Terminology Debate
By
–
Frontier models vs foundation models. "This time, at least the name is coined by groups that actually have a solid track record building such models, so I don't need to be grumpy right away :)" Solid burn
-
Chain of Thought Prompting: Breakthrough or Overrated Innovation?
By
–
brightest mind of our generation: invents chain of thought prompting. same legendary mind:
-
Flan-UL2 32B with improved tokenizer could dominate open source
By
–
a flan-ul2 32b with a new (not-horrible) tokenizer is going to beat all open source models out there today. someone at google should do it.
-
Vit-22b and Flan Collection Presented at ICML Conference
By
–
Vit-22b and flan collection are presented at @icmlconf
! -
Meta’s treatment of Llama-2 authors raises ethical concerns
By
–
Wow! Not cool meta! Gotta admit the first thing I looked for in the llama-2 paper was @GuillaumeLample on the author list. PS: Google always retained authors on the papers even after they left the company. Don't be evil! https://
x.com/GuillaumeLampl
/GuillaumeLample/status/1681346701766934543
… -
Sequence packing innovation brings efficiency to Vision Transformers
By
–
Nice work by @m__dehghani
! Very exciting to see sequence packing come to ViT.