We've updated the UL2 paper with details. http://
arxiv.org/abs/2205.05131. UL2 was only trained w 512 inputs, but this extends UL2 20B to 2048 inputs. While we found mode switching to help, it was admittedly v inconvenient. This update also removes the need to use mode tokens 🙂
LLMS
-
UL2 Paper Updated: 20B Model Extended to 2048 Token Inputs
By
–
-
Flan-UL2 20B Open Source Model Outperforms Flan-T5
By
–
New open source Flan-UL2 20B checkpoints 🙂
– Truly open source No forms! Apache license – Best OS model on MMLU/Big-Bench hard – Better than Flan-T5 XXL & competitive to Flan-PaLM 62B. – Size ceiling of Flan family just got higher! Blog: -
Flan-UL2 Outperforms Flan-T5 XXL by 3-7 Percent
By
–
When compared with Flan-T5 XXL, Flan-UL2 is about +3% better with up to +7% better on CoT setups. It is also competitive to Flan-PaLM 62B! An overall modest perf boost for those looking for something beyond Flan-T5 XXL
-
Go see the new video by DEFAKATOR_Off on ChatGPT
By
–
Go see the new video by @DEFAKATOR_Off on ChatGPT!
-
Meta AI Researchers Compare Language Models Brain Differences Predictions
By
–
New in Nature Human Behavior, Meta AI researchers show how current language models differ from the human brain & highlight the role of long-range & hierarchical predictions. We hope these findings will help inform the next generation of AI https://
go.nature.com/3SKb3gX -
ChatGPT Pravda 2.0: AI Deconstruction and Ideological Debate
By
–
Notre article avec @OlivierBabeau dans @Le_Figaro ChatGPT : la PRAVDA 2.0 ? #ChatGPT est une IA DÉCONSTRUITE qui va enchanter @sandrousseau et tous les wokistes de la #NUPES
-
Meta’s Llama Model Outperforms OpenAI’s GPT-3 in New Innovation
By
–
AI innovation continues to thrive, with exciting developments around every corner. Move over GPT-3, there's a new llama in town! India's Meta-S Llama model has outperformed @OpenAI
's famed language model. Read more here: https://
bit.ly/3y9zydC @MetaAI @Meta -
Stability AI Continues Support for EleutherAI Research Community
By
–
EleutherAI is one of the main research communities that Stability AI has been supporting and it's been amazing to see their progress over the last couple of years! We wish them luck on this next step of their journey and we're glad to continue our support!
-
GPT Latency Improvements: 500ms Lag in Conversational AI
By
–
Currently, there is some lag, let's say 500 ms, but it doesn't feel bad when you play it (we are talking about conversations and high-level behavioral decisions, not impulse reactions) I believe the latency of GPTs will improve too. It must.