@yitayml

–

02 March 2023 19h23

Ckpts can be grabbed at https://
github.com/google-researc
h/google-research/tree/master/ul2
….

Flan Model Update Released Following Academic Industry Success

By

–

02 March 2023 19h23

With the great success of Flan on both academia/ industry, this update was long overdue! In fact, this was the "bonus model" that we had plan to release because the original tweet got > 1k likes. lol. but i got busy . @jeremyphoward @_jasonwei https://
x.com/quocleix/statu
s/1583523186376785921
…

UL2 Paper Updated: 20B Model Extended to 2048 Token Inputs

By

–

02 March 2023 19h23

We've updated the UL2 paper with details. http://
arxiv.org/abs/2205.05131. UL2 was only trained w 512 inputs, but this extends UL2 20B to 2048 inputs. While we found mode switching to help, it was admittedly v inconvenient. This update also removes the need to use mode tokens 🙂

Flan-UL2 Outperforms Flan-T5 XXL by 3-7 Percent

By

–

02 March 2023 19h23

When compared with Flan-T5 XXL, Flan-UL2 is about +3% better with up to +7% better on CoT setups. It is also competitive to Flan-PaLM 62B! An overall modest perf boost for those looking for something beyond Flan-T5 XXL

Flan-UL2 20B Open Source Model Outperforms Flan-T5

By

–

02 March 2023 19h23

New open source Flan-UL2 20B checkpoints 🙂
– Truly open source No forms! Apache license – Best OS model on MMLU/Big-Bench hard – Better than Flan-T5 XXL & competitive to Flan-PaLM 62B. – Size ceiling of Flan family just got higher! Blog:

Guide to Practicing AI Research by Jason Wei

By

–

02 March 2023 17h32

Not exactly how to write research papers but @_jasonwei has this excellent post on doing AI research https://
jasonwei.net/blog/practicin
g-ai-research
….

Teacher Forcing in AI Training Methods Explained

By

–

02 March 2023 16h32

yea training is teacher forced.

ACL Rebranding Joke: Association for Cool Language Models

By

–

27 February 2023 6h50

I suddenly realized ACL is still called the association of computational linguistics. Damn! Maybe for coolness reasons we should rename it as the association for cool language models. https://
x.com/dystopiabreake
/dystopiabreaker/status/1630000358440448000
…