LLMS

→ View original post on X — @sama

Latest AI Models Now Available in API

By

@sama

–

04 November 2022 21h24

nothing to announce there yet, but the latest models available in our API are pretty good! check them out and let us know what you think.

Few-shot Prompt Setup Notes for Scaling Law Experiments

By

–

04 November 2022 19h55

Some setup notes (cc @EthanJPerez
) – We used the exact 2-shot prompt for Quote Repetition, which is already U-shaped for Gopher/Chinchilla – We used fewer shots for Hindsight – We did few-shot instead of 0-shot for Negation QA – We also showed inverse scaling up to PaLM 62B

U-shaped Scaling and the Limitations of Inverse Scaling Tasks

By

–

04 November 2022 19h55

Implications: 1. U-shaped scaling means that inverse scaling may not hold when extrapolated to larger models. 2. The term “inverse scaling task” is underspecified. A task can be inverse scaling for one type of prompting and positive or U-shaped for another type of prompting.

CoT Prompting Defends Against Inverse Scaling in Math Tasks

By

–

04 November 2022 19h55

Second, we show that CoT prompting can defend against inverse scaling. For instance, CoT prompting achieves 100% on 7 out of 8 subtasks for Redefine Math.

U-shaped Scaling Behavior Emerges at Higher Computational Budgets

By

–

04 November 2022 19h55

Our results first confirm inverse scaling behavior seen on prior models trained up to 500 zettaFLOPs. But at 2K zettaFLOPs, it becomes U-shaped. U-scaling has also been shown in prior work, such as BIG-Bench.

Inverse Scaling Becomes U-Shaped with Larger Language Models

By

–

04 November 2022 19h55

New preprint!

By evaluating 5x larger language models, inverse scaling can become “U-shaped scaling”, which means that performance increases sharply after decreasing.
https://t.co/bZQndKqlB6

These two tasks here are Third Prize winners from the Inverse Scaling Prize. pic.twitter.com/8d3pu8DDrk
— Jason Wei (@_jasonwei) 4 novembre 2022

New preprint! By evaluating 5x larger language models, inverse scaling can become “U-shaped scaling”, which means that performance increases sharply after decreasing. https://
arxiv.org/abs/2211.02011 These two tasks here are Third Prize winners from the Inverse Scaling Prize.

→ View original post on X — @aiatmeta

Meta AI Creates Largest Protein Language Model with 15B Parameters

By

@aiatmeta

–

04 November 2022 18h50

Meta AI researchers trained a language model to fill in protein sequence gaps across millions of diverse proteins & scaled up to 15B parameters, creating the largest language model of proteins to date. More on our latest breakthrough in protein folding https://
bit.ly/3WoWcK2