no–this is an investment and some ML-specific sessions, not an accelerator-like thing
RESEARCH
-
Rodney Brooks Compiles Archive of IEEE Spectrum Articles
By
–
I have collected links to everything I have written for @IEEESpectrum over the last decade and more. Here at my blog: https://
rodneybrooks.com/my-ieee-spectr
um-columns-and-articles/
… -
Memepocalypse: exponential meme evolution and cultural implications
By
–
In the end, we will be outnumbered by our own memes. Meme types are doubling every 6 months. And they are evolving more visual complexity over time. The memepocalypse is inevitable. https://
nature.com/articles/s4159
8-021-99468-6
… -
Research on AI Example Benefits: Output Syntax and Label Space Learning
By
–
I think what Min et al. shows is that much of the benefit of examples is learning the output syntax and label space. Results aren’t as clean as this anecdote suggests by itself, e.g. OOD labels dramatically hurt performance.
-
Few-shot Prompt Setup Notes for Scaling Law Experiments
By
–
Some setup notes (cc @EthanJPerez
) – We used the exact 2-shot prompt for Quote Repetition, which is already U-shaped for Gopher/Chinchilla – We used fewer shots for Hindsight – We did few-shot instead of 0-shot for Negation QA – We also showed inverse scaling up to PaLM 62B -
CoT Prompting Defends Against Inverse Scaling in Math Tasks
By
–
Second, we show that CoT prompting can defend against inverse scaling. For instance, CoT prompting achieves 100% on 7 out of 8 subtasks for Redefine Math.
-
Inverse Scaling Prize Round 2 Evaluation Announcement
By
–
Make sure to check out the inverse scaling prize, which is a great community effort! Looking forward to evaluating on the Round 2 winners 🙂
-
U-shaped Scaling and the Limitations of Inverse Scaling Tasks
By
–
Implications: 1. U-shaped scaling means that inverse scaling may not hold when extrapolated to larger models. 2. The term “inverse scaling task” is underspecified. A task can be inverse scaling for one type of prompting and positive or U-shaped for another type of prompting.
-
U-shaped Scaling Behavior Emerges at Higher Computational Budgets
By
–
Our results first confirm inverse scaling behavior seen on prior models trained up to 500 zettaFLOPs. But at 2K zettaFLOPs, it becomes U-shaped. U-scaling has also been shown in prior work, such as BIG-Bench.
-
Inverse Scaling Becomes U-Shaped with Larger Language Models
By
–
New preprint!
— Jason Wei (@_jasonwei) 4 novembre 2022
By evaluating 5x larger language models, inverse scaling can become “U-shaped scaling”, which means that performance increases sharply after decreasing.
https://t.co/bZQndKqlB6
These two tasks here are Third Prize winners from the Inverse Scaling Prize. pic.twitter.com/8d3pu8DDrkNew preprint! By evaluating 5x larger language models, inverse scaling can become “U-shaped scaling”, which means that performance increases sharply after decreasing. https://
arxiv.org/abs/2211.02011 These two tasks here are Third Prize winners from the Inverse Scaling Prize.