As language models increase in scale they sometimes exhibit various useful emergent capabilities. In today’s post, we explore this phenomenon to better understand its dependence on model properties and its potential impact in applications. Read more at https://
goo.gle/3hvJMAb
RESEARCH
-
Emergent Capabilities in Large-Scale Language Models Explained
By
–
-
Hadamard Transform Essential for Dimension Changes and Degeneracy
By
–
The dimension changes need Hadamard transform and without it you have bad degeneracy. We discuss this in detail
-
Larger AI Models Show Increased Bias Demonstration
By
–
It looks like the bigger the model, the harder it gets for it not to demonstrate biases. Very cool stuff from @mathemakitten
! https://
huggingface.co/blog/zero-shot
-eval-on-the-hub
… -
What Computer Scientists Should Know Before MIT
By
–
What computer scientists wish they knew before coming to MIT: http://
bit.ly/3G4NdYN -
Argonne Nominated for Gordon Bell Special Prize HPC COVID Research
By
–
@Argonne was nominated for the 2022 Gordon Bell Special Prize for HPC Based COVID-19 Research, based on their work on CS-2! Join us for this event at #SC22! Read more about our research here: https://
hubs.li/Q01rJhNW0 -
Building AI Products Panel at Mila Quebec Research Center
By
–
Our very own @ellieevsss on a panel discussing 'Building AI Products' at @Mila_Quebec
, the world's largest academic research center in deep learning. -
Diffusion Models for Audio: Denoising as Subtractive Synthesis
By
–
Always thought diffusion was best suited for audio. Denoising = subtractive synthesis.
-
Deterministic Initialization for Any Network Architecture
By
–
Thank you for tweeting our paper. This is the actual TLDR: We design the first deterministic initialization that works with any network architecture. This includes identity initialization when weight matrix is square and Hadamard initialization when weight matrix is rectangular.
-
The Stack Dataset Trending on Hugging Face Hub
By
–
The Stack from @BigCodeProject is the #1 trending dataset on @huggingface today. Anyone started training some fun stuff on it yet? https://
huggingface.co/datasets/bigco
de/the-stack
… -
CS-2 System 470x Faster Than Joule Supercomputer
By
–
Today, in partnership with @NETL_DOE
, we demonstrated that our CS-2 system is 470 times faster than the Joule Supercomputer in field equation modeling, delivering speeds beyond what either CPUs or GPUs are able to achieve! Read about this milestone here: