MLPerf benchmark needs some of these mitigations https://
x.com/jaschasd/statu
s/1589424193946648576?s=46&t=Yrwu6ciEa83A6HcKopDkNw
…
MACHINE LEARNING
-
MLPerf benchmark requires implementation of mitigations
By
–
-
Model Capabilities Emerge with Scale: Sentiment Classification Study
By
–
This is a wonderful study on the emergence of model capabilities with scale. It is remarkable to see sentiment classification working with 1 data point, and fascinating to see the effectiveness of chain of thought prompting growing with scale.
-
VIOLA: UT Austin & Sony AI Advance Robot Manipulation Learning
By
–
UT Austin & Sony AI’s VIOLA Object-Centric Imitation Learning Method for Robot Manipulation Outperforms the SOTA by 45.8%
-
ZerO Initialization with Hadamard Transforms for Rectangular Matrices
By
–
And an overview of how ZerO initialization can overcome degeneracy by using Hadamard transforms for rectangular weight matrices
-
Training Degeneracy in Rectangular Weight Matrices Without Hadamard
By
–
This is the training degeneracy that @jiawzhao is talking about if you only do identity initialization and not Hadamard for rectangular weight matrices
-
AGI Parameter Count vs Brain Synapses: Current Models Scale
By
–
A common view is that human level AGI will require a parameter count in the order of magnitude of the brain’s 100 trillion synapses. The large language models and image generators are only about 1/1000 of that, but they already contain more information than a single human
-
Training Transformers at Scale for Scientific Applications
By
–
Transformers and other large language models have shown impressive capabilities as foundation models for domains such as NLP and CV. Join Andy Hock, our VP of Product, at #SC22, as he discusses new algorithms, software, and hardware for training transformers at scale for science
-
Cool Applications Of Artificial Intelligence In Modern Technology
By
–
Cool Applications Of #ArtificialIntelligence
Via @ingliguori #BigData #Analytics #DataScience #AI #IoT #IIoT #PyTorch #Python #TensorFlow #Java #JavaScript #CloudComputing #DevCommunity #DataScientist #Programming #DigitalTransformation #Cloud #MachineLearning #100DaysofCode -
Detecting Major AI Breakthroughs Before They Happen
By
–
a thing about research i didn't get before openai: frequently before a big idea gets figured it out, multiple teams can sort of detect it on radar through the fog. you get an idea of where it's going to be and the rough shape far before anyone actually lays eyes on it.
-
Emergent Capabilities in Large-Scale Language Models Explained
By
–
As language models increase in scale they sometimes exhibit various useful emergent capabilities. In today’s post, we explore this phenomenon to better understand its dependence on model properties and its potential impact in applications. Read more at https://
goo.gle/3hvJMAb