And an overview of how ZerO initialization can overcome degeneracy by using Hadamard transforms for rectangular weight matrices
RESEARCH
-
Training Degeneracy in Rectangular Weight Matrices Without Hadamard
By
–
This is the training degeneracy that @jiawzhao is talking about if you only do identity initialization and not Hadamard for rectangular weight matrices
-
AGI Parameter Count vs Brain Synapses: Current Models Scale
By
–
A common view is that human level AGI will require a parameter count in the order of magnitude of the brain’s 100 trillion synapses. The large language models and image generators are only about 1/1000 of that, but they already contain more information than a single human
-
Human-level AGI could run in a box, not data centers
By
–
could ever possibly know. It is at least plausible that human level AGI might initially run in a box instead of an entire data center. Some still hope for quantum magic in the neurons; I think it more likely that they are actually kind of crappy computational elements.
-
Training Transformers at Scale for Scientific Applications
By
–
Transformers and other large language models have shown impressive capabilities as foundation models for domains such as NLP and CV. Join Andy Hock, our VP of Product, at #SC22, as he discusses new algorithms, software, and hardware for training transformers at scale for science
-
Cool Applications Of Artificial Intelligence In Modern Technology
By
–
Cool Applications Of #ArtificialIntelligence
Via @ingliguori #BigData #Analytics #DataScience #AI #IoT #IIoT #PyTorch #Python #TensorFlow #Java #JavaScript #CloudComputing #DevCommunity #DataScientist #Programming #DigitalTransformation #Cloud #MachineLearning #100DaysofCode -
Prompt Examples: Impact on Model Performance and Variation
By
–
Depends on the prompt! More examples is especially helpful when the examples relate to one another and contain explicit information to inform future generations. But too many examples can lead to decreased performance and variation in responses.
-
Mastering Prompt Engineering: Learning Through Practice and Intuition
By
–
It’s fascinating to see how others prompt GPT-3. There’s no ‘one right answer’ to design any prompt. Because there’s no go-to way to learn it, most people learn by doing, building unique intuition along the way. So, everyone’s prompts look very different!
-
Detecting Major AI Breakthroughs Before They Happen
By
–
a thing about research i didn't get before openai: frequently before a big idea gets figured it out, multiple teams can sort of detect it on radar through the fog. you get an idea of where it's going to be and the rough shape far before anyone actually lays eyes on it.
-
IBM announces most powerful quantum processor Osprey with 433 qubits
By
–
[INNOVER]
#IBM annonce le processeur #quantique le plus puissant et de loin https://
buff.ly/3hs9KVk v/ @engadget Le nouveau processeur #Osprey affiche 433 qubits, soit le triple de la version Eagle de 2021, en attendant 10k qubits en 2026 #Tech #FlashTweet