Yeah I read the study pretty carefully and thought it seemed reasonable, although they didn't share code or data or model details so hard to verify
RESEARCH
-
Yann LeCun on Developing Autonomous Artificial Intelligence Systems
By
–
Yann LeCun: How to Develop Autonomous Artificial Intelligence https://
techopedia.com/yann-lecun-how
-to-develop-autonomous-artificial-intelligence/2/34812
… @techopedia #AI #MachineLearning #DataScience #BigData #Analytics #100DaysofCode #IoT #serverless #DEVCommunity #womenwhocode #DeepLearning #DigitalTransformation #Python #TensorFlow #Robots -
ResearchPaper.Eth: Web3 Marketplace for Research Paper NFTs
By
–
RESEARCHPAPER.Eth #WeAreBuilding A Marketplace For Research Paper NFTs ResearchPaper.Eth will likely be to Web3 what arXiv is to Web2. Discord: http://
discord.gg/montrealai By: http://
MONTREAL.AI #MontrealAI #NFTCollector #ResearchPaper -
Unlocking Reasoning Capabilities in Smaller Language Models
By
–
Unlocking reasoning in smaller language models is a great direction
-
Intel Achieves SOTA Speed Ups Running Fast Transformers CPUs
By
–
Running Fast Transformers on CPUs: Intel Approach Achieves Significant Speed Ups and SOTA Performance https://
syncedreview.com/2022/11/17/run
ning-fast-transformers-on-cpus-intel-approach-achieves-significant-speed-ups-and-sota-performance/
… -
General-Purpose Neural Networks Through Simple Scalable Objectives
By
–
when the core unlock was achieving a kind of general-purpose computer neural net via simple scalable objectives that have strong training signal (many bits of contraints per training example). Like language modeling, and not like reinforcement learning.
So that was interesting 😀 -
Scaling Language Models: Neural Networks as General-Purpose Text Computers
By
–
TLDR: LMs have been around forever. Not obvious finding: turns out that if you scale up the training set and use a powerful enough neural net (Transformer), the network becomes a kind of general-purpose computer over text.
-
A Decade of AI Progress: Language Models and Neural Networks
By
–
I wrote this thread because I spent the last ~decade, obsessing over directions that would make fastest progress in AI, and was very interested in language models (e.g. my semi-famous 2015 post "The Unreasonable Effectiveness of Recurrent Neural Networks" https://
karpathy.github.io/2015/05/21/rnn
-effectiveness/
…) -
Mispredicting scaling potential: Early distraction with reinforcement learning
By
–
But I still mispredicted in how much fertile ground there was in scaling up the paradigm. Like many others in AI I got distracted by Reinforcement Learning too soon, a kind of putting the cart before the horse, …
-
Transformer Training Objective Difficulty Forces Weight Space Optimization
By
–
The second critical ingredient is that while a Transformer seems ~able to act as a general-purpose computer in principle, the training objective has to be hard enough to actually force the optimization to discover and converge onto it in the "weights space" of the network.