it seems pretty likely distillation would increase capacity if that’s what u mean
RESEARCH
-

Data Scarcity and Pretraining: The New AI Exploration Era
By
–
Good blog on "era of exploration" – Data scarcity is the new bottleneck. LLMs consume data far faster than humans can produce it. We're running out of high-quality training data. – Pretraining solved exploration by accident. Pretraining effectively pays a massive, upfront
-
Compression: Key to AI Model Efficiency and Performance
By
–
i think the answer has something to do with compression btw
-
Deafness Cure Research Cancelled Over Equity Policy Controversy
By
–
Study by my genius friend to CURE DEAFNESS cancelled because, I shit you not: "Research programs based primarily on artificial and non-scientific categories, including amorphous equity objectives, are antithetical to the scientific inquiry". Decel anti-science brain-dead take. https://
x.com/manorlaborator
/manorlaboratory/status/1941703571910566059
… -
AI struggles with harder versions of the game task
By
–
Oh yes I know! Just saying that that's by far the easiest version; and often it can't figure out the final category description (which isn't part of the evals AFAICT). It makes no progress on the other 2 versions of the game that I mentioned.
-

o4-mini struggles explaining UCP3 protein biochemistry context
By
–
Currently I am badgering o4-mini to explain the surrounding context on UCP3 (the "start deliberately wasting ATP" protein), and yeah o4-mini says some stuff where you don't need to know *that* much biochemistry to press Doubt.
-

Physics-based Deep Learning: Download 287-page PDF eBook
By
–
[Download 287-page PDF eBook] Physics-based #DeepLearning: http://
arxiv.org/abs/2109.05237 Learn more: http://
physicsbaseddeeplearning.org
—————
#AI #MachineLearning #DataScience #Physics #NeuralNetworks #Algorithms #Mathematics #ReinforcementLearning #DataScientist -
Semantic Understanding in AI Evaluation Metrics
By
–
Our family rules are that it only counts as "solved" if you figure out the final category descriptor correctly (i.e semantically close, not identical wording). NYT's Connections Bot mentions the same thing sometimes. I don't think the current evals check this AFAIK.
-

60+ Python Notebook Exercises for Understanding Deep Learning
By
–
Now available! 60+ #Python notebook exercises for the book "Understanding #DeepLearning" Download 541-page PDF eBook & notebooks here: http://
udlbook.github.io/udlbook/ +Get print copy: http://
amzn.to/3sydc6B
——————
#AI #MachineLearning #ML #DataScience #DataScientist #NeuralNetworks