The code that specifies the architecture is tiny.
That's precisely my point.
There is a tiny amount of prior structure and a lot of learning.
RESEARCH
-
Minimal Code Architecture Enables Massive Learning
By
–
-
Language Hardwiring Constraints Within 8MB Memory Limits
By
–
Whatever "hardwiring" is required for language has to fit in 8MB.
-
Brain structure and learning in artificial intelligence
By
–
A tiny amount of brain structure and lots of learning.
-
Sensory Data and Multimodal Learning in AI Development
By
–
Yes, that's one of my points.
Text is insufficient.
We need sensory inputs to learn how the world works.
We can estimate the total amount of visual data seen by a 2 year-old: 2 years = 2x365x12x3600 or roughly 32 million seconds.
We have 2 million optical nerve fibers, carrying -
Genome Storage vs LLM Size: Evolution’s Compression Challenge
By
–
Whatever it is that we learned through evolution has to be squeezed in 800MB (the size of the genome, uncompressed).
most of it is just low-level biochemical machinery. Even a tiny LLM requires 14GB. -
Karpathy’s LLM Technical Explainer: Essential 101 Guide
By
–
3 different people have sent me this video in under 24hrs, so I watched it and can confirm this is a VERY clear, accessible and spot on practical technical explainer of LLMs… great 101… by the ever-awesome @karpathy – highly recommended
-
Building Transformers: Concept and Implementation from Scratch
By
–
Transformer: Concept and code from scratch https://
bit.ly/45YXobu #AI #MachineLearning #DeepLearning #LLMs #DataScience -
4 Open-Source GPT-4 Vision Alternatives: Qwen-VL and CogVLM
By
–
Looking for open-source #GPT4 Vision alternatives? Here are 4 excellent options Qwen-VL
– Code: https://
github.com/QwenLM/Qwen-VL
– Paper: https://
arxiv.org/abs/2308.12966
– Demo: https://
colab.research.google.com/github/camendu
ru/Qwen-VL-Chat-colab/blob/main/Qwen_VL_Chat_colab.ipynb
… CogVLM
– Code: https://
github.com/THUDM/CogVLM
– Paper: https://
arxiv.org/abs/2311.03079
– Demo: -
Evaluating DeepMind’s AI Breakthrough Claims and AGI Progress
By
–
@Alan_Couzens thank you for your summary & thoughts. As you point out without further details it is hard to evaluate whether it is a genuinely new AI breakthrough and pathway towards AGI. And let’s recall there were many announcements from DeepMind a few years ago relating to
-
Personal Intel Agency: AI-powered geopolitical and tech analysis
By
–
Personal Intel Agency: configured to approach questions the way professional analysts do. Like having your own team of geopolitical and tech analysts: