Not really.
That would have to be compressed in a tiny amount of information.
A small 7B LLM requires 14GB.
Your entire genome fits in 800MB (uncompressed).
The difference between human and chimp genome is 1% of that, or 8MB.
Not enough to encode a significant structure.
LLMS
-
LLM Size vs Genome: Information Compression Limits
By
–
-
New AI Architectures for Efficient Learning Beyond Data Scaling
By
–
Animals and humans get very smart very quickly with vastly smaller amounts of training data.
My money is on new architectures that would learn as efficiently as animals and humans.
Using more data (synthetic or not) is a temporary stopgap made necessary by the limitations of our -
Karpathy Releases Busy Person’s Introduction to Large Language Models
By
–
We are so back! Check out the busy person introduction to large language models by @karpathy
, just released https://
youtu.be/zjkBMFhNj_g?si
=1dXzxon6OMqjiIDs
… -
Internal OpenAI board actions linked to Q* AI breakthrough
By
–
“According to one of the sources, long-time executive Mira Murati told employees on Wednesday that a letter about the AI breakthrough called Q* (pronounced Q-Star), precipitated the board's actions.” https://
msn.com/en-ca/money/ot
her/exclusive-sam-altman-s-ouster-at-openai-was-precipitated-by-letter-to-board-about-ai-breakthrough-sources/ar-AA1knzMo
… -
GPT Pilot: AI-Powered Development Tool Accelerates App Creation
By
–
GPT Pilot – a dev tool that speeds up app development by 20x https://
bit.ly/3Pvy9Xs #AI #MachineLearning #DeepLearning #LLMs #DataScience -
Function Calling Now Available on Lifeboat Platform
By
–
Function calling is now available on the lifeboat. You can enforce JSON responses by specifying response_format: { type: "json_object" }. Or, you can define a specific schema by adding a tool with type function. https://
lifeboat.replicate.dev -
OpenAI Q* Breakthrough: AI Model Solves Math Problems
By
–
“OpenAI researchers used the innovation to build systems that could solve basic math problems, a difficult task for existing AI models. Jakub Pachocki and Szymon Sidor, two top researchers, used Sutskever’s work to build a model called Q* (“Q-Star”)…” https://
theinformation.com/articles/opena
i-made-an-ai-breakthrough-before-altman-firing-stoking-excitement-and-concern
… -
MoE as Alternative to O(n) Complexity in LLMs
By
–
Not O(n) here IIUC: http://
arxiv.org/abs/2311.10770
(And in general MoE seems like a way around O(n)) -
Practical Context Window Evaluation Beyond Statistical Metrics
By
–
I love the "Needle in a Haystack" analysis from @GregKamradt .
Helps us move past stats like "Larger Context Window" to something we care about which the Context Window actually ends up working in practice. -
Gemini Will Revolutionize YouTube, Google Search and More
By
–
Just wait for Gemini. It will revolutionize the way we use YouTube, searching via Google and all stuff.