I may have misheard but I recalled the focus on not regulating smaller firms who use the APIs. That leaves a question mark on startups building their own, more efficient LLMs via open source models? If anyone deploying a foundational model (LLM) should be regulated with a
LLMS
-
Comprehensive Practical Guide to Large Language Models Resources
By
–
A well-curated list of practical resources of large language models(LLMs). Contains LLMs evolutionary tree, example models, and papers. This is a handy resource for practitioners that want to dive deep into LLMs. Repo: https://
github.com/Mooler0410/LLM
sPracticalGuide
…
Related paper: -
LLMU Twitter Spaces: Comprehensive LLM Curriculum Overview
By
–
Following the launch of LLMU, @luis_likes_math and @jayalammar will be hosting Twitter Spaces next Tuesday. Join them as they provide a comprehensive overview of the curriculum, LLMs and what’s on the horizon! Save the date! https://
x.com/i/spaces/1ZkKz
XrLORwJv
… -
OpenAI CEO warns about ChatGPT risks
By
–
When the CEO of OpenAI tells you to worry about Chat GPT it's time to worry ! #ai #artificalintelligence #MachineLearning
#deeplearning #ChatGPT #OpenAI https://
bbc.co.uk/news/world-us-
canada-65616866
… -
GPT-4 Early AGI Version Capabilities According Lex Fridman
By
–
In the Lex Fridman interview, he seemed to think GPT-4 may be a very early version of AGI. So if there are capabilities they are holding back (which is very likely), it’s possible he believes they are way closer than most would know.
-
LLMs and Human Language: A Profound Insight from Stanford Research
By
–
Clever @google
/
@stanford paper on LLMs from my brother @jerryweiAI
. Performance boosts are great, but there is a more profound insight in this paper that was not explicitly stated: LLMs are trained on human language, but due to the nature of how language was developed (first -
Together Compute Raises $20M to Overcome Transformer Limitations
By
–
I spoke with @togethercompute which is exploring some new techniques to get past the Transformer wall. Together raised a $20m seed from Lux Capital and is also responsible for the RedPajamas open source dataset replicating LLaMA’s set.
-
MegaByte: Meta’s Latest Context Window Expansion Breakthrough
By
–
Finally a third one that is super recent is MegaByte, a paper coming out of Meta last week. This one has been getting a lot of buzz. The three approaches are attempts to expand the context window, which allows for more information coming in (and broader hidden prompts).
-
Sub-Quadratic Attention Techniques: FlashAttention and Hyena Advances
By
–
There are a few new techniques under review right now that try to get sub-quadratic attention. The most common one I hear about today is FlashAttention, which evolves the technique to increase the context window. Hyena out of Stanford’s Hazy lab explores convolutional approaches.
-
Context Window Limitations Restrict Advanced AI Model Capabilities
By
–
Some of that has to do with restrictions around context windows, which govern how much information can go into and come out of a prompt. Right now those windows generally fall in the 4k to 8k token range. Already that is showing signs of being insufficient for advanced uses.