I do think that distilled models are still useful. I.e., GPT 3.5 in ChatGPT is a distilled model. And as far as I heard GPT 4.0 in ChatGPT is now also distilled.
LLMS
-
PyTorch Distributed Training and Tensor Sharding with Fabric
By
–
None of these: just PyTorch with distributed training and tensor sharding. Optionally with CPU offloading for really big LLMs. I use Fabric as a convenient wrapper here.
-
Creating Games with ChatGPT and Other AI Models
By
–
make games using ChatGPT and other models pic.twitter.com/UthoalqQdR
— AK (@_akhaliq) 17 juin 2023make games using ChatGPT and other models
-
Open Source LLMs Webinar with Nomic AI and MosaicML
By
–
We did an awesome webinar on open source LLMs last Wednesday Thanks to @nomic_ai and @MosaicML for joining Catch up on it here
-
Understanding Encoder and Decoder LLMs: Dispelling Transformer Jargon
By
–
I often get requests to dispel some of the jargon behind transformers and LLMs! So here we go, my new article on "Understanding Encoder and Decoder LLMs":
-
T5X and SeqIO remain dominant LLM training frameworks
By
–
Looks good! Would love to play with this someday. Although I think it would be hard to dethrone t5x + seqio as the best LLM training framework.
-
LangChain Updates Documentation Links for Data Extraction and Tagging
By
–
updated links: extraction: https://
python.langchain.com/docs/modules/c
hains/additional/extraction
… tagging: https://
python.langchain.com/docs/modules/c
hains/additional/tagging
… -
Getting AI Models into the Right Mood for Problem Solving
By
–
A surprising amount of modern AI is getting the model into the right mood for solving your problem.
-
LLMs and Poetry Generation: A Critical Perspective
By
–
Long live LLMs for gifting us the wonders of poetry! /s
-
Defining Highly Capable AI Systems and Model Scaling
By
–
What does highly capable even mean Plus, one can get highly capable systems at smaller sizes too