Not all transformers are LLMs, since transformers can also be used for computer vision. And not all LLMs are transformers as there are large language models based on – recurrent (RWKV, https://
arxiv.org/abs/2305.13048) – or convolutional (Hyena, https://
arxiv.org/abs/2302.10866) approaches.
@rasbt
-
Transformers and LLMs: Beyond the Common Assumption
By
–
-
Bag-of-Words Results for IMDb Movie Review Dataset
By
–
Nice work! I also had the bag-of-words results for the IMDb movie review dataset in my post https://
magazine.sebastianraschka.com/p/large-langua
ge-models-and-nearest
… -
Open Source Licenses and Non-Commercial Restrictions Compatibility
By
–
Isn’t open source by definition incompatible with non-commercial restrictions. Ie if it has a non-commercial restriction it can’t be open source?
-
Training Personal Email Assistant with Conversation Variety
By
–
That’s a good point. Yeah if there is sufficient variety in the conversations, I can see how one can train a personal email assistant that way.
-
LLaMA 2 Base Model Inquiry
By
–
Awesome! You mentioned LLaMA 2 versions are coming soon, what’s the base model here?
-
Less Data Better Results Finetuning LLMs with Trimmed Datasets
By
–
Next to LIMA, this is another interesting paper highlighting that more data is not always better when finetuning LLMs: https://
arxiv.org/abs/2307.08701 Trimming the orig 52k Alpaca dataset to 9k can improve the performance when finetuning 7B and 13B parameter LLMs. -
Brief CNN testing experience, focus shifted to LLM work
By
–
Phew, it's been a while. I think I only briefly tried it on some basic CNNs, and it worked fine out of the box. But afaik it's mainly for computer vision applications (have been super focused on LLM work recently)
-
Figure 1 Users: Stop Requesting Permission for Each Use
By
–
"Hey, you >50k people using Figure 1 in your blog articles and lecture slides, pls don't email us for permission each time."
-
Multiprocessing faster than tested alternative, future no-GIL Python promising
By
–
Ha, tried that, but it was slower than multiprocessing. But yeah, with No Gil future Python it might be nice!
-
NN+Gzip optimization: caching and parallel processing improvements
By
–
Update on the NN+Gzip front. Added
1) a caching mechanism, which speeds it up 25%.
2) Parallel processing to speed it up 400% (on a MB Air). Love this method. It's a nice test case for teaching multiprocessing! Code: https://
github.com/rasbt/nn_plus_
gzip/blob/main/1_2_caching-multiprocessing.py
…