Yeah, agreed. But unfortunately, that wouldn't be possible with LLMs.
@rasbt
-
CUDA Deterministic Mode Limitations Across Different Hardware
By
–
I think this is not possible because of CUDA requirements; you can force deterministic mode but in my experience results will still differ across hardware.
-
Free NLP and Large Language Models Course with Recommended Writings
By
–
I have a free intro course on Natural Language Processing and Large Language Models here that might be useful: https://
lightning.ai/pages/courses/
deep-learning-fundamentals/unit-8.0-natural-language-processing-and-large-language-models/
… I also have a bunch of writings here (Including a list of papers I recommend): https://
magazine.sebastianraschka.com -
Upcoming Update on Current Project Coming This Week
By
–
Yes, coincidentally, I am currently working on it. Stay (fine)tuned for an update on that this week!
-
Mathematical Definition of Nut Cracking Using Set Theory
By
–
Haha true. Let N be the set of all nuts, and let
C : N → P(P(N)) be the function for cracking a nut, where P(N) is the power set of N. Then, For every nut n in N, if n is cracked via C(n)), the cardinality of the resulting set of nutshells |C(n)| > 2. -
LLaMA-Adapter and LoRA: Multimodal Input Advantages
By
–
Good question. I'd say LLaMA-Adapter and LoRA are both good methods. The advantage of LLaMA-Adapter though is that you can also embed multimodal inputs (images and others)
-
LoRA Implementation Added to Lit-Parrot Project
By
–
Just added LoRA the other day (PR still in progress: https://
github.com/Lightning-AI/l
it-parrot/pull/128#pullrequestreview-1477273133
…) and will do some comparisons soon! -
LLaMA-Adapter: Efficient LLM Fine-tuning with Tunable Prefixes
By
–
I really like the LLaMA-Adapter method! In a nutshell, it's about finetuning LLMs more efficiently. In a nutshell, it's adding a tunable prefix to the key and value tensors in the self-attention layers. Fun fact: it's not specific to LLaMA. Use it with any LLM!
-
FNet performance limitations compared to attention-based architectures
By
–
Yeah, for some applications it may be sufficient. But FNet doesn't outperform a contemporary attention-based architecture though
-
Humorous commitment to converting someone to PyTorch
By
–
maybe not this year, but I'll convert you to PyTorch eventually haha