MACHINE LEARNING

FlashAttention and KV Caching for Improved Model Performance

By

–

29 June 2023 16h11

No, absolutely. But I think for this it's maybe worthwhile adopting methods that don't negatively impact modeling performance, like FlashAttention and kv caching.

→ View original post on X — @marktabnet

AutoGPT: Self-Iterating AI Agents with GPT-3.5 and GPT-4

By

@marktabnet

–

29 June 2023 16h07

AutoGPT combines GPT-3.5 and GPT-4 via API, allowing projects to be created that have been iterating on their own prompts and reviewing each iteration to improve and build upon it. AutoGPT: Everything You Need To Know – KDnuggets https://
bit.ly/43Z1f7X

State-of-the-Art Models First, Efficiency Through Distillation Later

By

–

29 June 2023 16h04

For most, the goal is to push the state-of-the-art and show that these models can achieve impressive results. You can always distill models later to make them more efficient.

Trade-offs Between Efficiency and Model Performance in Pretraining

By

–

29 June 2023 16h03

You are right. But yeah the point is that these are based on assumptions and produce approximations. You'd hurt modeling performance and people don't want that, which is why the "efficient alternatives that do approximations" for/during pretraining haven't caught on.

Low-Rank Matrices in Task-Specific Training vs Pretraining

By

–

29 June 2023 15h59

I think we are talking about the same thing. That would only work after training. It's based on the assumption that the matrices have low rank for a specific target task but are not low rank in general (for the pretraining tasks).

LoRA Weight Matrix Factorization During Fine-tuning Explained

By

–

29 June 2023 15h44

You mean as in LoRA (low-rank adaptation)? You are factorizing the weight matrices but only during finetuning. The self-attention mechanism computation is still the same (if you ignore that the weight matrices are different).

→ View original post on X — @id_aa_carmack

GPU Heat and Noise Issues Require Laundry Room Setup

By

@id_aa_carmack

–

29 June 2023 15h43

I considered it still too hot and noisy to be in the same room with. I probably need to put a rack in the laundry room if I still want ML GPUs now.

Self-Attention Alternatives Failed; FlashAttention Gains Popularity

By

–

29 June 2023 15h20

You mean alternatives to self-attention specifically (versus parameter efficient finetuning etc.)? I think none of them stood the test of time. I think the problem is that they are all approximations. The relatively recent FlashAttention mechanism is super popular though.

Reward Model Training vs User Feedback: Preferences and Finetuning

By

–

29 June 2023 15h16

Good question. In their original finetuning, they train a reward model based on relative preference (rankings among multiple responses). And from the user feedback, there's only thumbs up & down. You can probably use that for supervised finetuning I guess.