Likewise, your Deep (Learning) Focus newsletter is an absolute gem!
@rasbt
-
LLM Knowledge Retrieval Search Recommendation
By
–
Hm, maybe try to also search for "LLM knowledge retrieval". Off the top of my head, I can't make a particular recommendation, but @cwolferesearch probably has some resources handy there 🙂
-
SLURM cluster management: benefits and challenges for experimentation
By
–
That’s an interesting one. I have both fond and not so fond memories using SLURM on the departmental cluster. It’s actually great if you know exactly what you want to run. But yeah also make sure you have access to an interactive node for experimentation or you’ll go crazy 😛
-
PyTorch dominance in latest LLM and vision transformer research
By
–
That's totally true, but I think the problem is once you want to adopt or use more of the recent (research) stuff and/or need more customization. E.g., all the latest LLM and vision transformer is basically exclusively PyTorch.
-
PyTorch Preference: Balancing Ease of Use with Flexibility
By
–
For me it's PyTorch because I like the trade-off between being ease of use but also being flexible and customizeable. Plus almost all people I interact with (research and companies) are using PyTorch. But yeah, I think your mileage may vary based on your collaborators.
-
Fabric positioned between PyTorch and Trainer abstraction levels
By
–
Glad to hear it's useful! Btw if you are already using the Trainer, then you probably don't need Fabric. I would think of Fabric as a thing between pure PyTorch and using the Trainer 🙂
-
PyTorch Bridge: Minimal Code Changes for Lightning and HF Integration
By
–
Good point. It's essentially a stepping stone between PyTorch and the Lightning Trainer or HF Trainer. Lots of people have existing PyTorch code bases and don't want to do a bigger restructure of their code. So, here, you really only need to change a few lines.
-
Accelerating PyTorch Training with Mixed-Precision and Fully Sharded Data Parallelism
By
–
"Accelerating PyTorch Model Training Using Mixed-Precision and Fully Sharded Data Parallelism" https://
magazine.sebastianraschka.com/p/accelerating
-pytorch-model-training
… A little write-up of a talk I gave last week! -
Clarity needed on evaluation setup problems beyond consent issues
By
–
Fair, but it would have been more clear to spell out the problems with the evaluation setup in that letter. Reading the letter, it sounds like the main issue is with (dataset) consent etc.
-
Model Performance Varies by Dataset and Target Task
By
–
sure, ymmv based on the model, dataset, and target task. But I mean it in a more general terms.