Good point. It's essentially a stepping stone between PyTorch and the Lightning Trainer or HF Trainer. Lots of people have existing PyTorch code bases and don't want to do a bigger restructure of their code. So, here, you really only need to change a few lines.
@rasbt
-
Accelerating PyTorch Training with Mixed-Precision and Fully Sharded Data Parallelism
By
–
"Accelerating PyTorch Model Training Using Mixed-Precision and Fully Sharded Data Parallelism" https://
magazine.sebastianraschka.com/p/accelerating
-pytorch-model-training
… A little write-up of a talk I gave last week! -
Clarity needed on evaluation setup problems beyond consent issues
By
–
Fair, but it would have been more clear to spell out the problems with the evaluation setup in that letter. Reading the letter, it sounds like the main issue is with (dataset) consent etc.
-
Model Performance Varies by Dataset and Target Task
By
–
sure, ymmv based on the model, dataset, and target task. But I mean it in a more general terms.
-
LLMs Simplified: Pretraining and Fine-tuning Made Easy with PyTorch
By
–
The "Sweet Lesson" is that using LLMs is simpler than we thought! Instead of complicated symbolic computations, we download a pretrained open-source LLM and finetune it in supervised fashion via <100 lines of PyTorch code.
-
Critique des problèmes méthodologiques dans l’évaluation GPT-4
By
–
> "what hit most was that it got uploaded without coauthor permission" Sure, this was another issue. But wasn't the core issue the flawed evaluation setup the authors used? Using GPT-4 to score itself? And repeated prompting until the answer was correct, thus reaching 100%?
-
Critique of Flawed Research Paper and Inadequate Apology
By
–
"[…] could have been a very interesting and valuable paper had the data been collected with consent" So, if people gave consent for using the data, the flawed paper would become interesting and "valuable"? Long story short: Problematic paper, problematic apology letter.
-
Dataset Consent vs Flawed Evaluation: Main Issues Debate
By
–
Hm, it seems the letter largely highlights the issues regarding dataset consent. Of course, this is an issue. But wasn't the main issue of this study the flawed evaluation and, consequently, the wrong claims?
-
No Prominent Imitation Model for Coding Yet
By
–
No worries! On that note, I don't think there is an imitation model for coding, yet. At least nothing prominent as far as I know.
-
Blocking ML and AI researchers is closed-minded and counterproductive
By
–
No, not that I’m aware. But yeah, arbitrarily blocking ML & AI researchers is not a good look and quite close-minded. I honestly don’t want to waste time on people who behave like that.