but yeah, I am training classifiers in both cases, the same memory-saving techniques could potentially be applied to the recent transformers for tabular data.
@rasbt
-
Vision Transformers for Images and LLMs for Text
By
–
Oh this post is about vision transformers for images and LLMs for text
-
Reducing GPU Memory for LLMs and Vision Transformers in PyTorch
By
–
One of the big bottlenecks with LLMs & Vision Transformers is GPU memory on consumer devices. Wrote about my favorite techniques for reducing peak memory in PyTorch: https://
lightning.ai/pages/communit
y/tutorial/pytorch-memory-vit-llm/
… Focused on techniques that don't require architecture changes! Suggestions welcome! -
ChatGPT’s Knowledge Cutoff Limitation and Synthetic Data Challenges
By
–
3/3 Intuitively one major criticism of ChatGPT is it's 2020 knowledge cut-off. If you are only using synthetic data from old models, it'd be challenging to get more contemporary knowledge into the model.
-
Synthetic Data Generation Limitations in Predictive Model Improvement
By
–
2/2 I think there's been a lot of research on generating synthetic data to boost predictive models (incl. classifiers). The concern is that you'll never be able to generate anything "new" by recycling models (trained on a fixed dataset) that generate synthetic data.
-
Synthetic Data: Predictive Modeling vs Generative Modeling Distinction
By
–
I haven't heard of it yet, but it looks like it's related to generating medical images for image segmentation tasks. Looks like a solid approach. I think a crucial difference though is to distinguish between synthetic data for predictive modeling vs generative modeling 1/2.
-
New Machine Learning and AI Book Confidence Interval Reference
By
–
Nice, I haven't seen that, yet! Thanks for the ping! Should probably reference that in the confidence interval section of my new Machine Learning Q and AI book
-
Attending favorite computing conference and preparing deep learning tutorial
By
–
Pretty excited to me attending my favorite computing conference once more! If I’ve been a bit quiet on socials lately that’s because I’m head down working on my deep learning tutorial for SciPy in a bit more than a week! Hope to see you there!
-
Specialized AI assistants preferred over general purpose for business
By
–
That’s only true for general purpose assistants. But for most business applications, you don’t want “general purpose”
-
Strategic Synthetic Data Generation Beyond Random Sampling
By
–
Synthetic data is a broad term. I think the paper looks at a very extreme case. One can blindly generate synthetic data by sampling from an upstream model. But yeah, I can see how being more strategic than that can actually be beneficial.