when doing inference on lots of samples, how much speed up can you expect from increasing the batch size? context: i'm doing inference for lots of images using resnet100 on an a6000 gpu. increased batch size 32 -> 2048 (64x!) and only getting a 20% speedup. how is this possible?
@jxmnop
-
Embeddings Training Specificity and Cross-Model Compatibility
By
–
well they train this in; if you try this with other embeddings, it won’t work, for more or less the reasons the guy is saying in the middle
-
Axolotl AI Library Gains Popularity in Open Source Community
By
–
just heard there is a popular AI github library out there now called Axolotl. honestly always loved those little guys and hoped to be the first to name my software in their honor. however i did nothing; after years of twiddling my thumbs, the name is now taken. best of luck
-
Tweet Volume and Citation Impact: Correlation vs Causation
By
–
is this causal though? couldn’t people just be tweeting more about the papers they’re proud of which happen to garner more citations as well?
-
Dataset Sharding Performance Optimization Depends Multiple Factors
By
–
i think the speedup depends on a lot of stuff, like how big your dataset is, how big the shards are, how many CPUs you have, what disk you have, etc. Unless you have a really big dataset I doubt this will help you nearly as much.
-
Threading Integration Discussion in HuggingFace Datasets Library
By
–
i was looking this up and there's actually some discussion on eventually integrating this: http://
github.com/huggingface/da
tasets/issues/2252
… i think there are some complicated issues that make the threading hard to merge for the general case, not sure exactly why so i just post code here instead 🙂 -
Optimizing HuggingFace Dataset Loading Performance
By
–
wrote some faster dataloading code for HuggingFace datasets – sped up datasets.load_from_disk() from 17 minutes to around 30s the problem I was running into is that our virtual disk at school is really slow. HF loads datasets in a single thread and doesn't always memory-map.
-
Improving Model Accuracy with Nonlinear MLP Mapping Techniques
By
–
I just checked and my accuracy is more like 70%! with a linear mapping. it goes up with a nonlinear MLP.
