i was looking this up and there's actually some discussion on eventually integrating this: http://
github.com/huggingface/da
tasets/issues/2252
… i think there are some complicated issues that make the threading hard to merge for the general case, not sure exactly why so i just post code here instead 🙂
CODE
-
Threading Integration Discussion in HuggingFace Datasets Library
By
–
-
Optimizing HuggingFace Dataset Loading Performance
By
–
wrote some faster dataloading code for HuggingFace datasets – sped up datasets.load_from_disk() from 17 minutes to around 30s the problem I was running into is that our virtual disk at school is really slow. HF loads datasets in a single thread and doesn't always memory-map.
-
TokenFlow Bug Fix Release New Version
By
–
We've pushed a new version that fixes this bug https://
replicate.com/cjwbw/tokenflo
w/versions/d456a26c0c1764ac4d9b933a564fe70e660c46423ba0afb5d3200f1a46060728
… -
GPT-4 limitations for large-scale data automation use cases
By
–
I have no objections to using automation to do large-scale data processing. But it needs to be developed & tested for the use-case. "zeroshot" (ugh) or not, GPT-4 isn't that. You don't even know what its training data is.
-
TEL launches Model Lab tutorial for training inference cost optimization
By
–
Our Japanese partner TEL has put together a great English tutorial of Model Lab, a tool we built to help you balance training vs. inference costs for different model/data sizes. Check it out!
-
ComfyUI Workflow API Integration Now Available
By
–
Want to run your ComfyUI workflow with a single API call? We got you https://t.co/tXKmQam01x
— Replicate (@replicate) 22 janvier 2024Want to run your ComfyUI workflow with a single API call? We got you
-
ComfyUI Workflows Shared in API Format on GitHub
By
–
I've also created a place to share ComfyUI workflows in the API format here:https://t.co/Ezg01WWY7d https://t.co/XhmUQTWG7Q
— fofr (@fofrAI) 22 janvier 2024I've also created a place to share ComfyUI workflows in the API format here: https://
github.com/fofr/cog-comfy
ui/discussions/1
… -
Model feature not appearing in latest Nightly build
By
–
Do I have to enable this somewhere? I'm on the latest Nightly build, but I don't see the option for this model
-
Most Incredible AI Tools Released Last Two Weeks
By
–
The most incredible AI tools that came out in the last 2 weeks:
-
ComfyUI and ComfyDeploy: Open Source Production AI Deployment
By
–
I'd also like to point out that this is just one of the open source ways to use ComfyUI in production. @BennyKokMusic is doing great work on ComfyDeploy: