How did we do it? By loading the model weights into memory once before training begins and inserting them as numpy arrays into the #Ray object store, we can then zero-copy read the weights directly from shared memory into each GPU worker process.
SOFTWARE
-
Loading Pretrained Checkpoints: Multi-GPU Memory Challenge
By
–
Before you even get to multi-GPU training with model parallel frameworks like #Deepspeed, you need to load the pretrained checkpoint into memory. To make matters worse for machines with multiple GPUs, you need to load the checkpoint into host memory once for each GPU in your job!
-
SAP BTP Healthcare Response Pandemic Management Discussion
By
–
How did @SAP BTP help @uniklinik_hd respond to the #pandemic? Prof. Dr. Norbert Frey, @by_IT & I discuss how on @SAP #BetterTogetherStories. Listen in: https://
bit.ly/3fUclTq #healthtech -
ChatGPT Quality-of-Life Update: Persistent Login Feature
By
–
Quality-of-life updates to ChatGPT. No more constantly having to re-login!
-
Lessons from Building No-Code Data Analytics Platform
By
–
Overview of lessons learned: https://
cascade.io What we learned about the data and analytics market: https://
cascade.io/epilogue/the-d
ata-and-analytics-market
… What we learned about building a "no-code" toolkit: https://
cascade.io/epilogue/build
ing-a-no-code-toolkit
… What we learned about taking on an entrenched incumbent: -
H2O.ai and Snowflake Enable Secure Containerized ML Development
By
–
http://
H2O.ai and @SnowflakeDB enable developers to Train, Deploy, and Score Containerized Software Without Compromising Data Security. @h2oai Link to the blogpost: https://
h2o.ai/blog/h2o-ai-an
d-snowflake-enable-developers-to-train-deploy-and-score-containerized-software-without-compromising-data-security/
… -
Fine-tuning Llama-2 with Managed Autoscaling LLM Infrastructure
By
–
There are a lot of ways to #finetune LLaMa-2, but how many of these "solutions" address the #infra challenge? Check out our latest tutorial to learn how to fine-tune #Llama2 on top of fully managed, autoscaling #LLM infra right inside your VPC.
-
DataRobot Snowpark Integration Enables Production Model Deployment
By
–
DataRobot is on @Medium
! Our first post by Atalia Horenshtien dives into how the native integration between DataRobot and @SnowflakeDB #Snowpark addresses the core challenge of data science teams — building, deploying, and monitoring models in production that drive value for -
LangChain New Syntax Overview Live Discussion
By
–
A good overview of some samples of the new LangChain syntax! As a reminder, we're going live in ~25 minutes with the one and only @nfcampos to discuss the motivation, the interface, and some examples https://
crowdcast.io/c/ckw1tydg29er