SYSTEMS

→ View original post on X — @mjasay

Database Types and Their Varying Data Handling Capabilities

By

@mjasay

–

01 May 2023 18h28

I was just looking at old decks and Jared's use of that image was all over them. It *is* true, if perhaps overstated. I think we can agree that different types of databases handle different kinds of data better (or worse).

Containerization and Kubernetes scaling for ML workloads

By

–

01 May 2023 16h00

1) Containerization provides consistent, reproducible behavior required for ML workloads.
2) Scaling gets first-class support in Kubernetes, unlocking scaling for training and experimentation.

Ray Parallelization and Job Checkpointing for Distributed AI

By

–

01 May 2023 16h00

3) Leveraging Ray to parallelize work at large scale across multiple worker pods in the cluster to achieve performance benchmarks
4) Implementing job checkpointing ensures that jobs always run to completion and users see minimal interruption.

Autoscaling compute clusters for parallel job execution

By

–

01 May 2023 16h00

5) Enabling autoscaling of the compute cluster to execute more jobs in parallel during periods of high activity and save cost during periods of low activity.

Kubernetes Lessons Learned Running ML at Snorkel

By

–

01 May 2023 16h00

Snorkel tackles heavy ML workloads for some of the world's largest organizations. We have built our infrastructure on Kubernetes—even though it wasn't designed for ML! Here are a few lessons learned from @wheeliamhuang
, Tech Lead Manager at Snorkel https://
snorkel.ai/kubernetes-les
sons-learned-at-snorkel-ai/
…