AI Dynamics

Global AI News Aggregator

Serving ChatGPT to 100M users with 1k H100s efficiently

By

@rasbt

–

Load balancing, caching, dynamic batching… with 10% DAU you probably don’t need much more than 1k H100s to serve ChatGPT to 100 million customers.

→ View original post on X — @rasbt,

Commentaires

Leave a Reply

Your email address will not be published. Required fields are marked *

MORE ARTICLES