AI Dynamics

Global AI News Aggregator

About

Continuous batching approach and batch size impact discussion

Yes, and their continuous batching approach – I figured it's largely orthogonal, although I wondered if it makes less of a difference there since it's not really a batch-size of 1 generally.

→ View original post on X — @jeremyphoward