You should try text-generation-inference, that’s what we’re using at HuggingFace. Don’t use the transformers library for generation with large LM. https://
github.com/huggingface/te
xt-generation-inference
…

→ View original post on X — @thom_wolf,