yeah the multiprocessing in this method isn't properly handled and hangs if you scale to millions of sentences. i (and others) have struggled with it a lot. you can just wrap your model in DataParallel and do a regular inference loop, no need for explicit multiprocessing

→ View original post on X — @jxmnop