1. there is no fork of pytorch fwiw. only a fork of nccl. clearly you overheard wrong.
2. torch.compile wasn't even considered because it doesnt have precompilation yet and that was deemed a requirement for large-scale xlformers runs. so the whole thing about graph breaks is
PyTorch Compilation Strategy for Large-Scale Transformer Models
By
–