it's on our list of things to do (more targeting checkpointing, not FSDP). I wrote up a plan / doc early last year about it.
Targeting checkpointing improvements planned for machine learning
By
–
By
–
it's on our list of things to do (more targeting checkpointing, not FSDP). I wrote up a plan / doc early last year about it.