For those interested in the implementation details, we’ve open-sourced the reference code for this paper. The repository includes our sparse training code and the custom CUDA kernels designed for H100 GPUs leveraging the TwELL packing format. GitHub:

→ View original post on X — @sakanaailabs