We build the architecture from scratch and pre-train it on a small dataset. But we keep the pre-training short to keep it accessible. Then, we load pre-trained weights and further fine-tune it.

→ View original post on X — @rasbt,