No worries. And in this case, you are right but that's a smaller architecture, not a Llama 4 sized one trained from scratch. Otherwise, the original NoPE also had ablation studies
Llama 4 Architecture and NoPE Ablation Studies Analysis
By
–
By
–
No worries. And in this case, you are right but that's a smaller architecture, not a Llama 4 sized one trained from scratch. Otherwise, the original NoPE also had ablation studies