In the Pythia paper, they found that it doesn’t really make a difference if I recall correctly: https://
arxiv.org/abs/2304.01373 @BlancheMinerva may have some additional insights
Pythia Paper Findings on Model Architecture Differences
By
–
By
–
In the Pythia paper, they found that it doesn’t really make a difference if I recall correctly: https://
arxiv.org/abs/2304.01373 @BlancheMinerva may have some additional insights