And another recent one. That one shows that too much duplicated data (here, ~ training for many epochs) leads to overfitting:
@rasbt
-
Pythia Paper Findings on Model Architecture Differences
By
–
In the Pythia paper, they found that it doesn’t really make a difference if I recall correctly: https://
arxiv.org/abs/2304.01373 @BlancheMinerva may have some additional insights -
New NoPE Paper Released on ArXiv
By
–
Not yet not yet! But you might like this new NoPE paper: https://
arxiv.org/abs/2305.19466 -
Comparison of Rank Consistent Methods: Performance Against CORAL
By
–
And b) it has better predictive performance than other rank consistent methods like CORAL. Here's a side-by-side comparison in terms of how the methods differ.