I think they made to common mistake during RAG-training of pretty much *always* having relevant context available, so the model never learns to ignore it.
RAG Training Flaw: Models Fail to Ignore Irrelevant Context
By
–
By
–
I think they made to common mistake during RAG-training of pretty much *always* having relevant context available, so the model never learns to ignore it.