"Memory Decoder at Scale: A Pretrained, Parametric Long-Term Memory" Most LMs still store knowledge and reasoning in the same weights, so making the model remember more usually means scaling the whole backbone. This paper makes long-term memory a separate pretrained module. It
Memory Decoder Separates Long-Term Language Model Memory Into Pretrained Module
By
–
