Since Hinton has actually replied let me clarify some things – LLMS don’t *always* regurgitate
– LLMs don’t literally store full texts
– but given the mechanisms that they use they do sometimes regurgitate long passes through their process of resynthesis
– the evidence for that

→ View original post on X — @garymarcus