When people bring up how the human brain works just like a Transformer, I remember those simpler times when people argued for the biological plausibility of LSTM (back when a stack of LSTM layers trained with deep RL seemed like the golden path to AGI). https://arxiv.org/abs/1604.06635