Yes. There was a nice paper on that, which looked into some other modifications: Simplifying Transformer Blocks, https://
arxiv.org/abs/2311.01906

→ View original post on X — @rasbt