It's been a while and would have to check again, but wasn't BigBird a encoder LLM? Versus this is a decoder-style LLM for generating texts. But generally yes!

→ View original post on X — @rasbt