3/5 Sequence packing helps for transformers, but it relies on architecture-specific support that is often missing and introduces implementation risks for non-transformer or hybrid models like @AI21Labs' Jamba.

→ View original post on X — @ai21labs,