The Jamba whitepaper details our in-depth ablations on this novel hybrid SSM-Transformer architecture, and how we chose to interleave Mamba, Transformer and MoE.
Jamba Whitepaper: Hybrid SSM-Transformer Architecture Details
By
–
By
–
The Jamba whitepaper details our in-depth ablations on this novel hybrid SSM-Transformer architecture, and how we chose to interleave Mamba, Transformer and MoE.