AI Dynamics

Global AI News Aggregator

Jamba Modeling and Mamba Cache Management Updates

Things in this PR:
– Jamba modeling file, supporting Jamba variations
– Mamba cache management (also benefit other Mamba-based models)
– Send more request related properties to the forward pass, for model-specific implementations (including things needed for speculative decoding)

→ View original post on X — @ai21labs,

Commentaires

Leave a Reply

Your email address will not be published. Required fields are marked *