Mamba-Hybrid ecosystem is growing

Zyphra’s Zamba-7B:

- a 7B Mamba/Attention hybrid model
- matches Mistral-7B & Gemma-7B performance w/ only 1T open training tokens
- surpasses Llama-2 7B & OLMo-7B
- set to release all training checkpoints under Apache 2.0

this model looks very strong.

Mamba-Hybrid ecosystem is growing

Zyphra’s Zamba-7B:

- a 7B Mamba/Attention hybrid model
- matches Mistral-7B & Gemma-7B performance w/ only 1T open training tokens
- surpasses Llama-2 7B & OLMo-7B
- set to release all training checkpoints under Apache 2.0

this model looks very strong.
https://www.zyphra.com/zamba

Hey, has anyone used a mamba model? Feedback?