gm8xx8

Reaching 1B Context Length With RAG

Zyphra: https://www.zyphra.com/post/reaching-1b-context-length-with-rag

retrieval system enables LLMs to process up to 1 billion tokens efficiently on a standard CPU using a sparse graph-based approach. Outperforms RAG methods with dense embeddings or long-context transformers.

I’m impressed with the work Zyphra has been doing in the SSM space (most recently Zamba2-7B) so I’m eager to see more.

Big if true—I was hesitant to share and even deleted my post on 𝕏 (for reasons) but given their work around SSM, I felt compelled to share. Also, magicailabs.twitter is trash (iykyk)