fam in ~18 hours we'll have 157 million casts with embeddings, pray for me it doesn't crash overnight

Have you considered int8 embeddings?

https://huggingface.co/blog/embedding-quantization

no i didn't even know that was a thing, thank you!!!!!! if i can't get the FP16 working i'll switch to this

right now i'm working on uploading all the chunks to: https://huggingface.co/datasets/jc4p/farcaster-casts-embeddings/tree/main

then was planning on nuking the HDs and resuming from the last chunk on each instance (it's 4xGH200)

Here's a notebook to help you get started.

Here's a notebook to help you get started.

https://github.com/mixedbread-ai/binary-embeddings/blob/main/mxbai_binary_quantization.ipynb