Kasra Rahjerdi
@jc4p
(screams internally)
8 replies
4 recasts
47 reactions
vanishingideal
@vanishingideal
Have you considered int8 embeddings? https://huggingface.co/blog/embedding-quantization
1 reply
0 recast
4 reactions
Kasra Rahjerdi
@jc4p
no i didn't even know that was a thing, thank you!!!!!! if i can't get the FP16 working i'll switch to this
2 replies
0 recast
1 reaction
