Hey all, maybe you can help me? Thinking of a small RAG project, but would prefer to run it locally. My idea would be to use langchain, vectordb, but I'm not quite sure on the inference/embeddings part yet. Is there a commonly used way to generate embeddings from documents? Are there local ways to generate them? Thanks for helping a dummy out!

I write code and sometimes stories. jakeacasey.com

Building /frametown. Amusement, wonder, and uncertainty ✴︎ libred.org


You can simplify your stack further by just iterating over all your embeddings. There is no need for vector dbs for single-machine low-volume use case, like yours. Plus, if it’s a learning exercise, you can learn a lot of about matrices :)

Building https://srcbook.com // creator of the https://blockchainsmokers.xyz

js. Is there things I should be looking out for when evaluating/picking the embedding model? I just want to give it a shot, so think like... regular pdfs full of dumb random info. I've definitely looked at the langchain guide for rag, more trying to figure out what I don't know yet, haha.

Yes. There are local ways.

You need to pick an embedding model and a vector DB.

Are you doing python or JavaScript?

Your best bet is to find a langchain or llama index guide. There should be some doing exactly this