Content
@
0 reply
0 recast
0 reaction
phil
@phil
What are the best tools for running a scaleable cloud instance of Llama3 for text generation? It doesn't need to go into production, but I want to test something on a machine larger than my MBP. I've tried Replit (not supported) and AWS (way too complicated for such a simple task). Any obvious tools?
4 replies
2 recasts
11 reactions
Jason
@jachian
+1 to Modal having good docs for sample code to set up a Llama instance Other option is to use some of these secondary hosting platforms for open source models like Lorax on Predibase
0 reply
0 recast
0 reaction
