aichannel

some insights from my attempt to run full deepseek r1 model.

you will need at least 6x a100/h100 level gpus to run q4_k_m quantization.

i rented 8x a100 for $14/hr, so i had 640gb vram at my disposal. the quickest way to set this up is ollama. model download size is 404gb.

the…

banteg

creating and destroying; built flying cars & autonomous vehicles, now building roc.camera, writings: july.rocks

@july wdyt? cc @banteg 

https://x.com/bantg/status/1884972148722225610

Working on Farcaster and Warpcast. Longer thoughts at https://dwr.email

@banteg where did you get your GPUs for $14/hr?

Follow to get the best of Farcaster using @mbd - AI Recommendation Engine for Web3. @a16zcrypto css23. Ex-ML Lead @Chainlink. Ex-Sr Data Scientist @AWS.