ideally, your personal AI assistant should operate locally, accessing all your data directly on your device for maximum security and efficiency. for tasks that exceed local capabilities, it can selectively transfer only essential data to an online agent for assistance.

(sips tea)

Agree. I can build it myself, from available stuff, which is not a bad idea. 

The only issue I have now is how to repurpose slow devices for that. On my RPI 4, I get like 8 t/s for a very simple 1B model. For a multimodal, I will need dedicated, beefy hardware which leads to a high power consumption 😬

(sips mate)