Has anyone here been able to use small (~7B params) open source models for anything remotely useful? I mean it’s incredibly cool that you can run them locally on your phone but they’re just dumb as rocks. The only example I can think of is the model underlying autocorrect in iOS which AFAIK runs on device.

Use a Mistral fine tune via Ollama in your laptop to get better flavor for where things are going. LLMs are not really good for storing facts but combined with external search I think going to be useful in the near term.

Yeah, I think that's where the current LLM approach isn't the final say in AI agents. The idea of storing all of the world's knowledge inside the LLM is kind of weird if you think about it. Would be much better to have a universal processing engine that can understand human language and act on it. 1/

Computational biologist, data scientist, digital artist | 
https://clauswilke.art/links | Hosting /ai-art & /clauswilke

If it encounters topics/materials it doesn't know, it just goes out and searches for them/reads wikipedia/whatever. That's how humans operate and why shouldn't an AI do the same? Then the model only needs to be large enough for basic language capabilities, and that should easily fit on a phone. 2/