https://x.com/PratyushRT/status/1873428775171670435

Things like this are a big part of why I'm skeptical of just putting a single AI in charge of things (eg. DAOs, funding mechanisms), and favor an open market where many AIs can participate and humans judge AIs based on some sample of outcomes.

Backdoor attacks: The hidden threat to open-source LLMs

Open-source LLMs efforts are very important as they democratize access to intelligence. But there is a catch we need to talk about more - especially as more organizations rush to release open-source models. For models…

Pratyush Ranjan Tiwari

I think we should treat AI agents the same way we treat human agents. For example:

1. Tests and exams that lead to certifications: test how they perform in specific scenarios, just like we would test a new hire, a driver, or a doctor. 

2. History: An agent's history can be treated as a long term test. It's an agent's CV: "This model has been working for us for 10 years, it has dealt with these cases, and it has done ok, with the exception of cases of type X."

3. Stake: An agent should be able to bear the consequences of bad decisions. A stake is a way to implement this, and also to measure the amount of trust that can be put on one. (Stake could also have the for of insurance, or something similar).

Dad, geek, conversation liquidity provider. vrypan.net (home), /weatherxm (work), @l--o--l (fun). Also https://paragraph.xyz/@purplesubmarine