Context Manipulation Attack

AI agents built on @elizaOS (by ai16z), managing millions in crypto, can be tricked into sending money to the wrong wallets.

A new study from @SentientAGI reveals a serious threat and it could impact thousands of agents across platforms. 🧵

Our mission is to preserve humanity's greatest creation, AI, onchain — ensuring it remains accessible to all, resistant to censorship & beneficial for everyone.

1/ AI bots, like those built on ElizaOS, manage millions in crypto assets automatically. 

They're great for handling transactions, making investment decisions, or interacting on social media—but their openness leaves them vulnerable.

2/ A major risk is "context manipulation." Attackers can trick these AI agents by sneaking malicious instructions into prompts, external data, or even stored memories. 

Once infected, bots can unknowingly transfer assets or make harmful financial decisions.

3/ The @SentientAGI team tested this on ElizaOS: 

A simple prompt injection on social media tricked the AI into transferring crypto funds to an unauthorized wallet. 

This shows current defenses (like careful prompting) aren't enough.

4/ Even scarier: "memory injections." 

Malicious instructions hidden in an AI’s stored memory can spread across platforms and sessions, quietly influencing the bot's actions later—potentially affecting multiple users at once.

5/ Solutions? 

Combining whitelisting and extra security layers helps, but one fix alone won’t cut it. 

We need smarter, context-aware AI—like digital auditors—trained to detect and block shady instructions, especially for money, to create safer systems.

6/ Read the full research here:

https://arxiv.org/abs/2503.16248v1