Justin Hunter on Warpcast

Content pfp

0 reply

0 recast

0 reaction

Justin Hunter pfp

@polluterofminds

Maybe I’m being very naive, but why can’t LLMs audit smart contracts? Feels like there’s plenty of public data for the models to be trained on. https://warpcast.com/polluterofminds/0x0a371693

13 replies

0 recast

5 reactions

✳️ dcposch on daimo pfp

✳️ dcposch on daimo

Definitely won’t replace a careful audit from a team that actually understands the spec and context but I could see it being a useful first-pass audit to flag potential bugs Or places where comments and code don’t match Like static analysis, but heuristic and kind of vibes based

1 reply

0 recast

2 reactions

Kyle pfp

Here’s a cool paper on vulnerability detection: https://arxiv.org/pdf/2310.01152.pdf

1 reply

0 recast

1 reaction

notdevin pfp

You can give any code to gpt to assess, as always though, quality outcomes are predicated on the quality of direction you provide to the LLM

1 reply

0 recast

1 reaction

Matt pfp

I think it’s that not all of the public contracts it’s trained on are secure

1 reply

0 recast

1 reaction

briandoyle81 🌊🎩 pfp

briandoyle81 🌊🎩

I can't remember if it's public or not, so I won't say who, but I've heard of someone making a serious push on this.

1 reply

0 recast

1 reaction

Wesley pfp

I usually punch my contracts into chat gpt but it’s still a bit hit or miss. Not that great at specific errors but it’s good at suggestion what types of vulnerabilities you might have given the nature of the code

1 reply

0 recast

1 reaction

🤷🚂👲🧑‍🤝‍🧑 pfp

🤷🚂👲🧑‍🤝‍🧑

something like chaosnets? https://mirror.xyz/apeworx.eth/mIKYEt54RgNs7R12Np1vEyp8z5HuBKBdCjCWqrnNfe8 idk if I'd trust a service that just does static analysis.

1 reply

0 recast

0 reaction

tldr (tim reilly) pfp

tldr (tim reilly)

1 reply

0 recast

2 reactions

caz.eth pfp

At this stage it should probably be humans verifying LLMs rather than the other way round.

0 reply

0 recast

1 reaction

Constant pfp

Maybe you could use an LLM to write a fuzzer for the contract.

0 reply

0 recast

1 reaction

typeof.eth 🔵 pfp

typeof.eth 🔵

IMO, they’re a good first pass at an audit now, and it’s only matter of time until they can basically replace auditing firms for most contracts. They can also do a large part of the work of audits, so audit prices will come down significantly since the value they provide will be on the margins.

1 reply

0 recast

1 reaction

Cameron Armstrong pfp

Cameron Armstrong

@briang care to comment

1 reply

0 recast

1 reaction

kevin mcliney pfp

great Q — I’ll take a stab LLMs are trained with historical datasets (which could work for teams looking to just check a box and say they’ve been audited) given the nature of zero days, an LLM can’t predict exploits that haven’t yet occurred just my 2¢

0 reply

0 recast

0 reaction