Giuliano Giacaglia pfp
Giuliano Giacaglia
@giu
"We took a look at LLMs and specifically GPT-3, and we thought GPT is actually pretty good at understanding code. And even though at the time a lot of people were really focused on the text generation use cases, we had heard that it was pretty good at code and so we wanted to try out showing it some suspicious code. And so we prototyped a system that did that. And it was interesting, but not that impressive. It got like very high false positive rate. But It seemed like a dead end. And so we kind of tabled it and didn't deploy it to production. Then GPT-4 came out and we were able to get early access to that through an investor. And, we switched over, like one line change to GPT-4 and suddenly it started performing much better. So it was still pretty high false positives, but at least now it was something like 1/3 findings was really, malicious"
0 reply
0 recast
43 reactions