- cross-posted to:
- [email protected]
- [email protected]
- programming
- Aii
- cross-posted to:
- [email protected]
- [email protected]
- programming
- Aii
cross-posted from: https://feddit.org/post/28915274
cross-posted from: https://feddit.org/post/28915273
[…]
That marketing may have outstripped reality. Early reports from Mythos preview users including AWS and Mozilla indicate that while the model is very good and very fast at finding vulnerabilities, and requires less hands-on guidance from security engineers - making it a welcome time-saver for the human teams - it has yet to eclipse human security researchers.
“So far we’ve found no category or complexity of vulnerability that humans can find that this model can’t,” Mozilla CTO Bobby Holley said, after revealing that Mythos found 271 vulnerabilities in Firefox 150. Then he added: “We also haven’t seen any bugs that couldn’t have been found by an elite human researcher.” In other words, it’s like adding an automated security researcher to your team. Not a zero-day machine that’s too dangerous for the world.
Usually with new security tools, e,g. fuzzers, you catch a whole bunch of bugs, and then that class of bugs is essentially eliminated, but the security arms race switches to different classes of bugs not solved by the tools. So yôu have a big initial peak of bugs found/fixed that slows to trickle. Remains to be seen if LLMs follow the same pattern.