Codex (and GPT-4) can’t beat humans on smart contract audits

By Artem Dinaburg, Chief Technology Officer; Josselin Feist, Principal Engineer; and Riccardo Schirone, Security Engineer

Is artificial intelligence (AI) capable of powering software security audits? Over the last four months, we piloted a project called Toucan to find out. Toucan was intended to integrate OpenAI’s Codex into our Solidity auditing workflow. This experiment went far beyond writing “where is the bug?” in a prompt and expecting sound and complete results.

Our multi-functional team, consisting of auditors, developers, and machine learning …

artificial artificial intelligence auditing audits beyond bug called chief chief technology officer codex contract engineer find gpt gpt-4 humans integrate intelligence officer openai project results security security engineer smart smart contract software software security solidity sound technology writing