all InfoSec news
Evaluation of ChatGPT Model for Vulnerability Detection. (arXiv:2304.07232v1 [cs.CR])
cs.CR updates on arXiv.org arxiv.org
In this technical report, we evaluated the performance of the ChatGPT and
GPT-3 models for the task of vulnerability detection in code. Our evaluation
was conducted on our real-world dataset, using binary and multi-label
classification tasks on CWE vulnerabilities. We decided to evaluate the model
because it has shown good performance on other code-based tasks, such as
solving programming challenges and understanding code at a high level. However,
we found that the ChatGPT model performed no better than a dummy …
binary challenges chatgpt classification code code vulnerability cwe detection evaluation gpt gpt-3 high performance programming report task technical understanding vulnerabilities vulnerability vulnerability detection world