Finetuning Large Language Models for Vulnerability Detection

arXiv:2401.17010v2 Announce Type: replace
Abstract: This paper presents the results of finetuning large language models (LLMs) for the task of detecting vulnerabilities in source code. We leverage WizardCoder, a recent improvement of the state-of-the-art LLM StarCoder, and adapt it for vulnerability detection through further finetuning. To accelerate training, we modify WizardCoder's training procedure, also we investigate optimal training regimes. For the imbalanced dataset with many more negative examples than positive, we also explore different techniques to improve classification performance. The …

accelerate art arxiv code cs.ai cs.cr cs.lg detection finetuning improvement language language models large llm llms procedure results source code state task training vulnerabilities vulnerability vulnerability detection