Finetuning Large Language Models for Vulnerability Detection. (arXiv:2401.17010v1 [cs.CR])

This paper presents the results of finetuning large language models (LLMs)
for the task of detecting vulnerabilities in source code. We leverage
WizardCoder, a recent improvement of the state-of-the-art LLM StarCoder, and
adapt it for vulnerability detection through further finetuning. To accelerate
training, we modify WizardCoder's training procedure, also we investigate
optimal training regimes. For the imbalanced dataset with many more negative
examples than positive, we also explore different techniques to improve
classification performance. The finetuned WizardCoder model achieves
improvement …

accelerate art arxiv code detection finetuning improvement language language models large llm llms procedure results source code state task training vulnerabilities vulnerability vulnerability detection