Bi-Directional Transformers vs. word2vec: Discovering Vulnerabilities in Lifted Compiled Code

arXiv:2405.20611v1 Announce Type: new
Abstract: Detecting vulnerabilities within compiled binaries is challenging due to lost high-level code structures and other factors such as architectural dependencies, compilers, and optimization options. To address these obstacles, this research explores vulnerability detection by using natural language processing (NLP) embedding techniques with word2vec, BERT, and RoBERTa to learn semantics from intermediate representation (LLVM) code. Long short-term memory (LSTM) neural networks were trained on embeddings from encoders created using approximately 118k LLVM functions from the Juliet …

address arxiv bert code compilers cs.cl cs.cr cs.lg cs.se dependencies detection high language lost natural natural language natural language processing nlp optimization options research techniques transformers using vulnerabilities vulnerability vulnerability detection