Symmetry-Preserving Program Representations for Learning Code Semantics. (arXiv:2308.03312v2 [cs.LG] UPDATED)

Large Language Models (LLMs) have shown promise in automated program
reasoning, a crucial aspect of many security tasks. However, existing LLM
architectures for code are often borrowed from other domains like natural
language processing, raising concerns about their generalization and robustness
to unseen code. A key generalization challenge is to incorporate the knowledge
of code semantics, including control and data flow, into the LLM architectures.

Drawing inspiration from examples of convolution layers exploiting
translation symmetry, we explore how code symmetries …

aspect automated challenge code domains key language language models large llm llms natural natural language natural language processing program reasoning robustness security