Evaluation of LLM Chatbots for OSINT-based Cyber Threat Awareness

arXiv:2401.15127v2 Announce Type: replace
Abstract: Knowledge sharing about emerging threats is crucial in the rapidly advancing field of cybersecurity and forms the foundation of Cyber Threat Intelligence (CTI). In this context, Large Language Models are becoming increasingly significant in the field of cybersecurity, presenting a wide range of opportunities. This study surveys the performance of ChatGPT, GPT4all, Dolly, Stanford Alpaca, Alpaca-LoRA, Falcon, and Vicuna chatbots in binary classification and Named Entity Recognition (NER) tasks performed using Open Source INTelligence (OSINT). …

arxiv awareness chatbots context cs.cl cs.cr cs.lg cti cyber cybersecurity cyber threat cyber threat intelligence emerging emerging threats evaluation forms foundation intelligence knowledge language language models large llm opportunities osint sharing study threat threat intelligence threats