Online Safety Analysis for LLMs: a Benchmark, an Assessment, and a Path Forward

arXiv:2404.08517v1 Announce Type: cross
Abstract: While Large Language Models (LLMs) have seen widespread applications across numerous fields, their limited interpretability poses concerns regarding their safe operations from multiple aspects, e.g., truthfulness, robustness, and fairness. Recent research has started developing quality assurance methods for LLMs, introducing techniques such as offline detector-based or uncertainty estimation methods. However, these approaches predominantly concentrate on post-generation analysis, leaving the online safety analysis for LLMs during the generation phase an unexplored area. To bridge this gap, …

analysis applications arxiv assessment assurance benchmark cs.ai cs.cl cs.cr cs.lg cs.se fairness forward language language models large llms online safety operations path quality research robustness safe safety techniques