PrivLM-Bench: A Multi-level Privacy Evaluation Benchmark for Language Models

arXiv:2311.04044v2 Announce Type: replace-cross
Abstract: The rapid development of language models (LMs) brings unprecedented accessibility and usage for both models and users. On the one hand, powerful LMs achieve state-of-the-art performance over numerous downstream NLP tasks. On the other hand, more and more attention is paid to unrestricted model accesses that may bring malicious privacy risks of data leakage. To address these issues, many recent works propose privacy-preserving language models (PPLMs) with differential privacy (DP). Unfortunately, different DP implementations make …

accessibility art arxiv attention benchmark cs.cl cs.cr development evaluation language language models lms nlp paid performance privacy rapid state unprecedented