Watermarking Makes Language Models Radioactive

arXiv:2402.14904v1 Announce Type: new
Abstract: This paper investigates the radioactivity of LLM-generated texts, i.e. whether it is possible to detect that such input was used as training data. Conventional methods like membership inference can carry out this detection with some level of accuracy. We show that watermarked training data leaves traces easier to detect and much more reliable than membership inference. We link the contamination level to the watermark robustness, its proportion in the training set, and the fine-tuning process. …

accuracy arxiv can cs.ai cs.cl cs.cr cs.lg data detect detection easier generated input language language models llm texts traces training training data watermarking