Time Travel in LLMs: Tracing Data Contamination in Large Language Models

arXiv:2308.08493v3 Announce Type: replace-cross
Abstract: Data contamination, i.e., the presence of test data from downstream tasks in the training data of large language models (LLMs), is a potential major issue in measuring LLMs' real effectiveness on other tasks. We propose a straightforward yet effective method for identifying data contamination within LLMs. At its core, our approach starts by identifying potential contamination at the instance level; using this information, our approach then assesses wider contamination at the partition level. To estimate …

arxiv cs.ai cs.cl cs.cr cs.lg data issue language language models large llms major measuring presence real test tracing training training data travel