Synthetic Query Generation for Privacy-Preserving Deep Retrieval Systems using Differentially Private Language Models

arXiv:2305.05973v3 Announce Type: replace-cross
Abstract: We address the challenge of ensuring differential privacy (DP) guarantees in training deep retrieval systems. Training these systems often involves the use of contrastive-style losses, which are typically non-per-example decomposable, making them difficult to directly DP-train with since common techniques require per-example gradients. To address this issue, we propose an approach that prioritizes ensuring query privacy prior to training a deep retrieval system. Our method employs DP language models (LMs) to generate private synthetic queries …

address arxiv challenge cs.cl cs.cr cs.ir differential privacy language language models losses making non privacy private query synthetic systems techniques train training