DP-RDM: Adapting Diffusion Models to Private Domains Without Fine-Tuning

arXiv:2403.14421v2 Announce Type: replace-cross
Abstract: Text-to-image diffusion models have been shown to suffer from sample-level memorization, possibly reproducing near-perfect replica of images that they are trained on, which may be undesirable. To remedy this issue, we develop the first differentially private (DP) retrieval-augmented generation algorithm that is capable of generating high-quality image samples while providing provable privacy guarantees. Specifically, we assume access to a text-to-image diffusion model trained on a small amount of public data, and design a DP retrieval …

algorithm arxiv cs.cr cs.cv cs.lg diffusion models domains fine-tuning high image images issue may near perfect private quality remedy sample text