Privacy-Preserving Diffusion Model Using Homomorphic Encryption

arXiv:2403.05794v1 Announce Type: new
Abstract: In this paper, we introduce a privacy-preserving stable diffusion framework leveraging homomorphic encryption, called HE-Diffusion, which primarily focuses on protecting the denoising phase of the diffusion process. HE-Diffusion is a tailored encryption framework specifically designed to align with the unique architecture of stable diffusion, ensuring both privacy and functionality. To address the inherent computational challenges, we propose a novel min-distortion method that enables efficient partial image encryption, significantly reducing the overhead without compromising the model's …

architecture arxiv called cs.ai cs.cr encryption framework homomorphic encryption privacy process protecting stable diffusion