Do SSL Models Have D\'ej\`a Vu? A Case of Unintended Memorization in Self-supervised Learning. (arXiv:2304.13850v1 [cs.CV])

Self-supervised learning (SSL) algorithms can produce useful image
representations by learning to associate different parts of natural images with
one another. However, when taken to the extreme, SSL models can unintendedly
memorize specific parts in individual training samples rather than learning
semantically meaningful associations. In this work, we perform a systematic
study of the unintended memorization of image-specific information in SSL
models -- which we refer to as d\'ej\`a vu memorization. Concretely, we show
that given the trained model and …

algorithms case images information meaningful ssl study taken training work