On the Difficulty of Defending Self-Supervised Learning against Model Extraction. (arXiv:2205.07890v1 [cs.LG])

Self-Supervised Learning (SSL) is an increasingly popular ML paradigm that
trains models to transform complex inputs into representations without relying
on explicit labels. These representations encode similarity structures that
enable efficient learning of multiple downstream tasks. Recently,
ML-as-a-Service providers have commenced offering trained SSL models over
inference APIs, which transform user inputs into useful representations for a
fee. However, the high cost involved to train these models and their exposure
over APIs both make black-box extraction a realistic security threat. …

lg