Visual Transformer Meets CutMix for Improved Accuracy, Communication Efficiency, and Data Privacy in Split Learning. (arXiv:2207.00234v1 [cs.LG])

This article seeks for a distributed learning solution for the visual
transformer (ViT) architectures. Compared to convolutional neural network (CNN)
architectures, ViTs often have larger model sizes, and are computationally
expensive, making federated learning (FL) ill-suited. Split learning (SL) can
detour this problem by splitting a model and communicating the hidden
representations at the split-layer, also known as smashed data.
Notwithstanding, the smashed data of ViT are as large as and as similar as the
input data, negating the communication …

communication data data privacy lg privacy split learning