Resilience of Entropy Model in Distributed Neural Networks

arXiv:2403.00942v1 Announce Type: cross
Abstract: Distributed deep neural networks (DNNs) have emerged as a key technique to reduce communication overhead without sacrificing performance in edge computing systems. Recently, entropy coding has been introduced to further reduce the communication overhead. The key idea is to train the distributed DNN jointly with an entropy model, which is used as side information during inference time to adaptively encode latent representations into bit streams with variable length. To the best of our knowledge, the …

arxiv coding communication computing cs.ai cs.cr cs.lg distributed edge edge computing entropy idea key networks neural networks performance resilience systems the key train