Isometric Representations in Neural Networks Improve Robustness. (arXiv:2211.01236v1 [cs.LG])

Artificial and biological agents cannon learn given completely random and
unstructured data. The structure of data is encoded in the metric relationships
between data points. In the context of neural networks, neuronal activity
within a layer forms a representation reflecting the transformation that the
layer implements on its inputs. In order to utilize the structure in the data
in a truthful manner, such representations should reflect the input distances
and thus be continuous and isometric. Supporting this statement, recent
findings …

networks neural networks robustness