ALBERTA: ALgorithm-Based Error Resilience in Transformer Architectures

Vision Transformers are being increasingly deployed in safety-critical applications that demand high reliability. It is crucial to ensure the correctness of their execution in spite of potential errors such as transient hardware errors. We propose a novel algorithm-based resilience framework called ALBERTA that allows us to perform end-to-end resilience analysis and protection of transformer-based architectures. First, our work develops an efficient process of computing and ranking the resilience of transformers layers. We find that due to the large size of …

alberta algorithm analysis applications architectures called correctness critical cs.cr cs.dc demand end end-to-end error errors framework hardware high novel protection reliability resilience safety safety-critical transformers