Optimistic Verifiable Training by Controlling Hardware Nondeterminism

arXiv:2403.09603v1 Announce Type: new
Abstract: The increasing compute demands of AI systems has led to the emergence of services that train models on behalf of clients lacking necessary resources. However, ensuring correctness of training and guarding against potential training-time attacks, such as data poisoning, poses challenges. Existing works on verifiable training largely fall into two classes: proof-based systems, which struggle to scale due to requiring cryptographic techniques, and "optimistic" methods that consider a trusted third-party auditor who replicates the training …

arxiv attacks challenges clients compute correctness cs.ai cs.cr cs.lg data data poisoning demands hardware led poisoning resources services systems train training