Why You Should Not Trust Interpretations in Machine Learning: Adversarial Attacks on Partial Dependence Plots

arXiv:2404.18702v2 Announce Type: replace-cross
Abstract: The adoption of artificial intelligence (AI) across industries has led to the widespread use of complex black-box models and interpretation tools for decision making. This paper proposes an adversarial framework to uncover the vulnerability of permutation-based interpretation methods for machine learning tasks, with a particular focus on partial dependence (PD) plots. This adversarial framework modifies the original black box model to manipulate its predictions for instances in the extrapolation domain. As a result, it produces …

adoption adversarial adversarial attacks artificial artificial intelligence arxiv attacks box cs.cr cs.lg decision decision making framework industries intelligence led machine machine learning making partial stat.ap stat.ml tools trust uncover vulnerability