On the Limits of Data Poisoning Attacks

Abstract

Current machine learning models, trained on large amounts of data from untrusted sources, are susceptible to data poisoning attacks. Contrary to the commonly perceived devastating impact of these attacks, this talk aims to demystify the actual limits of two types of data poisoning: indiscriminate, which compromises model performance globally, and subpopulation, targeting performance on specific subgroups. To address this, I will first introduce an empirical poisoning attack that is applicable to both indiscriminate and subpopulation objectives, demonstrating its state-of-the-art performance and establishing tighter lower bounds on poisoning limits. Second, I will explore how certain characteristics of subpopulations correlate with the success of the best-known attacks, noting that some groups naturally resist poisoning. Third, I will delve into how the distributional properties of data affect the success of indiscriminate poisoning, identifying distributions that are inherently more robust to such attacks. This insight leads to new defensive strategies, such as leveraging improved feature representations to enhance robustness. Finally, I will also briefly discuss some of my future research plans.

Biography

Fnu Suya is a MC2 postdoctoral fellow at the Maryland Cybersecurity Center, University of Maryland, College Park. Previously, he obtained his PhD in computer science from the University of Virginia, advised by Professor of Computer Science David Evans and Visiting Assistant Professor of Computer Science Yuan Tian. His research area is in trustworthy machine learning and machine learning for security, with an interest in the realistic analysis of risks associated with deploying machine learning models in real-world scenarios, especially in contaminated training environments. Suya's work has been published in top-tier conferences, including Usenix Security, IEEE S&P, CVPR, ICML, and NeurIPS. He received a best paper award at the VISxAI workshop in 2022 and has been recognized as a top reviewer for ICLR and NeurIPS.