Adversarial attacks are considered as one of the most critical security threats for Machine Learning (ML). These attacks apply small perturbations to some original examples in order to produce adversarial examples, specifically designed to fool ML model decision.

In order to enable the secure deployment of ML models in the real world, it is essential to properly assess their robustness to adversarial attacks and develop means to make models more robust. A common way to assess robustness is to empirically compute the model performance on the adversarial examples that an attack produced from a set of original examples.

Similarly, the established way to harden ML models is adversarial hardening, i.e. training processes that make models learn to make correct predictions on adversarial examples.

Traditional adversarial attacks were designed for image recognition and assume that every image pixel can be modified independently to its full range of values. In many domains, however, these attacks fail to consider that only specific perturbations could occur in practice due to the hard domain constraints that delimit the set of valid inputs (e.g., financial transactions must have a positive amount, text must be linguistically consistent, medical images can change depending on the machine used and patients’ morphology, etc.).

Because of this, they almost-always produce examples that are not feasible (i.e. could not exist in the real world). 

As a result, research has developed real-world adversarial attacks that either manipulate real objects through a series of problem-space transformations (i.e. problem-space attacks) or generate feature perturbations that satisfy predefined domain constraints (i.e. constrained feature space attacks).

In this lecture, we will review the scientific literature on these attacks and report on our experience in applying them to real-world cases.