Adversarial ML Defenses Advanced

This lesson covers the most effective defense strategies against adversarial attacks. While no single defense is perfect, combining multiple approaches creates layered protection. We cover adversarial training (the gold standard), defensive distillation, input transformation, ensemble methods, and adversarial detection using the Adversarial Robustness Toolbox (ART).

Adversarial Training

Adversarial training is currently the most effective empirical defense. It augments the training process by including adversarial examples, teaching the model to be robust against perturbations:

def adversarial_training_step(model, images, labels,
                                optimizer, epsilon, alpha, pgd_steps):
    """One step of PGD adversarial training."""
    model.train()

    # Generate adversarial examples using PGD
    adv_images = pgd_attack(model, images, labels,
                            epsilon, alpha, pgd_steps)

    # Train on adversarial examples
    optimizer.zero_grad()
    outputs = model(adv_images)
    loss = F.cross_entropy(outputs, labels)
    loss.backward()
    optimizer.step()

    return loss.item()

# Training loop
for epoch in range(num_epochs):
    for images, labels in train_loader:
        loss = adversarial_training_step(
            model, images, labels, optimizer,
            epsilon=8/255, alpha=2/255, pgd_steps=7
        )

Input Transformation Defenses

Preprocess inputs to destroy adversarial perturbations before they reach the model:

Defensive Distillation

Train a second "distilled" model using the soft probability outputs of the original model as training targets. The distilled model has smoother decision surfaces that are harder to attack with gradient-based methods. However, Carlini and Wagner showed that the C&W attack can defeat defensive distillation.

Ensemble Methods

• Diverse model ensemble - Use models with different architectures; adversarial examples rarely transfer to all models
• Ensemble adversarial training - Train each model on adversarial examples generated from all other models
• Prediction agreement - Flag inputs where ensemble members disagree as potentially adversarial

Using ART for Defenses

from art.defences.preprocessor import (
    JpegCompression, SpatialSmoothing, FeatureSqueezing
)
from art.defences.trainer import AdversarialTrainerMadryPGD

# Preprocessing defenses
jpeg_defense = JpegCompression(clip_values=(0, 1), quality=75)
smoothing = SpatialSmoothing(window_size=3)
squeezing = FeatureSqueezing(clip_values=(0, 1), bit_depth=4)

# Apply preprocessing to classifier
classifier.set_preprocessor(jpeg_defense)

# Adversarial training with ART
trainer = AdversarialTrainerMadryPGD(
    classifier=classifier,
    nb_epochs=50,
    eps=0.03,
    eps_step=0.007,
    max_iter=7
)
trainer.fit(x_train, y_train)