CleverHans

Overview

CleverHans was one of the first adversarial ML libraries to gain wide adoption, originally maintained by researchers at Google Brain. It provides faithful reference implementations of foundational attacks used throughout the adversarial ML literature, which makes it a reliable baseline when you need to reproduce a paper’s numbers or benchmark a defense against well-known attack implementations rather than a reimplementation.

Installation

Install from PyPI (PyTorch and TF2 functional APIs both included)

pip install cleverhans

Fast Gradient Sign Method

from cleverhans.torch.attacks.fast_gradient_method import fast_gradient_method

# Single-step gradient attack bounded by L-inf epsilon
x_adv = fast_gradient_method(model_fn, x, eps=0.03, norm=float("inf"))

Projected Gradient Descent

from cleverhans.torch.attacks.projected_gradient_descent import projected_gradient_descent

# Multi-step iterative attack, generally the strongest first-order baseline
x_adv = projected_gradient_descent(
    model_fn, x, eps=0.03, eps_iter=0.005, nb_iter=40, norm=float("inf"),
)

Carlini & Wagner L2

from cleverhans.torch.attacks.carlini_wagner_l2 import carlini_wagner_l2

# Optimization-based attack, minimizes perturbation magnitude
x_adv = carlini_wagner_l2(model_fn, x, n_classes=10, targeted=False)

Evaluating a Model

import torch

# Compare clean vs adversarial accuracy to quantify robustness
preds_clean = model_fn(x).argmax(1)
preds_adv = model_fn(x_adv).argmax(1)
acc_adv = (preds_adv == y).float().mean().item()
print(f"Adversarial accuracy: {acc_adv:.2%}")

Tips

• model_fn just needs to be any callable returning logits — wrap a HF or torchvision model directly, no special class required.
• For TensorFlow 2 workflows the equivalent functions live under cleverhans.tf2.attacks.

cleverhans.torch.attacks.fast_gradient_method(model_fn, x, eps, norm)
cleverhans.torch.attacks.projected_gradient_descent(model_fn, x, eps, eps_iter, nb_iter, norm)
cleverhans.torch.attacks.carlini_wagner_l2(model_fn, x, n_classes, targeted=False)
cleverhans.torch.attacks.spsa(model_fn, x, eps, nb_iter)          # black-box, gradient-free
cleverhans.torch.attacks.hop_skip_jump_attack(model_fn, x, norm)  # decision-based black-box

cleverhans.tf2.attacks.*   # TensorFlow 2 equivalents of the above

Vue d’ensemble

CleverHans a été l’une des premières bibliothèques de ML adverse à être largement adoptée, initialement maintenue par des chercheurs de Google Brain. Elle fournit des implémentations de référence fidèles des attaques fondamentales utilisées dans toute la littérature du ML adverse, ce qui en fait une baseline fiable quand vous devez reproduire les chiffres d’un papier ou évaluer une défense contre des implémentations d’attaques bien connues plutôt qu’une réimplémentation.

Installation

# Installer depuis PyPI (les APIs fonctionnelles PyTorch et TF2 sont toutes deux incluses)
pip install cleverhans

Fast Gradient Sign Method

from cleverhans.torch.attacks.fast_gradient_method import fast_gradient_method

# Attaque par gradient en une seule étape, bornée par un epsilon L-inf
x_adv = fast_gradient_method(model_fn, x, eps=0.03, norm=float("inf"))

Projected Gradient Descent

from cleverhans.torch.attacks.projected_gradient_descent import projected_gradient_descent

# Attaque itérative multi-étapes, généralement la meilleure baseline du premier ordre
x_adv = projected_gradient_descent(
    model_fn, x, eps=0.03, eps_iter=0.005, nb_iter=40, norm=float("inf"),
)

Carlini & Wagner L2

from cleverhans.torch.attacks.carlini_wagner_l2 import carlini_wagner_l2

# Attaque basée sur l'optimisation, minimise l'amplitude de la perturbation
x_adv = carlini_wagner_l2(model_fn, x, n_classes=10, targeted=False)

Évaluer un modèle

import torch

# Comparer la précision propre et adverse pour quantifier la robustesse
preds_clean = model_fn(x).argmax(1)
preds_adv = model_fn(x_adv).argmax(1)
acc_adv = (preds_adv == y).float().mean().item()
print(f"Adversarial accuracy: {acc_adv:.2%}")

Conseils

• model_fn doit juste être un callable renvoyant des logits : enveloppez directement un modèle HF ou torchvision, aucune classe spéciale requise.
• Pour les workflows TensorFlow 2, les fonctions équivalentes se trouvent sous cleverhans.tf2.attacks.

cleverhans.torch.attacks.fast_gradient_method(model_fn, x, eps, norm)
cleverhans.torch.attacks.projected_gradient_descent(model_fn, x, eps, eps_iter, nb_iter, norm)
cleverhans.torch.attacks.carlini_wagner_l2(model_fn, x, n_classes, targeted=False)
cleverhans.torch.attacks.spsa(model_fn, x, eps, nb_iter)          # black-box, gradient-free
cleverhans.torch.attacks.hop_skip_jump_attack(model_fn, x, norm)  # decision-based black-box

cleverhans.tf2.attacks.*   # TensorFlow 2 equivalents of the above