← retour aux outils back to tools

# GPTFuzz

Framework de fuzzing de jailbreak en boîte noire qui mute des templates de jailbreak écrits par des humains via un mutateur LLM et une sélection de graines basée sur MCTS, jugée par un modèle de scoring entraîné. Black-box jailbreak fuzzing framework that mutates human-written jailbreak templates via an LLM mutator and MCTS-based seed selection, judged by a trained scoring model.

↗ https://github.com/sherdencooper/GPTFuzz

## Overview

GPTFuzz (GPTFUZZER) applies AFL-style mutational fuzzing to LLM jailbreaking. Instead of hand-crafting prompts, it starts from a seed pool of known jailbreak templates, uses an LLM (e.g. GPT-3.5) to mutate them (rephrase, expand, crossover, shorten), and picks which seed to mutate next with an MCTS-Explore strategy that balances exploiting promising seeds against exploring new ones. A fine-tuned RoBERTa judgment model scores whether each generated prompt actually jailbroke the target, closing the loop without needing a human in every iteration. The original paper reports 90%+ attack success against ChatGPT and Llama-2 with this approach.

## Installation

    git clone https://github.com/sherdencooper/GPTFuzz.git
    cd GPTFuzz
    pip install -r requirements.txt

Download / point to the pretrained RoBERTa judgment model

    python download_judge_model.py

## Running a Fuzzing Campaign

Fuzz against a target model using the bundled seed templates

    python fuzz.py \
      --seed_path datasets/prompts/jailbreak_templates.csv \
      --question_path datasets/questions/harmful_questions.csv \
      --target_model gpt-3.5-turbo \
      --mutate_model gpt-3.5-turbo \
      --judge_model roberta \
      --max_query 1000 \
      --max_jailbreak 5 \
      --energy 1 \
      --output_file results/run1.csv

## Custom Target / Local Model

    from gptfuzzer.fuzzer import GPTFuzzer
    from gptfuzzer.fuzzer.mutator import MutateRandomSinglePolicy
    from gptfuzzer.llm import LocalLLM, OpenAILLM

    target = LocalLLM(model_path="/models/Llama-2-7b-chat-hf")
    mutate_model = OpenAILLM(model_path="gpt-3.5-turbo", api_key="<api_key>")

    fuzzer = GPTFuzzer(
        questions=["<harmful_question>"],
        target=target,
        predictor=judge_model,          # loaded RoBERTa classifier
        initial_seed=seed_templates,    # list of jailbreak template strings
        mutate_policy=MutateRandomSinglePolicy([...], mutate_model),
        energy=1,
        max_query=1000,
    )
    fuzzer.run()

## Tips

- The MCTS-Explore seed selector is the key differentiator over plain random mutation — it converges on effective template lineages faster, so prefer it over `--select_policy random` for longer campaigns.
- Swap in your own judge (regex-based refusal detector, another LLM-as-judge) via `--judge_model` if the bundled RoBERTa model doesn’t fit your target’s response style.
- Track `--max_query` carefully against hosted targets — each fuzzing iteration burns API calls on both the mutator and the target.

Help / Man page

    usage: fuzz.py [-h] --seed_path PATH --question_path PATH
                    --target_model MODEL --mutate_model MODEL
                    [--judge_model {roberta,gpt4,regex}]
                    [--max_query N] [--max_jailbreak N]
                    [--energy N] [--select_policy {mcts,random,round_robin}]
                    [--output_file FILE]

    required arguments:
      --seed_path PATH        CSV of seed jailbreak templates
      --question_path PATH    CSV of harmful questions to test
      --target_model MODEL    model under test (API name or local path)
      --mutate_model MODEL    LLM used to mutate seeds

    optional arguments:
      --judge_model NAME      response classifier (default: roberta)
      --max_query N           query budget for the campaign
      --max_jailbreak N       stop early after N successful jailbreaks
      --energy N              mutants generated per selected seed per round
      --select_policy NAME    seed selection strategy (default: mcts)
      --output_file FILE      CSV of prompts, responses, and judge scores

## Vue d’ensemble

GPTFuzz (GPTFUZZER) applique un fuzzing mutationnel de style AFL au jailbreaking de LLM. Au lieu de rédiger des prompts à la main, il part d’un pool de graines constitué de templates de jailbreak connus, utilise un LLM (par exemple GPT-3.5) pour les muter (reformulation, expansion, crossover, raccourcissement), et choisit quelle graine muter ensuite avec une stratégie MCTS-Explore qui équilibre l’exploitation des graines prometteuses et l’exploration de nouvelles graines. Un modèle de jugement RoBERTa affiné évalue si chaque prompt généré a effectivement jailbreaké la cible, bouclant le processus sans nécessiter d’humain à chaque itération. L’article original rapporte un taux de succès d’attaque supérieur à 90 % contre ChatGPT et Llama-2 avec cette approche.

## Installation

    git clone https://github.com/sherdencooper/GPTFuzz.git
    cd GPTFuzz
    pip install -r requirements.txt

Télécharger / pointer vers le modèle de jugement RoBERTa préentraîné

    python download_judge_model.py

## Lancer une campagne de fuzzing

Fuzzer contre un modèle cible avec les templates de graines fournis

    python fuzz.py \
      --seed_path datasets/prompts/jailbreak_templates.csv \
      --question_path datasets/questions/harmful_questions.csv \
      --target_model gpt-3.5-turbo \
      --mutate_model gpt-3.5-turbo \
      --judge_model roberta \
      --max_query 1000 \
      --max_jailbreak 5 \
      --energy 1 \
      --output_file results/run1.csv

## Cible personnalisée / modèle local

    from gptfuzzer.fuzzer import GPTFuzzer
    from gptfuzzer.fuzzer.mutator import MutateRandomSinglePolicy
    from gptfuzzer.llm import LocalLLM, OpenAILLM

    target = LocalLLM(model_path="/models/Llama-2-7b-chat-hf")
    mutate_model = OpenAILLM(model_path="gpt-3.5-turbo", api_key="<api_key>")

    fuzzer = GPTFuzzer(
        questions=["<harmful_question>"],
        target=target,
        predictor=judge_model,          # loaded RoBERTa classifier
        initial_seed=seed_templates,    # list of jailbreak template strings
        mutate_policy=MutateRandomSinglePolicy([...], mutate_model),
        energy=1,
        max_query=1000,
    )
    fuzzer.run()

## Conseils

- Le sélecteur de graines MCTS-Explore est le principal élément différenciant par rapport à une simple mutation aléatoire : il converge plus vite vers des lignées de templates efficaces, donc préférez-le à `--select_policy random` pour les campagnes longues.
- Remplacez le juge fourni (détecteur de refus basé sur regex, un autre LLM-as-judge) via `--judge_model` si le modèle RoBERTa fourni ne correspond pas au style de réponse de votre cible.
- Surveillez `--max_query` de près contre des cibles hébergées : chaque itération de fuzzing consomme des appels API à la fois sur le mutateur et sur la cible.

Aide / Page de manuel

    usage: fuzz.py [-h] --seed_path PATH --question_path PATH
                    --target_model MODEL --mutate_model MODEL
                    [--judge_model {roberta,gpt4,regex}]
                    [--max_query N] [--max_jailbreak N]
                    [--energy N] [--select_policy {mcts,random,round_robin}]
                    [--output_file FILE]

    required arguments:
      --seed_path PATH        CSV of seed jailbreak templates
      --question_path PATH    CSV of harmful questions to test
      --target_model MODEL    model under test (API name or local path)
      --mutate_model MODEL    LLM used to mutate seeds

    optional arguments:
      --judge_model NAME      response classifier (default: roberta)
      --max_query N           query budget for the campaign
      --max_jailbreak N       stop early after N successful jailbreaks
      --energy N              mutants generated per selected seed per round
      --select_policy NAME    seed selection strategy (default: mcts)
      --output_file FILE      CSV of prompts, responses, and judge scores