waybackpy

Overview

waybackpy provides a Python interface and CLI for querying the Wayback Machine (archive.org). For security research, it’s used to discover historical URLs (finding forgotten endpoints), retrieve old versions of pages (to find removed sensitive data), and extract URLs from archive CDX records.

Installation

pip3 install waybackpy

Basic Usage

Get newest archived snapshot URL

waybackpy --url "https://example.com" --newest

Get oldest archived snapshot URL

waybackpy --url "https://example.com" --oldest

Save snapshot

waybackpy --url "https://example.com" --save

Get all archived URLs (CDX)

waybackpy --url "https://example.com" --known-urls

Get URLs for entire domain

waybackpy --url "https://example.com/*" --known-urls

URL Discovery (Main Security Use Case)

Get all historical URLs for a domain

waybackpy --url "https://example.com/*" --known-urls

# Get URLs with specific extension
waybackpy --url "https://example.com/*.php" --known-urls
waybackpy --url "https://example.com/*.js" --known-urls
waybackpy --url "https://example.com/*.env" --known-urls
waybackpy --url "https://example.com/*.bak" --known-urls

Filter for API endpoints

waybackpy --url "https://example.com/api/*" --known-urls

Find admin paths

waybackpy --url "https://example.com/admin*" --known-urls

Alternative: waybackurls

waybackurls is a faster Go alternative for URL extraction

go install github.com/tomnomnom/waybackurls@latest

Get all archived URLs

waybackurls example.com > urls.txt

# Filter by extension
waybackurls example.com | grep "\.php"
waybackurls example.com | grep "admin"
waybackurls example.com | grep "api"

Find parameters (for fuzzing)

waybackurls example.com | grep "?" | sort -u

Python API

import waybackpy

# Get saved URLs
url = "https://example.com"
user_agent = "Mozilla/5.0"

api = waybackpy.Url(url, user_agent)

# Oldest version
oldest = api.oldest()
print(oldest.archive_url)

# Newest version  
newest = api.newest()
print(newest.archive_url)

# Near specific date
near = api.near(year=2020, month=6, day=1)
print(near.archive_url)

OSINT Workflows

Find forgotten/deleted sensitive files

waybackurls example.com | grep -E "\.(sql|db|bak|backup|zip|tar|gz)$"

Find old API endpoints

waybackurls example.com | grep "/api/" | sort -u

Find parameters for fuzzing

waybackurls example.com | \
  grep "?" | \
  sort -u | \
  cut -d "?" -f 2 | \
  tr "&" "\n" | \
  cut -d "=" -f 1 | \
  sort -u

Find exposed .git, .env files

waybackurls example.com | grep -E "\.git|\.env|config\."

Find old admin panels

waybackurls example.com | grep -iE "admin|dashboard|control"

Tips

• Wayback Machine often has snapshots of pages/files that were deleted years ago
• Combine with gau (Get All URLs) for broader historical URL coverage
• Old JavaScript files often contain API keys, internal endpoints, and deprecated auth
• Check wayback snapshots of login pages for old credential hints in source code

waybackpy [options]

--url URL          Target URL
--user-agent UA    User agent string
--newest           Get newest snapshot
--oldest           Get oldest snapshot
--near ARGS        Get snapshot near date (--year --month --day --hour)
--save             Save current page to Wayback
--json             JSON output
--known-urls       List all archived URLs for this URL pattern
--subdomain        Include subdomains in --known-urls

CLI:
waybackpy --url https://example.com --oldest
waybackpy --url "https://example.com/*" --known-urls

Vue d’ensemble

waybackpy fournit une interface Python et une CLI pour interroger la Wayback Machine (archive.org). Pour la recherche en sécurité, il sert à découvrir des URL historiques (retrouver des endpoints oubliés), récupérer d’anciennes versions de pages (pour retrouver des données sensibles supprimées), et extraire des URL à partir des enregistrements CDX de l’archive.

Installation

pip3 install waybackpy

Utilisation de base

# Récupérer l'URL du snapshot archivé le plus récent
waybackpy --url "https://example.com" --newest

# Récupérer l'URL du snapshot archivé le plus ancien
waybackpy --url "https://example.com" --oldest

# Sauvegarder un snapshot
waybackpy --url "https://example.com" --save

# Récupérer toutes les URL archivées (CDX)
waybackpy --url "https://example.com" --known-urls

# Récupérer les URL pour un domaine entier
waybackpy --url "https://example.com/*" --known-urls

Découverte d’URL (principal cas d’usage sécurité)

# Récupérer toutes les URL historiques d'un domaine
waybackpy --url "https://example.com/*" --known-urls

# Récupérer les URL avec une extension spécifique
waybackpy --url "https://example.com/*.php" --known-urls
waybackpy --url "https://example.com/*.js" --known-urls
waybackpy --url "https://example.com/*.env" --known-urls
waybackpy --url "https://example.com/*.bak" --known-urls

# Filtrer les endpoints d'API
waybackpy --url "https://example.com/api/*" --known-urls

# Trouver les chemins d'administration
waybackpy --url "https://example.com/admin*" --known-urls

Alternative : waybackurls

waybackurls est une alternative en Go, plus rapide, pour l’extraction d’URL

go install github.com/tomnomnom/waybackurls@latest

# Récupérer toutes les URL archivées
waybackurls example.com > urls.txt

# Filtrer par extension
waybackurls example.com | grep "\.php"
waybackurls example.com | grep "admin"
waybackurls example.com | grep "api"

# Trouver les paramètres (pour le fuzzing)
waybackurls example.com | grep "?" | sort -u

API Python

import waybackpy

# Récupérer les URL sauvegardées
url = "https://example.com"
user_agent = "Mozilla/5.0"

api = waybackpy.Url(url, user_agent)

# Version la plus ancienne
oldest = api.oldest()
print(oldest.archive_url)

# Version la plus récente
newest = api.newest()
print(newest.archive_url)

# Proche d'une date spécifique
near = api.near(year=2020, month=6, day=1)
print(near.archive_url)

Flux de travail OSINT

# Trouver des fichiers sensibles oubliés/supprimés
waybackurls example.com | grep -E "\.(sql|db|bak|backup|zip|tar|gz)$"

# Trouver d'anciens endpoints d'API
waybackurls example.com | grep "/api/" | sort -u

# Trouver des paramètres pour le fuzzing
waybackurls example.com | \
  grep "?" | \
  sort -u | \
  cut -d "?" -f 2 | \
  tr "&" "\n" | \
  cut -d "=" -f 1 | \
  sort -u

# Trouver des fichiers .git, .env exposés
waybackurls example.com | grep -E "\.git|\.env|config\."

# Trouver d'anciens panneaux d'administration
waybackurls example.com | grep -iE "admin|dashboard|control"

Conseils

• La Wayback Machine possède souvent des snapshots de pages/fichiers supprimés il y a des années
• Combiner avec gau (Get All URLs) pour une couverture historique plus large
• Les anciens fichiers JavaScript contiennent souvent des clés API, des endpoints internes et des mécanismes d’authentification obsolètes
• Vérifier les snapshots archivés des pages de connexion pour d’anciens indices d’identifiants dans le code source

waybackpy [options]

--url URL          Target URL
--user-agent UA    User agent string
--newest           Get newest snapshot
--oldest           Get oldest snapshot
--near ARGS        Get snapshot near date (--year --month --day --hour)
--save             Save current page to Wayback
--json             JSON output
--known-urls       List all archived URLs for this URL pattern
--subdomain        Include subdomains in --known-urls

CLI:
waybackpy --url https://example.com --oldest
waybackpy --url "https://example.com/*" --known-urls