SpiderFoot: Open source OSINT automation tool
Python reconnaissance framework with 200+ OSINT modules.
Learn more about SpiderFoot
SpiderFoot is an OSINT automation framework written in Python 3 that aggregates data from over 200 modules to perform reconnaissance on various entity types including IP addresses, domains, email addresses, and usernames. The tool uses a publisher/subscriber model where modules feed data to each other, enabling multi-stage reconnaissance workflows such as subdomain enumeration, threat intelligence lookups, and breach database searches. It includes a SQLite backend for data storage, a YAML-configurable correlation engine with predefined rules, and support for exporting results in CSV, JSON, and GEXF formats. SpiderFoot can be deployed via web interface, command-line, or Docker, and supports integration with external tools like Nmap, DNSTwist, and Whatweb.
Modular Pipeline Architecture
Over 200 modules operate in a publisher/subscriber pattern where output from one module automatically feeds into others. Enables complex multi-stage reconnaissance workflows like subdomain enumeration followed by threat intelligence lookups without manual data passing.
Multiple Deployment Options
Ships with embedded web UI, full-featured CLI, and Docker support in a single package. Run scripted operations headless or use the graphical interface for interactive investigations without switching tools.
YAML Correlation Engine
Includes 37 predefined rules that automatically identify relationships and patterns across collected data. Query results directly from the SQLite backend for custom analysis beyond the built-in correlation logic.
from spiderfoot import SpiderFootScanner
# Initialize scanner and start a basic scan
scanner = SpiderFootScanner()
scan_id = scanner.startScan(
name="Target Domain Scan",
target="example.com",
modules=["sfp_dnsresolve", "sfp_whois", "sfp_emailformat"]
)
print(f"Scan started with ID: {scan_id}")Related Repositories
Discover similar tools and frameworks used by developers
Subfinder
Passive subdomain discovery via DNS resolution and validation.
Sigma
YAML-based detection rules with SIEM query conversion.
Semgrep
Pattern-based code scanner for 30+ languages.
Gitleaks
Regex-based secret scanner for git repositories.
DOMPurify
DOM-based XSS sanitizer using native browser parsing.