Introducing a new way to review your PRs.

Read the announcement

lockfale / osint-framework

OSINT Framework

View on GitHub

8 documents · Updated Nov 16, 2026

overview

Knowledge Base updated: 2026-11-16

Knowledge Base Index

1. Orientation

This repository is the OSINT Framework static/Worker-backed site. The highest-level flow is: Cloudflare Worker serves the public app, the browser loads public/arf.json, and the UI renders the OSINT resource catalog for exploration.

Main parts:

  • Cloudflare Worker runtime — request entrypoint and deployment/runtime configuration.
  • Public explorer UI — HTML, JavaScript, D3 rendering, and CSS for the interactive catalog.
  • OSINT catalog data — arf.json plus enrichment artifacts that describe resources and metadata.
  • Maintenance tools — Python and Node utilities for extracting, checking, rechecking, pruning, and enriching catalog links.
  • Project tooling and CI — package metadata, lockfiles, ignore rules, and GitHub Actions validation.

Critical state lives in files rather than an external database:

2. Architecture diagrams

flowchart TD
  subgraph Runtime
    Worker[src/worker.js]
    Wrangler[wrangler.jsonc]
  end

  subgraph PublicSite[public/]
    HTML[index.html]
    UI[public/js/arf.js]
    D3[public/js/d3.min.js]
    CSS[public/css/*.css]
    Catalog[public/arf.json]
  end

  subgraph Maintenance
    Merge[merge_enrichments.py]
    PyTools[tools/*.py]
    Recheck[tools/link-recheck/recheck.mjs]
    Enrichment[enrichment artifacts]
  end

  subgraph Automation
    RootPkg[package.json]
    CI[.github/workflows/ci.yml]
  end

  Wrangler --> Worker
  Worker --> HTML
  Worker --> UI
  Worker --> CSS
  UI --> D3
  UI --> Catalog
  Merge --> Catalog
  Enrichment --> Merge
  PyTools --> Catalog
  Recheck --> Catalog
  RootPkg --> CI
  CI --> PublicSite
  CI --> Runtime
sequenceDiagram
  participant Browser
  participant Worker as Cloudflare Worker
  participant Static as public assets
  participant Catalog as public/arf.json

  Browser->>Worker: Request site URL
  Worker->>Static: Resolve index/assets
  Static-->>Worker: HTML, JS, CSS
  Worker-->>Browser: Public explorer app
  Browser->>Worker: Request arf.json
  Worker->>Catalog: Serve catalog data
  Catalog-->>Browser: Resource graph JSON
  Browser->>Browser: Render and interact with graph
flowchart LR
  Batch[enrichment batch files] --> Merge[merge_enrichments.py]
  Catalog[public/arf.json] --> Merge
  Merge --> UpdatedCatalog[updated public/arf.json]
  UpdatedCatalog --> UI[public/js/arf.js]
  UpdatedCatalog --> LinkTools[tools link checkers]

3. Glossary

4. Codebase-wide invariants

  • The primary catalog is a checked-in JSON file, not an external service; both the UI and maintenance tools route through public/arf.json.
  • Runtime deployment is Cloudflare Worker-oriented; the Worker entrypoint is src/worker.js and its deployment configuration is wrangler.jsonc.
  • Link maintenance is split across Python utilities under tools/*.py and a separate Node workflow under tools/link-recheck/**, each with its own dependency context where applicable.
  • Browser presentation assets live under public/ with a simple split between public/index.html, public/js/**, public/css/**, and public/arf.json.

5. Routing table