Introducing a new way to review your PRs.

Read the announcement

lockfale / osint-framework

OSINT Framework

View on GitHub

8 documents · Updated Nov 16, 2026

docs / project-tooling-and-ci

Project Tooling and CI

Overview

This repository is a small Node-managed static/Worker project whose root package metadata lives in package.json and is locked by package-lock.json. The package is named arf, has public/index.html as its main entry, and uses only dev dependencies: copyfiles, d3, and Cloudflare wrangler.

package.json scripts support three flows:

  • Static-site local serving: npm start changes into public/ and runs python3 -m http.server 8000, so it serves the checked-out static site directly rather than starting a Node app.
  • Static asset preparation: postinstall runs copyfiles -f ./node_modules/d3/dist/d3.min.js ./public/js/, copying the installed D3 browser bundle into the static asset tree after npm install/npm ci.
  • Worker/deployment operations: npm run preview runs wrangler dev, and npm run deploy runs wrangler deploy. The lockfile pins wrangler to 4.149.0; that version declares a Node >=22.0.0 engine and pulls Cloudflare runtime tooling such as workerd, miniflare, and platform-specific @cloudflare/workerd-* packages.

.github/…/ci.yml defines the repository validation automation. It runs on pushes and pull requests targeting master, on a weekly Saturday schedule at 08:00 UTC, and through manual workflow_dispatch. The push/PR path runs two checkout-only validation jobs:

The scheduled/manual data-maintenance path is the link-check job. It is explicitly gated with if: github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' so external links are not hammered on every push or PR. That job checks out the repo, runs python3 tools/extract_urls.py to produce urls.txt, sends those URLs through lycheeverse/lychee-action@v2, then installs Node 22 plus Playwright/Chromium under tools/link-recheck and runs node recheck.mjs ../../lychee/out.json ../../link-recheck-report.md. It always uploads lychee/out.json and link-recheck-report.md as the link-check-report artifact.

.gitignore keeps local/generated tooling state out of commits: agent directories (agents/, .agents/), dependency installs (node_modules, bower_components, public/js/vendor), the generated tools/link_audit_report.json, OS cruft, and Cloudflare/local-secret files (.wrangler, .dev.vars*, .env*) while allowing example env files.

Routing and ownership

This doc owns repository-level tooling and automation in these paths:

  • package.json — package scripts, package identity, and declared Node tooling dependencies.
  • package-lock.json — pinned dependency graph for copyfiles, d3, wrangler, and their transitive tooling.
  • .github/…/ci.yml — GitHub Actions validation, scheduled link checking, and report artifact upload.
  • .gitignore — ignored dependency folders, generated reports, local Wrangler state, and local environment files.

Coupling

  • If package.json changes the D3 dependency or the postinstall copy destination, also verify .github/…/ci.yml's smoke-test asset check for public/…/d3.min.js; the smoke job does not install dependencies before checking assets, so it only sees files present after checkout.
  • If wrangler is upgraded in package.json or package-lock.json, verify the Node version used by any CI job that invokes Cloudflare tooling. The current lockfile's node_modules/wrangler entry requires Node >=22.0.0, while .github/…/ci.yml only sets up Node 22 inside the scheduled/manual link-check job.
  • If the URL extraction or browser recheck tooling paths change, update .github/…/ci.yml together with those moves: the link-check job hard-codes tools/extract_urls.py, uses tools/link-recheck as its working directory, and expects lychee/out.json plus link-recheck-report.md for artifact upload.
  • If local secret conventions change, keep .gitignore aligned with Wrangler and environment-file usage. It ignores .wrangler, .dev.vars*, and .env* but intentionally allows .dev.vars.example and .env.example to remain tracked as safe templates.

Risks

  • .github/…/ci.yml can miss install-time asset generation problems on push/PR: smoke-test checks public/…/d3.min.js after checkout without running npm ci, while package.json generates that file via postinstall. If the checked-in asset is removed or the postinstall copy path changes, CI fails at the asset-presence step rather than exercising the install flow that would regenerate it.
  • package-lock.json pins wrangler@4.149.0 with a Node >=22.0.0 engine. Running npm run preview or npm run deploy from package.json on older local or CI Node versions will fail before Cloudflare Worker preview/deploy behavior is reached.