Node link recheck flow
Overview
The Node-based recheck flow is the second stage of the scheduled/manual link-check job for catalog URLs. The CI workflow in .github/…/ci.yml first runs tools/extract_urls.py, which reads public/arf.json, skips placeholder/template URLs, and writes urls.txt. Lychee then checks that URL list and writes its JSON output to lychee/out.json without failing the job. The owned Node entrypoint, tools/…/recheck.mjs, consumes that lychee JSON and decides whether any reported errors still fail when loaded in a real browser.
tools/…/package.json defines a private ESM package with one runtime dependency, playwright@1.63.0; tools/…/package-lock.json pins both playwright and playwright-core and records their Node engine requirement (>=20). The CI job uses actions/setup-node with Node 22, runs npm ci in tools/link-recheck, then installs Chromium with npx playwright install --with-deps chromium before invoking node recheck.mjs ../../lychee/out.json ../../link-recheck-report.md.
tools/…/recheck.mjs accepts node recheck.mjs <lychee-json> [report.md]. If the output path is omitted it writes link-recheck-report.md relative to the current working directory. lycheeFailures() reads error_map, fail_map, and timeout_map from the lychee report so the script works with both newer and older lychee JSON shapes; each map is expected to contain input-file keys whose values are result arrays with url and status fields. The script deduplicates by URL in insertion order and preserves lychee's status text as the comparison reason shown in the final report.
For each URL, recheck.mjs launches Playwright Chromium and creates one browser context with a non-headless-looking user agent (HeadlessChrome is replaced with Chrome) and en-US locale. Four workers share the context (CONCURRENCY = 4). Each URL is opened with page.goto(..., waitUntil: "domcontentloaded", timeout: 45_000), while a response listener tracks the last main-frame navigation response. After the first navigation, the script waits up to 10 seconds for JavaScript challenge pages to redirect to a real page; it considers the page settled when the main document is a non-202 status below 400, or a terminal 404/410.
Validation is intentionally browser-oriented rather than equivalent to lychee's HTTP probe. classify() returns ok when the browser gets a status below 400, failing for navigation errors, 404/410, and 5xx statuses, and blocked for 401/403/429 or challenge-page content containing markers such as just a moment, checking your browser, verify you are human, cf-chl, or captcha. Other 4xx responses are treated as blocked, not dead, because the goal is to avoid failing CI for sites that are up but reject automated checks. Network errors, DNS failures, certificate failures, refused/reset connections, timeouts, and 5xx responses remain failing and cause process exit code 1.
The output is a Markdown report from renderReport(). It starts with a summary table for Still failing, Up, but blocks automated browsers, and Loaded fine, then emits URL tables per non-empty group with the browser result and original lychee reason. The report is written to the requested path and appended to $GITHUB_STEP_SUMMARY when that environment variable is present. The workflow uploads both lychee/out.json and link-recheck-report.md as the link-check-report artifact; only the Node recheck's count of still-failing URLs determines whether the CI link-check job fails.
This flow complements, rather than replaces, the Python maintenance tools. tools/extract_urls.py is the CI source of truth for which catalog URLs from public/arf.json are checked by lychee, while tools/link_checker.py is a separate urllib-based audit tool that extracts URLs directly from public/arf.json and writes tools/link_audit_report.json. The Node flow does not read public/arf.json directly; its input is the lychee JSON generated from urls.txt.
Coupling
- If tools/extract_urls.py changes which public/arf.json URLs are written to urls.txt, verify .github/…/ci.yml and tools/…/recheck.mjs still agree on the artifact flow: lychee must continue to produce lychee/out.json, and the Node script only rechecks URLs present in lychee's failure/timeout maps.
- If the lychee action version or output format in .github/…/ci.yml changes, verify tools/…/recheck.mjs still reads the correct JSON maps. It currently only scans
error_map,fail_map, andtimeout_map, and silently ignores any new failure container name. - If tools/…/package.json or tools/…/package-lock.json changes the Playwright version, verify the CI Node version and browser install command in .github/…/ci.yml; the lockfile currently pins Playwright 1.63.0, whose packages require Node
>=20, while CI uses Node 22. - If
classify()in tools/…/recheck.mjs changes blocked/failing status handling, verify the lychee--acceptstatuses in .github/…/ci.yml. CI accepts 403 and 429 during the first pass, but the browser recheck also treats 401/403/429 and recognizable challenge pages as blocked rather than dead.
Risks
- tools/…/recheck.mjs assumes every result in lychee's failure maps has a stable
urlfield. If lychee emits a different result shape,lycheeFailures()can deduplicate everything underundefinedor omit failures, causing the link-check job to pass without rechecking the intended URLs. - tools/…/recheck.mjs shares one Playwright browser context across four concurrent workers. If a checked site mutates shared context state such as cookies, locale-dependent challenges, or per-domain rate limits, another URL checked in the same run may be classified differently than it would in a fresh browser context.
Routing and ownership
- Owns
tools/link-recheck/**: the Node package metadata, lockfile, and recheck.mjs browser recheck entrypoint. - Related but not owned: .github/…/ci.yml orchestrates this flow; tools/extract_urls.py prepares lychee input from public/arf.json; tools/link_checker.py is a separate Python audit path over the same catalog data.