Operations and deployment
World Monitor uses separate delivery paths for the web app, API, data workers, edge services, durable state, desktop clients, SDKs, CLI packages, and local containers. Build checks, release workflows, live probes, and durable failure reports work together so a change is checked before merge and its running target is checked after release.
Operational model
The system separates user traffic from work that collects and refreshes data. Each target has its own release boundary and its own evidence that it is working.
| Area | Purpose | Main operating boundary |
|---|---|---|
| Web and API | Serve the dashboard, public pages, API routes, and machine interfaces | Vercel or a full web container |
| Data workers | Collect source data and publish fresh values outside a user request | Railway native deployments, scheduled tasks, or long-running hosted services |
| Edge services | Apply request rules before traffic reaches an origin | Cloudflare Workers and the hosting edge |
| Durable state | Store application and user state | Convex and hosted data services |
| Desktop delivery | Build and publish one release line for supported systems | Tauri and GitHub Releases |
| SDK delivery | Publish client packages for language-specific consumers | Go module proxy, PyPI, RubyGems, crates.io, and GitHub Actions |
| CLI delivery | Test and publish the command-line package | npm and GitHub Actions OIDC |
| Local operation | Run the app, cache, and relay as a self-contained stack | Docker or Podman Compose |
Independent paths limit the blast radius of a failure. A worker can stop producing fresh data while the web app stays available. Health views can then report stale or missing data without declaring the whole site down.
Issue intake
The bug report form gives operations and development teams a consistent view of user-reported failures. It records the product variant and affected area before it collects reproduction details, so a report can reach the right service boundary.
The issue evidence workflow adds a small guard after an issue is opened or edited. When an outside report claims a UI problem without a usable image, it asks for evidence and removes the request after an attachment appears. It does not close or resolve the issue.
References: .github/…/bug_report.yml• L1-88 .github/…/issue-ui-evidence.yml• L1-44
Delivery lifecycle
Every delivery path turns a source change or release event into a target artifact. A separate check then confirms that the target can serve or publish its expected contract.
flowchart LR
Change[Change or release] --> Verify[Build and verification]
Verify --> Web[Vercel web and API]
Verify --> Worker[Hosted data worker]
Verify --> Edge[Edge service]
Verify --> State[Durable backend]
Verify --> Image[Container image]
Change --> Desktop[Desktop release]
Change --> SDK[SDK package]
Change --> CLI[CLI package]
Web --> Observe[Live and freshness checks]
Worker --> Observe
Edge --> Observe
State --> Observe
Image --> Operator[Self-hosted operator]
Desktop --> Client[Installed clients]
SDK --> Consumer[Package consumers]
CLI --> Consumer
Web delivery
The hosted web target builds the public product from the root build contract. Its route rules select dashboard variants and public pages. They also connect API and agent interfaces to their handlers and define redirects, cache rules, security headers, and cross-origin behavior that external clients rely on.
Vercel now checks the crawlable pulse and GitHub star snapshot before it runs the full build. These checks keep pages that show live or changing facts from publishing an old snapshot. Its change check can skip a build only when the comparison proves that the web target is unaffected. If the comparison base is missing or unsafe, it builds instead of skipping. The public open-data chokepoint directory paths resolve to their README, while named data files continue through the API handler. The same public boundary can run in a container with nginx in front of a Node API process.
References: vercel.json• L1-4 vercel.json• L5-40 vercel.json• L149-193 vercel.json• L195-225 scripts/vercel-ignore.sh• L1-104 Dockerfile• L23-61
Container delivery
The full Dockerfile produces a portable web and API runtime. The image builds the frontend and API output, keeps a smaller runtime dependency set for raw handlers, serves static files through nginx, and runs the API sidecar behind the same boundary. Its health check reaches the sidecar through nginx, so a healthy result covers both parts of the runtime.
The image build also checks that key public pages and the error page exist before it creates the runtime image. The final service runs as a non-root user and exposes one port for the nginx and API boundary.
Special-purpose images isolate work that should not run in the web process. The relay image builds inventory facts from its repository context, installs only its runtime dependencies, ships the relay's source and data graph, and exposes a health endpoint for container checks. The managed Umami image pins an upstream release and a reviewed session-data fix together, then verifies the patch, migration, and service build before its integration probes run. Digest notifications, bootstrap-tier publication, and seed bundles run other focused background jobs. The port-activity bundle carries the complete script tree for import safety, while the resilience validation bundle also carries the server code needed by its validation jobs.
The Redis REST image provides a small HTTP boundary for local workers. It requires a bearer token, permits only approved Redis commands or pinned scripts, and limits request bodies. The nginx configurations keep dashboard, embed, OAuth, YouTube, API, and static-data behavior distinct. The shared security-header contract keeps the self-hosted dashboard aligned with the hosted policy.
References: Dockerfile• L10-22 Dockerfile• L23-61 Dockerfile• L63-75 Dockerfile• L78-135 docker/Dockerfile• L3-7 docker/Dockerfile• L31-64 docker/Dockerfile• L67-86 docker/Dockerfile.redis-rest• L1-6 docker/redis-rest-proxy.mjs• L23-33 docker/redis-rest-proxy.mjs• L62-98 docker/redis-rest-proxy.mjs• L920-1015 Dockerfile.relay• L11-23 Dockerfile.relay• L235-243 Dockerfile.umami• L3-12 Dockerfile.umami• L102-165 Dockerfile.digest-notifications• L32-50 Dockerfile.digest-notifications• L67-140 Dockerfile.publish-bootstrap-tiers• L8-22 Dockerfile.seed-bundle-portwatch-port-activity• L18-31 Dockerfile.seed-bundle-resilience-validation• L14-52 .github/…/docker-publish.yml• L41-72 .github/…/docker-publish.yml• L175-270 docker/nginx-security-headers.conf• L1-18 docker/nginx.conf• L39-221 docker/nginx.conf.template• L33-177
Self-hosted image publication
The self-hosted image workflow publishes the three images needed by the Compose stack: the app, the AIS relay, and the Redis REST proxy. A main tag follows the Compose file on main, while release tags provide a version that operators can pin when they need a stable image set.
The workflow builds for both amd64 and arm64. It checks the crawlable pulse and GitHub star snapshot before building the app image, then starts that image and checks the main and Pro entry points before any image is pushed. A failed scheduled or main-branch rebuild creates or updates an issue so the published main tag cannot silently fall behind the Compose contract.
References: .github/…/docker-publish-self-hosted.yml• L1-42 .github/…/docker-publish-self-hosted.yml• L44-73 .github/…/docker-publish-self-hosted.yml• L115-123 .github/…/docker-publish-self-hosted.yml• L157-232 .github/…/docker-publish-self-hosted.yml• L234-270
Local container operation
The Compose stack gives a self-hosted operator one local boundary for the web app, relay, Redis, and the Redis REST proxy. Required session, relay, and Redis secrets stop startup when they are missing. Optional provider keys let individual data features degrade without blocking the stack.
The stack pulls images tagged main by default, which keeps a fresh clone aligned with the Compose file that starts it. Operators can pin a release with WM_IMAGE_TAG, or build all three images from source when a registry pull is not suitable. Redis keeps its data in a named volume, and both Redis and its REST boundary have memory limits that reduce the impact of large concurrent writes.
The nginx boundary serves static assets and generated public data with different cache rules, proxies API traffic to the local sidecar, blocks native management routes, and gives the embed surface a separate cross-origin policy. The container uses the same security goals as the hosted target rather than becoming a weaker deployment.
References: docker-compose.yml• L1-40 docker-compose.yml• L79-154 docker-compose.yml• L156-173 docker/entrypoint.sh• L1-30 docker/nginx.conf• L39-221 docker/nginx.conf.template• L33-177 docker/nginx-embed-security-headers.conf• L1-11 docker/nginx-oauth-authorize-security-headers.conf• L1-18 docker/nginx-youtube-embed-security-headers.conf• L1-18
Snapshot publication
Some expensive or time-sensitive reads become committed snapshots or durable archive entries that the public build or API can use. Scheduled workflows capture source data, save evidence, rebuild related public artifacts, and publish through either a review pull request or a controlled archive write.
The open-data chokepoint flow captures the public response once, stores that response and the generated files as an artifact, then publishes the daily snapshot to the Redis archive. Manual runs are dry runs unless an operator disables that mode. The archive can therefore update without waiting for a pull request or web deploy while the saved response remains available for repair.
Crawlable pulse refreshes now run weekly because their pages describe about 24 hours of movement. They create one ISO-week branch, retain the frozen capture as an artifact, rebuild the corpus, sitemap, forecast-accuracy section, and homepage welcome teaser, and open a review PR after verification. If verification fails, the branch still carries the capture as a draft so a test or generator fix can repair the same publication instead of losing an expensive run. A rehearsal workflow checks the refresh against a deliberately drifted capture before the weekly bot PR.
Resilience rankings, GitHub stars, and Search Console data refresh according to their freshness contracts. These flows create one period-specific branch, avoid duplicate review pull requests, retain evidence when needed, and arm auto-merge after verification. The resilience refresh also rebuilds the public and agent-facing surfaces that carry its current ranking and dispatches the new branch directly to the deployment gate. Optional Bing and Umami collections can add data without blocking the required Search Console snapshot.
The review pull requests depend on a separate token so their required checks can run. A scheduled token check warns before that token expires or when it no longer works.
References: .github/…/open-data-chokepoints.yml• L1-9 .github/…/open-data-chokepoints.yml• L27-64 .github/…/open-data-chokepoints.yml• L66-101 .github/…/crawlable-pulse-refresh.yml• L1-27 .github/…/crawlable-pulse-refresh.yml• L117-254 .github/…/crawlable-pulse-rehearsal.yml• L1-68 .github/…/resilience-snapshot-refresh.yml• L16-21 .github/…/resilience-snapshot-refresh.yml• L66-193 .github/…/resilience-snapshot-rehearsal.yml• L1-70 .github/…/github-stars-refresh.yml• L16-33 .github/…/github-stars-refresh.yml• L89-136 .github/…/seo-gsc-weekly.yml• L1-17 .github/…/seo-gsc-weekly.yml• L19-61 .github/…/seo-gsc-weekly.yml• L110-166 .github/…/seo-gsc-weekly.yml• L167-242 .github/…/review-pr-token-check.yml• L1-32
Worker delivery
A worker is a hosted service or scheduled task that refreshes source data or derived views outside a user request. Worker delivery keeps source collection and publication separate from the user-facing request path.
Railway's native GitHub deployment path owns normal production delivery. The service registry states the desired configuration, a registry sync applies the current main revision, and a separate viewer check confirms the live configuration. A six-hour drift check compares the running deployment with the same revision and reports configuration or deployment drift.
Manual Railway reconciliation is reserved for rollback or recovery. It admits only the exact current green main commit, separates preview from mutation, records immutable evidence, verifies terminal convergence, and uses a protected operator flow when a mutation is partial or uncertain. The normal deploy trigger must not become a second automatic deployment path.
Worker health depends on the age and coverage of the data it publishes, not only on whether its process is running. This separates missing data, stale data, partial coverage, and an upstream outage while unrelated product features remain available.
References: scripts/railway-services.json• L1-80 .github/…/railway-registry-sync.yml• L1-116 .github/…/railway-deploy-drift.yml• L1-54 .github/…/railway-deploy-trigger.yml• L1-319 .github/…/railway-reconcile-manual-recovery.yml• L1-303 .github/…/postmerge-deploy-monitor.yml• L1-62
Edge delivery
The CORS preflight Worker owns cross-origin access for api.worldmonitor.app. Its release path runs unit tests before deployment, serializes deployments so a delayed retry cannot overwrite newer code, waits for edge propagation, and then performs a live OPTIONS smoke test. Shadow telemetry is useful but does not block the CORS release when its secret update is unavailable.
References: .github/…/deploy-worker.yml• L1-38 .github/…/deploy-worker.yml• L40-103 .github/…/deploy-worker.yml• L105-124
Durable backend delivery
The Convex backend has a production delivery path separate from the web build. It compares the candidate with the commit recorded as actually deployed, deploys when backend code or an imported shared contract changes, and records the deployed commit for the next comparison. The deploy then seeds required production locks and runs an idempotent repair path so a successful code deploy does not leave required state absent.
The path is fail-closed when it cannot establish the deployed baseline or the changed-file list. A separate write job records the baseline after the deploy step succeeds, while seed failures remain visible to the deploy monitor. A manual run remains available for recovery.
References: .github/…/convex-deploy.yml• L25-55 .github/…/convex-deploy.yml• L58-169 .github/…/convex-deploy.yml• L171-274 .github/…/convex-deploy.yml• L276-353
Desktop delivery
The desktop release is one product line with one published binary version. The installed app selects its product variant after installation, so the release workflow builds the same release for supported desktop systems and publishes it only after all required platform assets and client values are present.
Each platform uploads to a draft release. The workflow publishes it only after the required assets exist, and it refuses to rebuild an already published release in place. Release and canary builds also require the client settings needed for sign-in, paid access, maps, and relay features. A scheduled Linux canary launches the installed AppImage and checks sidecar readiness, continued liveness, process survival, and rendered content. Publishing a desktop release also starts the related container image workflows because the release event comes from a token-authored action.
References: .github/…/build-desktop.yml• L1-86 .github/…/build-desktop.yml• L88-219 .github/…/build-desktop.yml• L318-394 .github/…/build-desktop.yml• L508-650 .github/…/desktop-release-train.yml• L1-85 .github/…/test-linux-app.yml• L1-15 .github/…/test-linux-app.yml• L79-154 .github/…/test-linux-app.yml• L154-324
SDK publication
Each SDK has an independent release boundary. A language-specific tag or manual release request first checks the package version and runs the package checks. Only the verified package is sent to its registry.
| Package | Release boundary | Publication control |
|---|---|---|
| Python | py-vX.Y.Z tag | PyPI trusted publishing through GitHub OIDC |
| Ruby | gem-vX.Y.Z tag | RubyGems trusted publishing through GitHub OIDC |
| Rust | Existing rust-vX.Y.Z tag | Main ancestry, locked checks, dry run, then crates.io OIDC |
| CLI | cli-vX.Y.Z tag | npm trusted publishing through GitHub OIDC |
Manual Python, Ruby, and CLI runs default to validation only. Rust publication also requires a verified tag on main. These controls keep package labels, tested source, and published artifacts aligned without long-lived registry tokens.
The Go SDK uses a tag as its release boundary and warms the module proxy after its checks pass. These release lines stay independent of desktop and one another.
References: .github/…/publish-python.yml• L1-15 .github/…/publish-python.yml• L17-71 .github/…/publish-ruby.yml• L1-18 .github/…/publish-ruby.yml• L20-69 .github/…/publish-rust.yml• L1-20 .github/…/publish-rust.yml• L22-88 .github/…/publish-go.yml• L1-78 .github/…/publish-cli.yml• L1-73
Plugin mirroring
The Claude plugin has a separate mirror repository because the consuming environment cannot read this repository's large archive. After a relevant change reaches main, the mirror flow copies the current plugin and pushes it with a deploy key. It checks current main so a retry cannot publish an older plugin over a newer one.
References: .github/…/mirror-claude-plugin.yml• L1-18 .github/…/mirror-claude-plugin.yml• L23-70
Configuration and dependency inputs
The system has two kinds of configuration. Hosting targets hold server and worker values, while build and runtime checks expose only safe feature availability to clients. A missing optional provider value can disable one feature without stopping unrelated feeds. A value required for a release or service must stop that release from being published.
Build configuration
The root build contract covers shared output, generated public data, page assets, API artifacts, and release preparation. Its lockfile fixes the dependency tree used by local builds, CI, Vercel, and web images. The frontend container accepts build-time variant and API values, while the runtime container receives its service settings at startup. The hosted route contract and both nginx targets must agree on the same public paths, cache rules, and security behavior.
References: package.json• L21-223 package-lock.json• L1-35 vercel.json• L1-4 vercel.json• L149-193 docker/Dockerfile• L23-30 docker/Dockerfile• L67-86 Dockerfile• L16-22 Dockerfile• L88-135
Background worker inputs
Seed workers use shared rules for provider access, retries, locks, cache reads, publication, and freshness metadata. A failed validation or exhausted upstream path preserves the last good value so health reporting can show age instead of treating a bad value as fresh.
Notification workers turn stored alert events into email, chat, push, and customer webhook deliveries. Webhook delivery uses HTTPS-only URLs, DNS checks, pinned public addresses, bounded responses, short timeouts, and signed request bodies.
References: scripts/_seed-utils.mjs• L74-118 scripts/_seed-utils.mjs• L453-651 scripts/_seed-utils.mjs• L701-779 scripts/…/notification-webhook-ssrf.cjs• L181-347 scripts/…/notification-webhook-signing.cjs• L7-53 Dockerfile.digest-notifications• L16-30
Generated public contracts
Generated public files turn source definitions into stable contracts for users, agents, paid plans, search systems, and external tools. Generation checks keep product facts, machine-readable interfaces, snapshots, and page assets aligned with their source rules before publication.
References: scripts/…/product-catalog.generated.json• L1-12 scripts/…/product-facts.generated.json• L1-25 scripts/build-openapi-json.mjs• L1-20 .github/…/proto-check.yml• L8-127 .github/…/proto-check.yml• L650-729
Pro bundle freshness
The Pro build is part of the deployment output, not a committed public bundle. Pull request checks rebuild it when its source or product facts change, verify that its served pages exist, and compare the committed catalog, locale, and product artifacts with regenerated output before the deployment can use them.
References: .github/…/pro-bundle-freshness.yml• L3-17 .github/…/pro-bundle-freshness.yml• L18-50 .github/…/pro-bundle-freshness.yml• L52-114 .github/…/test.yml• L701-724
Coverage gap checks
The submarine cable gap check compares the curated cable set with the upstream catalog and produces a review list of missing in-service cables. It fails on incomplete upstream reads, so an operator does not mistake a partial scan for a complete coverage result.
References: scripts/check-submarine-cable-gaps.mjs• L1-4 scripts/check-submarine-cable-gaps.mjs• L16-72
Verification and merge safety
Continuous integration selects checks from changed files, then uses aggregate jobs to require every selected check to pass or to be explicitly skipped. If GitHub returns an incomplete or uncertain file list, selection runs the broader check set. Pull request runs can be canceled when superseded, but a push to main keeps its own verdict for the deployment path.
Build and test checks
The test workflow reuses built output, spreads data tests across five shards, and adds focused checks for browser behavior, API and sidecar contracts, desktop drift, managed analytics, package integrations, and service boundaries. Aggregate jobs make a skipped or missing selected check visible instead of treating it as a complete result. Desktop checks also keep release configuration, sidecar bundles, shipped runtime routes, and installed-app behavior close to the release path.
Browser smoke tests no longer retry Playwright failures. A browser crash therefore fails its test instead of being absorbed as a flaky result, while the workflow still records memory and browser diagnostics for investigation. The same checks cover the dashboard variants, /pro gates, WebMCP, and plugin views.
The Pro checks build the deployment output before browser gates use it. The sidecar checks build the files that the desktop app ships and verify that each discovered route has a bundle. Desktop configuration checks cover the single Tauri configuration, client environment parity, the bundled Windows runtime, and Rust security floors. Managed Umami changes run a real PostgreSQL integration and runtime write probe when their image, migration, patch, or focused tests change.
References: .github/…/test.yml• L1-20 .github/…/test.yml• L67-220 .github/…/test.yml• L400-767 .github/…/test.yml• L768-1028 .github/…/test.yml• L1064-1436 .github/…/test.yml• L1506-1543
Visual evidence
The visual workflow checks committed map screenshots and captures a named Chrome gallery. It runs on relevant pull requests and on a nightly schedule, stores Playwright artifacts, and keeps visual evidence separate from the merge-blocking Test aggregate.
References: .github/…/e2e-visual.yml• L1-35 .github/…/e2e-visual.yml• L39-94 .github/…/e2e-visual.yml• L96-151
Code security scans
CodeQL scans Actions, Go, JavaScript and TypeScript, Python, Ruby, and Rust once each day or on manual request. It does not run on every merge, so it reduces merge queue cost while still providing a scheduled security signal. Production dependency audits remain a separate required check.
References: .github/…/codeql.yml• L1-16 .github/…/codeql.yml• L21-68 .github/…/security-audit.yml• L1-34 .github/…/security-audit.yml• L233-311
Code generation checks
The proto workflow checks compatibility for schema changes and keeps generated artifacts fresh. It treats fork-controlled generation as untrusted, allows a trusted internal path to create a generated commit, and then verifies the exact new head and merge result. A main-only cache warmer supplies the buf and protoc tools, while pull request jobs restore but do not write that shared cache.
References: .github/…/go-tools-cache.yml• L3-14 .github/…/go-tools-cache.yml• L15-82 .github/…/proto-check.yml• L8-20 .github/…/proto-check.yml• L117-137 .github/…/proto-check.yml• L239-298 .github/…/proto-check.yml• L300-429 .github/…/proto-check.yml• L431-529 .github/…/proto-check.yml• L536-729
Deployment gate
The deployment gate publishes a commit status only after the required checks pass for the same commit. It reacts to completed workflow runs and also sweeps pending or uncertain pull requests every 30 minutes. Exact-SHA manual recovery is available when event delivery or API timing strands a result.
The gate invalidates results that use an old required-check contract. It serializes evaluation per commit and treats planner, invalidation, recovery, and evaluation errors as failures. This makes a missing verdict visible to branch protection rather than allowing an absent check to look like approval.
References: .github/…/deploy-gate.yml• L1-49 .github/…/deploy-gate.yml• L51-179 .github/…/deploy-gate.yml• L181-212
Live health checks
Operational checks use more than process status. They inspect deployment evidence, producer freshness, storage room, public protocols, authentication, cache behavior, error rates, and selected third-party sources. A health result keeps a source failure separate from an observation failure: stale data is a data problem, while an unreadable monitoring record is an unknown.
The deployment error canary compares each high-traffic route's recent client error rate with that route's own seven-day baseline. It runs independently of repository state, so a production regression can still alert when main is green or no related change is present. The Umami storage monitor also keeps a bounded history and checks both database headroom and the retention worker, while allowing a recent stored sample to cover a temporary provider read failure.
The authenticated MCP tool canary checks every read-only production tool rather than only the MCP protocol shell. It uses a customer-bound key, validates each result against the tool's advertised schema, records failures in one tracking issue, and retains the run report as an artifact. Its scheduled run fails when a tool fails, so the issue history and workflow alert cover both repeated and immediate failures. The separate MCP live smoke also checks the anonymous production protocol, OAuth paths, aliases, and proxy surface every 15 minutes.
The status page monitor renders the public status page and checks all four uptime windows. It preserves every attempt and diagnostic report, then keeps one issue open through failure and closes it after recovery.
References: api/health.js• L37-91 src/…/health-freshness.ts• L26-35 .github/…/deploy-error-canary.yml• L1-37 .github/…/mcp-live-smoke.yml• L1-196 .github/…/mcp-tool-canary.yml• L1-77 .github/…/live-api-cache-auth.yml• L68-143 .github/…/tps-open-data-live.yml• L40-85 .github/…/status-page-monitor.yml• L1-115 .github/…/umami-storage-monitor.yml• L1-163
Live product review
The weekly dogfood review observes the live product as an anonymous visitor and turns findings into one review issue per ISO week. It separates browsing from issue publication so the browser agent has read-only access and cannot write arbitrary issue content; a maintainer decides which findings are real.
References: .github/…/dogfood-weekly.yml• L1-13 .github/…/dogfood-weekly.yml• L37-76 .github/…/dogfood-weekly.yml• L102-141 .github/…/dogfood-weekly.yml• L143-181
Deployment monitoring
Post-merge monitors check deployments that cannot gate the pull request that caused them. They read the latest hosted deployment state and fail loudly when main is green but production did not receive the change. Independent collector, freshness, error-rate, storage, Railway drift, status-page, dogfood, and CI-health monitors turn slow or silent operational decay into an issue, artifact, or visible workflow failure.
References: .github/…/postmerge-deploy-monitor.yml• L1-62 .github/…/analytics-collector-monitor.yml• L1-29 .github/…/pulse-freshness-monitor.yml• L1-53 .github/…/deploy-error-canary.yml• L14-37 .github/…/status-page-monitor.yml• L86-115 .github/…/ci-health-scan.yml• L1-42 .github/…/railway-deploy-drift.yml• L1-54 .github/…/umami-storage-monitor.yml• L1-163
Operational documentation
The automation page explains how repository automation is organized. The deployment page follows release and hosting paths in more detail. The operational guardrails page explains the fail-closed checks that protect production changes. The API platform page explains the public service and machine interface boundary. The data ingestion page explains how source data enters the product, and the application state page explains how durable user and product state supports running services. The seed pipeline orchestration page explains how related seed work is grouped and run. The alerts and notification delivery page follows alert creation, filtering, and delivery to users. The Tauri desktop application page explains the installed desktop client and its release behavior.
Change checklist
When these files or sections are changed, remember to consider these:
- When a root build script, build-time variable, or dependency changes, update the lockfile, hosted build contract, CI install path, and every image that installs the affected dependency.
- When the Vercel build command or freshness gate changes, keep the pulse and star age checks, full build, ignore decision, and public route contract aligned.
- When a public route, API upstream, cache rule, or security header changes, check both the Vercel contract and the nginx container paths. Run the matching live cache, protocol, or host-alias probe.
- When an open-data directory or file route changes, keep the directory landing response, file rewrite, API handler, and public cache or content type behavior aligned.
- When a container handler or health route changes, keep the proxy path and health probe aligned so the probe still checks the complete runtime.
- When the self-hosted Docker image, Compose file, or image workflow changes, keep the three image names, build contexts,
maintag behavior, release pin, and local service set aligned. - When the self-hosted app image changes, keep its page smoke checks and container health path aligned with the routes the image must serve.
- When relay, digest, bootstrap-tier, or seed-bundle imports change, update the image inputs and their import or build checks together.
- When the managed Umami release, patch, or migration changes, keep the pinned upstream commits, byte checks, runtime image, and PostgreSQL integration probes aligned.
- When the Redis REST proxy changes, keep its command and script authority, body limit, authentication, error status, and caller parity aligned.
- When the CORS Worker changes, keep its path trigger, deployment order, rate-limit retry, propagation wait, telemetry behavior, and live preflight smoke aligned.
- When a snapshot publication changes, keep its source response, freshness ceiling, period key, branch identity, dry-run behavior, generated consumers, and evidence retention aligned. Keep direct archive publication separate from review-PR publication.
- When a snapshot refresh has a rehearsal workflow, keep its path filter and drifted-capture runner aligned with every build and verification step in the scheduled refresh.
- When the set of resilience snapshot consumers changes, update the rebuild inventory, staged generated surfaces, and data-suite selection together so no public ranking surface stays stale.
- When the review PR credential changes, check every bot-created snapshot PR and the expiry monitor. A token that can push but cannot start required checks can leave a valid refresh waiting without a verdict.
- When Railway service configuration changes, update the desired registry, current-main resolution, native deployment checks, drift audit, and the manual recovery contract together.
- When a Railway recovery path changes, keep exact-head admission, lease or durable-state fencing, immutable result evidence, terminal verification, and operator decisions aligned.
- When path-aware CI selection changes, keep incomplete-file fallbacks, cancellation behavior, main-push verdicts, aggregate jobs, and selected-check names fail-closed.
- When generated proto paths or inputs change, update the input and output registries, compatibility check, trusted generation path, exact-head freshness check, and follow-up status together.
- When the proto tool cache changes, keep its main-only writer, restore-only pull request consumers, cache key, and generated tool set aligned.
- When a required-check contract changes, update the gate workflow list, same-SHA status logic, invalidation rules, bot handoff, and recovery sweep together.
- When a live probe is added or renamed, keep its trigger, required-run assertion, target URL, timeout, and issue or artifact report aligned. Keep production-only probes out of pull requests.
- When an authenticated MCP tool or schema changes, keep its read-only marking, deterministic canary input, output schema, credential allowance, issue report, and artifact report aligned.
- When the public MCP or OAuth surface changes, keep the 15-minute live smoke, deployment-status filter, alias and proxy probes, and production report aligned.
- When the status page changes, keep fixture coverage, rendered uptime windows, production-only scheduling, evidence retention, and failure-recovery issue behavior aligned.
- When the dogfood browser or issue writer changes, keep read-only browsing, empty browser environment, output-directory limits, inert issue fields, and maintainer review boundaries aligned.
- When an SDK release changes, keep the tag pattern, package version, verification commands, dry-run path, registry trust setup, and OIDC identity aligned.
- When a monitor changes, keep its independent trigger, concurrency policy, credentials, baseline or freshness source, and durable failure report aligned.
- When browser smoke retry or diagnostic behavior changes, keep crash handling, failure propagation, memory evidence, and Playwright artifact collection aligned.
- When the Pro build or product catalog changes, keep the deployment build, generated catalog and locale files, page existence checks, and Pro browser gates aligned.
- When the desktop release changes, keep the single version tag, client environment preflight, platform asset names, draft release gate, and post-publish image dispatch aligned.
- When the Convex backend or an imported shared contract changes, update the deployed-baseline path, production seed and repair steps, baseline marker, and post-deploy monitor together.
Important failure modes
| Trigger | Consequence | Guard |
|---|---|---|
| A needed path is missing from the Vercel or container build contract | A required deployment can be skipped or a container can serve the wrong route | Conservative build fallback, image assertions, route smoke tests, and live probes |
| Hosted route or security policy changes without the matching nginx contract | Self-hosted users receive different routing, cache, or browser security behavior | Shared security-header contract and parallel hosted/container review |
| An open-data directory path has no landing response | Users and crawlers reach an unclear or incorrect dataset entry point | Explicit directory redirects and file-level API rewrites |
| A special-purpose image omits a local import | A worker can fail at startup and stop feeds, validation, or notifications | Complete image inputs and import-coverage checks |
| A multi-platform image build fails during an unattended run | No verified image is published and the failure can remain unseen | Route smoke before publish and a durable rebuild issue |
The Compose file and published main images come from different revisions | A fresh self-hosted install can pair incompatible startup settings and services | Same-tree main image publication, release pinning, and rebuild failure reporting |
| The self-hosted app image serves an empty or wrong entry point | Operators can receive a published image that starts but does not serve the product | Pre-push checks for / and /pro/ plus the container health check |
| A Redis proxy update widens command access or drifts from a caller's pinned script | A local worker can lose atomic writes or gain unsafe data-store authority | Central command admission, exact script parity checks, bounded bodies, and bearer authentication |
| A delayed edge retry deploys older Worker code | A newer CORS change is overwritten | Serialized deployment and rate-limit-only retry behavior |
| A required check result belongs to another SHA or an old contract | Branch protection can accept evidence for code that was not evaluated | Same-SHA evaluation, invalidation, and recovery sweep |
| A file list is incomplete during CI selection | A needed check can be skipped and produce a false green result | Conservative fallback to the full check set and aggregate jobs |
| A generated artifact is stale on a trusted PR head or merge result | API clients or server handlers can use contracts that do not match their source | Proto compatibility, trusted generation, exact-head freshness, and follow-up validation |
| A shared codegen cache is written from an untrusted branch or lacks a required tool | Pull requests can use unsafe or incomplete generation inputs | Main-only cache writer, restore-only consumers, and binary presence checks |
| A production audit or scheduled CodeQL scan has no verdict | A dependency or code security issue can remain unreported while the schedule looks healthy | Per-lockfile verdicts, scheduled run checks, and visible failure reporting |
| The Pro deployment build omits a page or uses stale generated catalog data | The hosted /pro surface can return a 404 or show old product facts | Build-output existence checks and generated-file freshness checks |
| A Railway run applies an old trigger revision or live configuration drifts | Production can run an older service definition than current main expects | Current-main resolution, registry reconciliation, viewer verification, and six-hour drift checks |
| A Railway mutation ends with partial or uncertain evidence | An automatic retry can repeat an unknown production change | Immutable result manifest, durable fencing, terminal verifier, and protected manual recovery |
A Convex deploy skips a change outside convex/ that it imports | Production keeps an old shared rule while the web code moves ahead | Deployed-baseline diff and the imported-contract path registry |
| A post-deploy Convex seed or repair fails | New backend code can run with missing locks or stale derived state | Idempotent post-deploy seeds, repair job, and explicit seed verification |
| A desktop release lacks a client value or platform asset | Installed clients can lose a feature or be offered an unusable update | Release preflight, draft staging, asset completeness gate, and installed-app canary |
| A desktop release reruns against an already published tag | Installed clients can receive assets changed outside the release gate | Published-release refusal and one final draft-to-live publish step |
| An SDK tag does not match its package version | Consumers receive a mislabeled or untested package | Tag/version checks and verification before trusted publication |
| A snapshot is rebuilt from a stale branch or mixed response | The committed index can describe files that are not in the branch | Saved source response and fresh-main rebuild |
| A snapshot consumer is omitted from the rebuild or staged set | A public page or agent file keeps an old ranking after the snapshot is published | Consumer inventory, generated-surface staging, and data-suite checks |
| A snapshot rehearsal does not exercise the scheduled refresh's full verification set | A bot-created snapshot PR can fail on a value-dependent test that passed on its source PR | Drifted-capture rehearsal with a filter that covers every refresh input and test |
| A review PR token expires or loses CI access | A generated snapshot branch can wait without required checks or auto-merge | Scheduled token and expiry check plus period-specific branch reconciliation |
| An authenticated MCP tool returns a transport-success response with a bad result | Customers or agents can receive an unusable tool while the protocol endpoint appears healthy | Full read-only tool canary, output-schema validation, issue tracking, and artifact report |
| A production MCP alias, OAuth path, or proxy fails between deploys | Machine clients can fail while repository checks remain green | Fifteen-minute external smoke with alias, OAuth, and proxy coverage |
| A browser smoke test retries a crash instead of failing it | A real browser failure can appear as a flaky or green result | No Playwright retry plus browser-loss diagnostics and required smoke status |
| A visual job is treated as the merge gate without its required trigger or artifact path | A visual regression can be hidden or a non-required evidence job can block unrelated merges | Separate evidence workflow, explicit path and schedule triggers, and retained artifacts |
| A production error rate rises while repository checks remain green | Users can see a live regression with no merge-time signal | Independent route-baseline canary |
| A monitor depends on the repository's green state | A production outage can silence the monitor | Independent production trigger and durable report |
| The status page omits one uptime window or the monitor issue is not recovered | Operators can trust incomplete status data or miss that service returned | Four-window rendered smoke, retained evidence, and one failure-and-recovery issue |
| The dogfood browser can read secrets or publish agent-controlled issue text | Page content can influence repository writes or expose credentials | Separate browser user, empty environment, read-only permissions, restricted output, and inert field rendering |
| The Umami patch or migration does not match its pinned upstream commits | Analytics can run with an unreviewed application/schema mismatch | Commit identity checks, byte comparisons, migration checks, and PostgreSQL runtime probes |
Key files
| File | Why to read it |
|---|---|
| package.json | Defines the shared build, test, data refresh, and release commands. |
| vercel.json | Defines the hosted build and public route, header, cache, and discovery contract. |
| scripts/vercel-ignore.sh | Defines when Vercel skips a build and when uncertain branch state forces one. |
Dockerfile | Defines the full web and API image boundary and health path. |
Dockerfile.umami | Defines the pinned managed analytics image, upstream patch checks, and migration build. |
| docker-compose.yml | Defines the self-hosted service set, image tags, required secrets, persistence, and resource bounds. |
docker/Dockerfile | Defines the frontend-only image and its static nginx boundary. |
docker/nginx.conf | Defines the full-image static routes, local API proxy, cache policy, and protected management paths. |
docker/nginx.conf.template | Defines frontend-only static routes, API proxying, cache policy, and special security boundaries. |
docker/nginx-security-headers.conf | Defines the shared self-hosted dashboard security contract. |
| docker/redis-rest-proxy.mjs | Defines the authenticated Redis boundary, command policy, body limit, and pinned atomic scripts. |
Dockerfile.relay | Defines the relay image boundary, runtime inputs, generated inventory artifact, and health check. |
Dockerfile.digest-notifications | Defines the scheduled notification image and its cross-directory runtime contract. |
Dockerfile.seed-bundle-portwatch-port-activity | Defines the standalone port-activity worker image. |
Dockerfile.seed-bundle-resilience-validation | Defines the validation worker image and its server-code runtime contract. |
| .github/…/docker-publish.yml | Defines image triggers, route smoke tests, multi-platform publication, provenance, SBOM, and failure reporting. |
| .github/…/docker-publish-self-hosted.yml | Defines publication and pre-push smoke checks for the three images used by the self-hosted Compose stack. |
| .github/…/test.yml | Defines changed-file selection, build reuse, test shards, browser smoke behavior, desktop checks, analytics integration, and aggregate results. |
| .github/…/e2e-visual.yml | Defines visual evidence runs, screenshot suites, and retained galleries. |
| .github/…/lint-code.yml | Defines conservative code and documentation check selection. |
| .github/…/pro-bundle-freshness.yml | Defines the Pro deployment build and generated catalog freshness checks. |
| .github/…/proto-check.yml | Defines schema compatibility, trusted generation, and generated-artifact freshness. |
| .github/…/go-tools-cache.yml | Defines the main-only cache warmup for proto generation tools. |
| .github/…/security-audit.yml | Defines package and Rust audits and the aggregate security verdict. |
| .github/…/codeql.yml | Defines the scheduled multi-language CodeQL scan and its cache policy. |
| .github/…/deploy-gate.yml | Defines same-SHA deployment approval, stale-check invalidation, recovery, and status publication. |
| .github/…/convex-deploy.yml | Defines production backend change detection, deploy, seed, repair, and deployed-baseline recording. |
| .github/…/build-desktop.yml | Defines the single desktop release line, platform assets, draft release, client configuration checks, and publish gate. |
| .github/…/test-linux-app.yml | Defines the installed desktop canary and its sidecar, liveness, and render checks. |
| .github/…/open-data-chokepoints.yml | Defines the saved-response snapshot, dry-run mode, archive publication, and evidence artifact. |
| .github/…/crawlable-pulse-refresh.yml | Defines the weekly crawlable pulse capture, generated consumers, retained evidence, and review PR. |
| .github/…/crawlable-pulse-rehearsal.yml | Defines the drifted-capture rehearsal for the weekly pulse refresh. |
| .github/…/resilience-snapshot-refresh.yml | Defines the resilience snapshot publication path, its generated consumers, and direct Deploy Gate handoff. |
| .github/…/resilience-snapshot-rehearsal.yml | Defines the drifted-snapshot rehearsal for the resilience refresh. |
| .github/…/github-stars-refresh.yml | Defines the monthly public star-count snapshot and review path. |
| .github/…/seo-gsc-weekly.yml | Defines Search Console collection, optional source snapshots, and review-PR publication. |
| .github/…/review-pr-token-check.yml | Defines the early warning for the credential used by bot-created review pull requests. |
| .github/…/mcp-live-smoke.yml | Defines external production checks for the MCP, OAuth, aliases, and proxy surface. |
| .github/…/mcp-tool-canary.yml | Defines authenticated production checks for every read-only MCP tool and their durable failure report. |
| .github/…/dogfood-weekly.yml | Defines the isolated weekly live-product review and issue publication boundary. |
| .github/…/status-page-monitor.yml | Defines rendered status-page checks, evidence, and failure recovery. |
| .github/…/railway-registry-sync.yml | Defines current-main registry reconciliation and separate live verification. |
| .github/…/railway-deploy-drift.yml | Defines scheduled Railway deployment and configuration drift checks. |
| .github/…/railway-deploy-trigger.yml | Defines the manual, exact-head Railway recovery mutation and terminal verifier. |
| .github/…/railway-reconcile-manual-recovery.yml | Defines protected operator decisions for uncertain Railway reconciliation. |
| .github/…/umami-storage-monitor.yml | Defines bounded storage history, capacity checks, and retention-runner monitoring. |
| scripts/railway-services.json | Maps worker entry points to their service mode, inputs, and cadence. |
| api/health.js | Defines the server health view and its relationship to source and cache evidence. |
| src/…/health-freshness.ts | Defines cadence-aware producer freshness and data status classification. |