Automated test harness
The automated test harness checks the product at its main user and delivery boundaries. It covers detector behavior, command-line use, hooks, agent workflows, browser live mode, editor packages, plugin loading, engine downloads, and release safety. Fast local suites give repeatable checks. Browser, remote, and provider-backed suites run only when their tools and services are available. See also Distribution and engineering tooling, Build, release, and validation, Plugin packaging and installation, Live editing sessions, and Agent hook workflow.
Suite model
The test system groups checks by product boundary. Each suite has a clear purpose and a selection rule so a change runs the smallest useful set of checks.
Suite groups
The default local run covers deterministic product checks, the engine oracle, detector and extension packaging, native UI bundle parity, live helper behavior, framework fixtures, token budgets, motion capture timing, and the committed plugin in a real Claude Code loader when that CLI is available. Browser sweeps, remote install checks, and provider-backed agent checks remain opt-in because they need more time or external services.
| Suite | Main guarantee | Run mode |
|---|---|---|
core | Repository contracts, build output, provider transforms, hook manifests, plugin guards, release guards, workflow security, token budgets, motion capture timing, and native UI bundle parity | Default |
oracle | Stable command and engine behavior against recorded cases and expected files | Default; skips without an engine binary |
detector | Browser extension packaging and tracked detector registry output | Default when its inputs change |
live | Live-mode reference behavior and helper contracts | Default |
framework | Engine live injection, CSP detection, and source wrapping across project shapes | Default; skips without an engine binary |
plugin-e2e | Component discovery by the real Claude Code loader | Default; skips without claude |
live-e2e | Full browser click-to-accept behavior across runtime fixtures | Opt-in; Playwright |
new-work-e2e | Browser decisions for new-work concept, direction, build-path, and question flows | Opt-in; Playwright and an engine binary |
cli-remote-e2e | Remote CLI install and update smoke | Opt-in; behavior moved to the engine repository |
live-e2e-accept-cleanup | Provider-backed cleanup after live acceptance | Opt-in; Playwright and a provider key |
skill-behavior | Provider-backed setup, routing, questions, and write order | Opt-in; language model keys |
skill-workflow | Provider-backed attended builds, visual evidence, handoff, and system documentation | Opt-in; Playwright, engine, and language model keys |
The suite registry is the source of truth. A registry check finds every test file, requires it to belong to a suite, prevents duplicate default membership, and confirms that editing a test file selects a suite that runs it. The registry derives self-file triggers from each command, so a changed test file cannot silently fall outside change-based CI.
References: scripts/test-suites.mjs• L4-13 scripts/test-suites.mjs• L28-117 scripts/test-suites.mjs• L118-235 tests/test-suites.test.mjs• L14-75
Test selection
Local commands expand default, all-local, and all into named suites. CI always runs the core checks, then maps the event and changed paths to deterministic jobs. A pull request normally runs only affected deterministic groups. A manual run enables opt-in groups. The nightly schedule adds the full live browser matrix without enabling paid provider runs.
The selection path turns an event and its changed files into workflow outputs. Jobs then install only the browser, engine, or provider tools that their selected checks need. The runner starts each command in its own process group. It applies time limits, stops a wedged group, and fails the suite if the run leaves a marked live server behind.
This diagram shows how a change becomes a set of verification jobs.
flowchart LR
C[Event and changed paths] --> P[CI test plan]
P --> D[Deterministic jobs]
P --> E[Opt-in jobs]
D --> O[Core, oracle, detector, live, framework]
E --> B[Browser, remote, or provider checks]
O --> V[Build and tracked-output checks]
References: scripts/run-tests.mjs• L21-55 scripts/run-tests.mjs• L65-119 scripts/ci-test-plan.mjs• L6-59 tests/ci-test-plan.test.mjs• L10-135 .github/…/ci.yml• L24-143
Shared test environments
The harness stages real project trees and tool processes in temporary locations. Tests can observe source writes, generated files, process output, and final state without changing the checkout, the developer's home directory, or a real agent installation.
Temporary workspaces
Temporary workspaces give CLI, engine, live, release, and agent tests a known file system. The agent harness limits file tools to workspace-relative paths and records reads, writes, questions, commands, and changed paths. This makes the order of an agent's actions part of the evidence, not only its final answer.
The engine locator accepts an explicit IMPECCABLE_BIN, a fetched platform binary, or a local release build. Oracle and browser suites can therefore use the same engine contract in CI and during local source work. Provider-backed runs use explicit credentials and bounded turns. Local credentials can come from a private ~/…/dev.env file or an override path. Child processes receive a filtered environment so provider keys do not spread into fixture servers or shell helpers.
References: tests/…/harness.mjs• L1-19 tests/…/harness.mjs• L37-61 tests/…/harness.mjs• L119-191 tests/…/dev-keys.mjs• L1-12 tests/…/dev-keys.mjs• L48-87 tests/…/engine-bin.mjs• L1-49 tests/…/session.mjs• L48-64
Framework fixtures
Framework fixtures are small project shapes with expected live-mode results. The static fixture pass stages each tree as a temporary Git project and checks injection, removal, content security policy classification, and source target selection. Fixtures with a runtime block also install dependencies, start a development server, and enter the browser suite.
The fixture contract lets one project shape support both narrow engine checks and a user-visible browser flow. It now covers a nested application below a repository root, Vite 8 with React, Vite 7 with a Vue single-file component, manual copy changes, duplicate targets, orphaned wrappers, and foreign sessions. The Vue case keeps reactive form state in source while the harness checks that a live session can select, cycle, and accept the form without losing its source structure. These cases protect root selection, source ownership, state preservation, and safe recovery without treating a successful browser render as proof that source edits are correct.
References: tests/framework-fixtures.test.mjs• L30-74 tests/framework-fixtures.test.mjs• L93-264 tests/…/README.md• L1-14 tests/…/README.md• L102-121 tests/…/README.md• L144-176 tests/…/fixture.json• L1-41 tests/…/fixture.json• L1-55 tests/…/fixture.json• L1-63 tests/…/App.vue• L1-14
Process cleanup
Live tests mark every server that they start. In-process cleanup handles normal exits and signals. A detached reaper handles a killed test process or a timeout that prevents normal teardown. The top-level runner adds a second check after each command, so a leaked server cannot poison a later test with a stale port or session.
References: tests/…/live-servers.mjs• L1-19 tests/…/live-servers.mjs• L45-83 tests/live-server-leak.test.mjs• L129-213 tests/…/lib.mjs• L31-39
Behavioral verification
The harness compares user-visible behavior with stable expected results. Recorded cases provide broad command coverage. End-to-end runs cover behavior that needs a browser, a real loader, or a real provider.
Oracle corpus
The oracle corpus is a behavior record for engine commands. Each case supplies a command, inputs, a staged workspace, and files to inspect. The runner compares output, errors, exit status, signals, and selected files after it normalizes paths and other run-specific values.
The corpus covers context and doctor checks, detector commands, CLI help and version output, hooks, live commands, design work, questions, image helpers, component-review planning, manual copy apply and discard, skill installation, local control commands, and motion-capture command boundaries. Recent cases pin loop help and unknown-verb failures, signed-out critique preference state, DESIGN.md sync settings and refresh behavior, setup status output, and motion-capture URL-scheme refusal. Detector records also include wider edge cases for layout, contrast, typography, page-level styles, overlays, mock frames, SVG text, moving evidence, and source forms. Hook records cover provider-shaped envelopes, duplicate events, session cache state, per-edit scans, Stop events, linked stylesheet ownership, the Hermes plugin's Stop decision path, and a VS Code patch-string response. Context records cover monorepo surface-brief targets from the caller's directory and workspace discovery in a tooling-heavy monorepo. Question records cover state recorded under another project root, missing state, hidden-tab heartbeats, answer collection, and stopping one root without removing another. Live records now cover locator text that resolves to a later file or search directory, dynamic rendered text, style-only locator hits, and insert or wrap fallback when source text is absent. Comparison records also pin the offset used for each component-diff region.
Oracle workspaces use real paths and stable placeholders for the workspace, repository, home directory, timestamps, process IDs, UUIDs, and other run-specific values. Multi-step cases keep one workspace alive across commands, so the record can prove state transitions rather than only one response.
Reviewed deltas are part of the contract. A golden may differ from the previous implementation only when the behavior has a recorded explanation and review entry. Recent records cover detector fixture growth, reviewed page-level and overlay decisions, evidence motion, live manual removal, installer detection, ignore and advisory boundaries, output changes caused by the native engine, question state lookup, monorepo brief lookup, directory scans that include .mjs and .cjs files, the exact additional-context shape for a VS Code patch event, and per-region component-diff offsets.
References: tests/…/lib.mjs• L1-25 tests/…/lib.mjs• L47-101 tests/…/lib.mjs• L248-305 tests/…/run.mjs• L1-32 tests/…/README.md• L1-35 tests/…/motion.mjs• L1-7 tests/…/critiques.mjs• L1-21 tests/…/health-design.mjs• L1-21 tests/…/setup.mjs• L1-5 tests/…/context.mjs• L1-25 tests/…/hooks.mjs• L1-18 tests/…/hooks.mjs• L98-126 tests/…/live-wrap.mjs• L1-11 tests/…/live-wrap.mjs• L89-100 tests/…/package.json• L1-5 tests/…/DELTAS.md• L7334-7351 tests/…/critiques-on-signed-out.json• L1-19 tests/…/health-design-on.json• L1-8 tests/…/setup-status-default.json• L1-7 tests/…/live-wrap-text-picks-later-file.json• L1-3 tests/…/comp-diff-json.json• L1-7
Detector and extension checks
Detector verification uses the engine's own tests and the oracle corpus for rule behavior. The repository-side detector suite checks the browser extension package and the tracked rule registry. Fixtures pair flag and pass cases across common source formats and project shapes, so a rule must identify a real issue without treating valid, hidden, disabled, or unpainted states as failures.
The fixture corpus now tests painted color palettes, text and control contrast, gradients, spacing at visible edges, em-dash use, heading structure, nested cards, mock frames, font loading, page-level colors, motion, SVG text, overlays, and evidence that changes after a scan. Paired cases also separate static source checks from browser-only judgments such as paint order, masks, covered panels, viewport geometry, loaded fonts, and the page canvas. The broken-image checks treat srcset and framework source bindings as valid image sources while still reporting empty or unrelated bindings. Source scans also ignore image-like text inside JavaScript and TypeScript literals. Directory scans keep their file set aligned with direct scans by walking JavaScript module files such as .mjs and .cjs.
The extension checks exercise startup in Chrome-style and Firefox-style hosts. They prove that concurrent scans share one core host, that a failed host can retry, that scanning waits for a successful readiness response, and that WASM startup errors are reported instead of becoming a false clean result. DevTools panel checks add the copy boundary: advisory findings stay advice, mapped advisory fixes remain available, and ordinary findings keep their repair suggestions. Popup checks add the active-tab boundary: partial reports remain visible but cannot be copied, navigation and failed scans invalidate old results, and delayed state or clipboard work cannot overwrite a newer report. Repeated scans keep one overlay, and browser-facing smoke pages cover clean and flagged CSP-constrained pages.
Account checks extend this boundary to sign-in and paid walks. They verify PKCE callbacks, issuer and endpoint pairing, scope negotiation, durable refresh-token state, worker restart recovery, sender checks, server switching, sign-out, journey notices, persona data, and safe image loading. Agent checks verify the native host link, developer-build admission, store-build removal, side-panel consent, per-site access, secret redaction, same-tab control, session stop, and host loss. Walk checks verify hand-off origin and socket pairing, store and Firefox capability differences, cross-origin write guards, sign-in pauses, page reads, takeover, cleanup, and the shared web-app protocol.
The shared report formatter is checked separately from its popup and DevTools callers. It keeps the scanned URL, finding count, selector, category, detail, page-level location, advisory section, and mapped skill suggestion consistent across both copy surfaces. Aggregate sweeps keep counted findings separate from advisory notes. A missing linked stylesheet is reported as an incomplete scan warning rather than as a false finding. Directory checks also distinguish configured template files from plain files and verify that ignored files, rules, and glob boundaries stay ignored in direct and page-mediated scans.
References: tests/…/detect.mjs• L45-220 crates/…/broken_image_source.rs• L1-39 crates/…/missing.html• L1-30 crates/…/broken_image.rs• L1-57 tests/…/ai-color-palette.html• L8-14 tests/…/evidence-after-scan.html• L8-29 tests/…/intro-unnamed-clears.html• L16-20 tests/…/mockup-structure.html• L5-35 tests/extension-build.test.mjs• L66-160 tests/extension-build.test.mjs• L213-282 tests/extension-build.test.mjs• L329-518 tests/extension-devtools.test.mjs• L56-156 tests/extension-popup.test.mjs• L92-287 tests/extension-popup.test.mjs• L360-861 tests/extension-account.test.mjs• L19-81 tests/extension-account.test.mjs• L152-280 tests/extension-account.test.mjs• L285-476 tests/extension-account.test.mjs• L722-803 tests/extension-agent.test.mjs• L16-36 tests/extension-agent.test.mjs• L149-340 tests/extension-walks.test.mjs• L46-124 tests/extension-walks.test.mjs• L236-280 tests/extension-walks.test.mjs• L446-600 tests/extension-report.test.mjs• L12-38 tests/…/README.md• L1-106 tests/…/deferred.html• L20-25 tests/…/detect-fixture-json-page-level-forms-dark-contents-html.json• L1-7 tests/…/detect-fixture-text-framework-next-modules.json• L1-7
Motion capture checks
Motion capture checks verify both command boundaries and browser evidence. The browser-free oracle cases pin help, usage, and unsafe URL handling. The native fixture suite runs a real capture and checks findings, frame phases, rendered frame files, reduced-motion behavior, and settled completion.
The shared capture harness models playback rate, current position, reverse playback, finished entrance animations, frozen timelines, and sub-millisecond animations. The smooth fixture supplies a clean view transition and checks that reduced motion makes it instant instead of sampling a false transition.
References: tests/motion-capture.test.mjs• L6-80 tests/…/motion.mjs• L1-7 tests/…/smooth.html• L1-4 crates/…/motion.rs• L11-65 crates/…/motion.js• L220-268
Live browser flow
The live end-to-end suite follows a real design session from a staged project to an accepted source change. It proves the browser handshake, page steering, element pick or insertion, agent output, variant cycling, tuning, acceptance, cleanup, and reload behavior.
The default agent is deterministic. Its variants have distinct rendered styles, so the test can prove that the selected variant mounted in the browser instead of trusting only a status label. A language-model agent can use the same interface for selected runs. The fake agent also supplies insert variants, component-preview variants, parameter data, and partial manual-edit results so the browser can verify safe progress.
The browser flow also covers a nested app root, Svelte component previews, state preservation, manual copy staging, element removal, annotations, insert mode, agent-targeted work, discard recovery, and exit. It checks source cleanliness after acceptance and confirms that temporary preview files do not become product source. A focused provider-backed regression also checks that accepted source cleanup reaches durable completion and removes stale browser wrappers. Live-server security checks add the boundary around this flow: in-workspace symlinks work, links outside the workspace are denied, broken links return not found, project ignores and page roots reach the live script, page-controlled poller fields do not reach the agent, and ref-scoped discard preserves sibling edits. Source contracts also check that generation checkpoints follow acknowledged sessions, manual edits carry source and element context, safe discard restores only the intended text or element, framework-owned wrappers yield to reload recovery, and accepted sessions release the foreground UI without losing cleanup.
This sequence shows the main live proof from project setup to clean source.
sequenceDiagram
participant F as Fixture project
participant S as Live server
participant B as Browser
participant A as Agent
F->>S: Start and inject live mode
B->>S: Complete handshake
B->>A: Send steer or generate event
A->>S: Write variants and report progress
S-->>B: Stream state and variants
B->>B: Cycle, tune, edit, and accept
S->>F: Persist accepted source and clean preview state
B->>B: Verify DOM, requests, console, and reload
The browser checks reject missing preview resources, stale wrappers, failed mounts, and console errors that an unchanged original page could hide. Helper tests also protect ignore matching, session persistence, UUID fallback, generation ordering, manual-edit staging, safe discard restoration, removal undo, framework-owned cleanup, and recovery after reload or hot-module replacement. Extra scenarios cover mount retry, republish, storage loss, missed completion events, event ordering, agent targeting, modal top-layer behavior, and accepted-session cleanup.
The session helper stages the fixture, installs its dependencies, starts the live server, injects the runtime, starts the framework server, and returns one teardown path. It supports applications below the repository root so root selection and app selection are tested as part of the user flow. The server reaper protects runs where normal teardown does not happen.
Static-page acceptance covers the case where no hot-module reload or auto-reload can repaint the page after acceptance. It checks that the accepted variant keeps its scoped style and tuned values, that the original and rejected variants stay hidden, and that a second selection remains safe after the first acceptance.
References: crates/…/live_server_security.rs• L1-5 crates/…/live_server_security.rs• L65-170 crates/…/live_server_security.rs• L173-260 tests/live-e2e.test.mjs• L1-19 tests/live-e2e.test.mjs• L78-180 tests/live-e2e.test.mjs• L618-740 tests/live-e2e-accept-cleanup-regression.test.mjs• L47-141 tests/…/agent.mjs• L85-142 tests/…/agent.mjs• L223-237 tests/…/session.mjs• L1-20 tests/…/session.mjs• L287-352 tests/…/ui.mjs• L1-68 tests/live-browser-source.test.mjs• L11-83 tests/live-browser-source.test.mjs• L352-489 tests/live-browser-source.test.mjs• L491-755 tests/live-browser-source.test.mjs• L853-1083 tests/live-browser-source.test.mjs• L1106-1248 tests/live-svelte-adapter-deepseek.test.mjs• L71-154 tests/live-static-accept.test.mjs• L1-147
New-work browser flow
The new-work end-to-end suite checks the browser decision page and its command-line round lifecycle. It uses a real Chromium session, a real engine binary, and deterministic fake image generation so concept choices, re-rolls, follow-up rounds, build-path changes, sketches, composition approval, tab closure, and stalled or dead servers can be checked without language-model calls.
The flow starts a question round from an isolated workspace, observes the page through a scripted user bot, and collects the answer through --wait. A re-roll keeps the round open until a new hand arrives. A terminal choice removes the round state. Heartbeats, deadlines, reloads, server loss, and recorded roots prove that the page does not claim a choice when the round cannot accept it.
This sequence shows how a new-work decision moves between the engine, browser, and workspace.
sequenceDiagram
participant W as Workspace
participant E as Engine
participant P as Browser page
participant U as User bot
W->>E: Start question round
E-->>P: Serve decision page
U->>P: Pick, steer, re-roll, or close
P->>E: Answer or heartbeat
E-->>W: Record answer and round state
E-->>P: Deliver next hand or bounded failure
P->>W: Expose selected direction or approved comp
The same suite checks that synthetic images are repeatable for one prompt, vary for another prompt, and carry a clear synthetic marker. This keeps visual decision tests deterministic while still proving that the page can load and display the expected media forms.
References: tests/new-work-e2e.test.mjs• L1-20 tests/new-work-e2e.test.mjs• L96-134 tests/new-work-e2e.test.mjs• L139-351 tests/new-work-e2e.test.mjs• L587-695 tests/new-work-e2e.test.mjs• L704-805 tests/new-work-e2e.test.mjs• L990-1167 tests/new-work-e2e.test.mjs• L1173-1225
Agent workflow
Provider-backed skill tests check how an agent prepares to work, not how well it invents a page. The harness gives the model a bounded workspace and records whether it loads product and design context, selects the right reference, asks required questions, and writes only after the required decision.
The skill-behavior scenarios cover missing context, existing context, command routing, native platform guidance, update notices, read-only advice, launcher failure, safe fallback behavior, and the difference between a browser variant request and a plain refinement. Routing cases stop at protocol checkpoints. They do not claim that a page was built or reviewed. A separate activation evaluation checks whether the skill description causes a model to load the skill for relevant requests without false activation for unrelated work. The separate skill-workflow suite owns claims about completed builds, visual evidence, handoff, and system documentation. Reference-authoring checks also keep direction contracts, native-platform limits, reduced-motion guidance, content fingerprints, visual decision fallback, card identity, hand sizing, touch checks, and audit score tables aligned with the skill flow. The live reference checks keep the foreground poll loop, manual-edit handoff, provider-specific guidance, capability-based CSS guidance, action vocabulary, and the README's Apply subprocess route aligned with the runtime. The live reference checks also keep setup read-only by default, monorepo guidance target-driven, and manual-edit dispatch ordered after prefetch and before exit.
The workflow scope checks use a valid target change as the starting point. They then prove that protected page content stays intact while target text or styles change, and that unrelated links, text, missing regions, comments, formatting-only edits, unused CSS, or unused custom properties do not count as a valid artifact. This keeps completion evidence tied to the named target rather than to any file change.
The live fake agent follows the same poll, wrap, write, and accept contract as a provider agent. Its stable output gives the browser a reliable proof of rendered variants. The agent contract now also checks exact variant counts, preserved visible copy, material source changes, progressive parameter delivery, valid CSS and HTML shape, safe insert roots, and source edits that preserve typed data and coupled lookup keys. Manual Apply reports applied and failed entries so the workflow can test safe partial progress rather than treating a write as successful only when every edit lands.
Agent-target checks extend this contract to requests that start in the CLI and resolve in a browser overlay. They verify action vocabulary, input validation, authenticated live-server access, one-tab claims, lease recovery, busy and no-match roll calls, reconnect behavior, bounded waits, and timeout results. Late or foreign results cannot create an unannounced session, and the target request is not left pending when its serving browser disappears.
The Hermes plugin uses the same launcher boundary as the other agent integrations. Its tests verify trusted project selection, launcher fallback, write and patch event translation, per-file findings, Stop decisions, disabled hooks, and reuse of the launcher selected during the edit session. Native hook tests extend this boundary with session identity safety, session-scoped completion reminders, shipped-build ownership, baseline attribution, bounded output, and provider-specific Stop decisions.
Provider-backed live runs select OpenAI, Anthropic, or DeepSeek from explicit settings and available keys. They skip when no key is available. The DeepSeek Svelte adapter sweep exercises the full flow with a real provider, including steer, replace, discard, copy Apply, annotations, insert, refresh recovery, accept, cleanup, and exit.
References: tests/…/README.md• L1-29 tests/…/README.md• L38-68 tests/…/activation.eval.mjs• L1-41 tests/…/activation.eval.mjs• L201-318 tests/…/harness.mjs• L89-117 tests/…/harness.mjs• L254-352 tests/…/assertions.mjs• L1-109 tests/skill-reference.test.mjs• L9-202 tests/skill-workflow-scope.test.mjs• L1-49 crates/…/hook_tests.rs• L22-196 crates/…/hook_tests.rs• L219-322 crates/…/hook_tests.rs• L628-988 tests/…/agent.mjs• L57-79 tests/…/agent.mjs• L96-142 tests/…/llm-agent.mjs• L54-98 tests/…/llm-agent.mjs• L100-168 tests/…/llm-agent.mjs• L205-259 tests/live-e2e-llm-agent.test.mjs• L23-125 tests/live-e2e-llm-agent.test.mjs• L1523-1692 tests/live-svelte-adapter-deepseek.test.mjs• L57-154 tests/hermes-plugin.test.mjs• L112-267 tests/live-reference.test.mjs• L9-217 tests/live-agent-target.test.mjs• L1-1176 crates/…/vocabulary.rs• L1-80
CLI and package loading
CLI verification protects the path from an installed command to a usable engine. Launcher checks cover verified download, checksum refusal, cache placement, retry diagnostics, concurrent first runs, interruption cleanup, stale staging, and quiet cached execution on both shell platforms. The npm shim is tested as a separate fallback: it honors a preinstalled binary or platform package, refuses missing or bad checksum records, answers version requests without touching a binary, and leaves no unverified cache artifact.
The npm shim tests force each lookup arm in an isolated staged package. They check the precedence of an explicit binary and an installed platform package before the cache and download paths. Downloaded bytes must match the shipped manifest. A cached binary must also match its digest, size, and verification stamp before it runs. Missing or bad records, stale stamps, host errors, and mismatched payloads fail closed without leaving cache files, while version requests remain local.
Hook-build checks protect the provider manifest contract across Claude, Codex, Cursor, GitHub Copilot, Grok, Gemini, and Antigravity. They verify event shape, launcher path, existence guards, time limits, Windows command forms where supported, plugin-root paths, and routing for unknown providers. Hook commands call the shipped launcher without a Node runtime or an old script probe. Hermes is checked as a plugin payload rather than as a hook manifest. Oracle hook cases then exercise the installed boundary over per-edit scans, Stop events, provider-shaped envelopes, session cache state, duplicate events, linked stylesheet ignore rules, and the VS Code patch-string envelope.
The Windows hook smoke test runs the generated shared command from paths with spaces and non-ASCII characters. It checks both cmd.exe and available PowerShell forms, preserves a launcher failure exit code, and treats a missing launcher as a safe no-op.
Build checks bind shared skill source to provider-specific outputs. They verify skill placement, native agent files, degraded fallbacks, provider placeholders, launcher payload contents, and the absence of engine binaries or bundled detector entry points from the source payload. They also verify that native agent files and degraded references carry the right provider-specific fields while retaining the shared agent body. Transformer checks add a narrow contract for required metadata, allowlisted optional fields, user-invocable hints, copied references, and provider markers without rewriting executable code.
The build integration checks now follow agent output across native and degraded provider surfaces. They verify that shared bodies resolve placeholders and links everywhere they ship, that provider-only blocks compile correctly, that tool restrictions remain meaningful for each host, and that Windows launcher files keep the required line endings and version alignment. OpenAI plugin staging has its own boundary: it packages the Codex skill, agent files, metadata, and icon into the ChatGPT plugin shape while excluding Claude content and bundled hooks. The release step also keeps the plugin upload visible when a skill release completes.
Cursor and VS Code checks add package-level boundaries. Cursor staging rewrites paths for a relocated plugin and preserves the executable launcher, hooks, links, and version agreement. VS Code packages the skill as a chat skill without an editor runtime or project-install sidecar. Both checks run the relocated launcher from a path with spaces and confirm that it keeps the user's project as its working directory. Installer checks also cover detected-only behavior and interactive provider choice: an explicitly selected harness is installed even when another harness already exists, while a detected but unselected harness stays unchanged. Scope checks protect home-rooted installs, provider agent freshness, relocated bundle staging, and global hook paths.
References: crates/…/drift_ports_tests.rs• L1-15 crates/…/drift_ports_tests.rs• L167-355 crates/…/drift_ports_tests.rs• L450-631 crates/…/install_picker_tests.rs• L1-129 crates/…/hook_manifest_tests.rs• L48-170 crates/…/hook_manifest_tests.rs• L464-510 tests/launcher-download.test.mjs• L22-177 tests/launcher-download.test.mjs• L201-377 tests/launcher-download.test.mjs• L466-534 tests/cli-shim.test.mjs• L1-27 tests/cli-shim.test.mjs• L152-316 tests/check-engine-release.test.mjs• L1-159 tests/hook-build.test.mjs• L33-50 tests/hook-build.test.mjs• L72-347 tests/hook-build.test.mjs• L350-507 tests/hook-build-windows.test.mjs• L1-44 tests/openai-plugin.test.mjs• L15-103 tests/build.test.js• L128-256 tests/build.test.js• L381-471 tests/build.test.js• L474-781 tests/…/provider-blocks.test.js• L5-99 tests/…/providers.test.js• L18-130 tests/…/factory.test.js• L30-203 tests/cursor-plugin.test.mjs• L37-119 tests/plugin-paths.test.js• L32-123 tests/vscode-extension.test.mjs• L27-89 tests/…/skills.mjs• L23-33
Release and artifact validation
Release checks run publishing guards in a disposable Git repository with a local bare origin. Dry runs exercise version, changelog, artifact, tag, branch, engine-source, signing, component-manifest, and release-key checks without tagging, pushing, publishing, or building a real release. The release script tests cover engine, skill, CLI, and extension paths, including release-note conversion and the pre-tag refusal boundary.
Engine release checks use a local server with fake platform assets, sidecars, and npm packages. They prove that the checksum manifest records every served asset, that a changed or missing line is reported, and that the fetch path refuses malformed or mismatched data before it writes a binary.
Loop release checks apply the same fail-closed rule to versioned loop archives. They validate semantic-version pins, require HTTPS release locations except for controlled local test bases, follow only permitted signed redirects, compare fetched bytes with the recorded digest, and refuse to overwrite an existing pin with different bytes.
Other checks bind release artifacts together. Signing tests verify that the exact bundle, version, size, digest, artifact name, and trusted key are covered by the signature. Platform package tests verify each target package and its binary checksum. Workflow tests verify action pinning, read-only CI access, protected signing, same-run artifact downloads, signing order, publication dependencies, and the absence of unsigned assets from the published set. The release workflow test also checks that publication uses one release creation step after signed assets are available. The engine release check retries a transient asset failure, so a short service outage does not look like a missing release asset. The release guard suite also scans compiled source inclusions so a file embedded by the engine cannot escape the engine-source change check.
References: tests/release.test.mjs• L1-12 tests/release.test.mjs• L46-68 tests/release.test.mjs• L70-121 tests/release.test.mjs• L146-237 tests/release.test.mjs• L294-368 tests/release.test.mjs• L388-488 tests/check-engine-release.test.mjs• L20-159 tests/check-loop-release.test.mjs• L1-58 tests/bundle-signing.test.mjs• L12-85 tests/publish-platform-packages.test.mjs• L24-113 tests/release-engine-workflow.test.js• L7-79 tests/workflow-security.test.js• L9-37
Repository maintenance checks
The core suite also verifies the GitHub pull-request sheriff. This check turns pull-request state into a safe maintenance plan: it distinguishes contributor work from maintainer decisions, keeps blocker labels current, warns after a waiting period, and closes only after a trusted warning and the configured close window. The test uses fixed dates and synthetic GitHub data, so time-based policy stays repeatable.
The checks cover review threads, requested changes, merge conflicts, failing checks, policy labels, draft transitions, contributor responses, trusted warning markers, regular-contributor exemptions, and explicit maintainer handoffs. A command-level case also proves that the sheriff can hydrate old comments and label events beyond the initial API window before it evaluates a pull request. This protects the difference between incomplete history and a real absence of a warning or blocker.
References: tests/github-sheriff.test.mjs• L21-186 tests/github-sheriff.test.mjs• L204-354 tests/github-sheriff.test.mjs• L356-498 tests/github-sheriff.test.mjs• L500-562 tests/github-sheriff.test.mjs• L564-883 scripts/…/sheriff.mjs• L147-318 scripts/…/sheriff.mjs• L321-407 scripts/…/sheriff.mjs• L605-704
Change checklist
When these files or sections are changed, remember to consider these:
- Add every new test file to a suite whose trigger selects the file and whose command runs it.
- When an engine command changes, update its oracle cases and reviewed deltas with the binary behavior.
- When CLI control commands change, keep their usage errors, signed-out state, persisted settings, and no-network behavior aligned with oracle cases and golden files.
- When detector rules, scopes, extensions, or output formats change, update fixture flag/pass pairs and the paired JSON and text records.
- When detector advisory or ignore behavior changes, update direct scans, linked-file scans, hook results, and aggregate sweeps together.
- When a new fixture has linked styles, framework syntax, or browser-only assumptions, keep its source files, directory sweeps, and both output modes aligned.
- When framework fixture behavior changes, keep the fixture manifest, source shape, install contract, runtime probes, and expected accepted source aligned.
- When broken-image source parsing changes, keep HTML, framework bindings,
srcset, and JavaScript or TypeScript literal cases distinct in both native and oracle coverage. - When a directory scan extension changes, recheck direct-file parity, regex-only file counts, scope scans, and accepted sweep deltas.
- When a consent or overlay fixture hides site state, keep hidden-page scoring, true wall or gate classification, evidence, and restored page state aligned.
- When extension bundle inputs change, check the rule registry, generated output, package paths, browser startup, account and native-agent boundaries, walk hand-off rules, popup state, shared copy formatting, DevTools behavior, and browser-specific hosts as one boundary.
- When popup or report state changes, keep active-tab identity, scan completion, navigation invalidation, clipboard status, advisory grouping, and URL or selector formatting aligned across popup and DevTools.
- When account sign-in or persona flows change, keep callback checks, issuer selection, token persistence, worker recovery, sender checks, server switching, and side-panel messages aligned.
- When walk protocol or browser-agent behavior changes, keep hand-off validation, socket pairing, permissions, sign-in pauses, takeover, cleanup, and store or Firefox staging aligned.
- When deferred extension checks change, keep the fixture's viewport position, image source, worker-restart path, current-document recovery, and cancellation behavior aligned.
- When motion capture changes, keep the browser timing harness, motion fixtures, oracle command cases, report frame phases, reduced-motion output, and native findings aligned.
- When live event, variant, parameter, manual-edit, targeting, modal, or cleanup shapes change, update the fake agent, session helper, browser assertions, and source-side checks together.
- When agent-target requests change, keep CLI validation, live-server vocabulary, overlay claims, leases, roll calls, timeout verdicts, and generate-event fencing aligned.
- When live-server routes or event ingestion changes, check workspace symlink confinement, project-ignore serialization, poller field filtering, and ref-scoped manual discard together.
- When live locator or source-wrap behavior changes, keep text disambiguation, later-file search, dynamic text fallback, generated-file refusal, and HTML or component fixture coverage aligned.
- When static-page acceptance changes, keep immediate wrapper removal, scoped styles, tuned values, hidden variant state, and the next-session interaction aligned.
- When question state can move between a caller directory and a recorded project root, update start, wait, update, stop, heartbeat, cleanup, and the no-state diagnostic as one contract.
- When a hook builder or provider path changes, check generated manifests, launcher guards, platform variants, version alignment, Hermes packaging, OpenAI plugin staging, and real loader discovery.
- When Windows hook commands change, run the spaced-path shell checks and preserve both launcher failure propagation and missing-launcher no-op behavior.
- When hook session ownership or baseline attribution changes, keep per-edit, Stop, continuation, provider envelope, cache, and output-budget cases aligned.
- When a launcher or npm shim download path changes, preserve lookup precedence, per-run staging, checksum refusal, cache cleanup, retry diagnostics, and quiet cached execution on both shell platforms.
- When provider-key loading changes, preserve environment-over-file precedence, private file checks, child-environment filtering, and provider skip behavior.
- When skill text or its loading plan changes, keep file caps, use caps, and CLI context and hook output caps aligned with scripts/token-budget.json.
- When surface-brief target rules change, check reads and writes from repository roots and workspace child directories without confusing a real nested directory with a workspace prefix.
- When interactive install selection changes, check explicit picks, mixed installed and missing providers, detected-only behavior, scope, and provider-specific bridge files.
- When skill or agent source changes, verify every emitted provider surface, degraded fallback, placeholder replacement, relative link, and protected-content workflow check.
- When engine release assets or checksum manifests change, check every target asset, sidecar, npm package pin, loop archive pin, and fetch failure path together.
- When release guards or release-source inclusion rules change, keep component manifests, changelog labels, signing sidecars, tag state, engine tags, embedded inputs, and dry-run refusal order aligned.
- When a comparison report changes, keep region alignment, per-region offsets, scores, verdicts, and no-spec output consistent.
- When a release input or workflow changes, check version sources, checksums, signing order, package publication, action pins, artifact permissions, and token scope together.
- When pull-request policy changes, keep blocker classification, maintainer handoffs, trusted warning and close markers, label events, hydrated history, and dry-run or scheduled thresholds aligned.
Important failure modes
| Trigger | Consequence | Guard |
|---|---|---|
| A test file is not in the suite registry | Changed-path CI can omit the new test | Registry tests compare discovered files, suite membership, and own-file triggers |
| An oracle golden changes without a reviewed delta | A behavior change can enter the engine unnoticed | Oracle replay fails on missing or unreviewed differences |
| A CLI control command changes its default or signed-out state without new records | Automation can misread usage, consent, or persisted settings | Oracle cases compare stdout, stderr, exit status, and isolated state files |
| Detector behavior changes without matching JSON and text records | CLI output can drift between output modes | Detector oracle records compare fixture, directory, scope, and format results |
| Source image parsing treats a binding or code literal as markup | Framework code can report false broken-image findings or miss real empty sources | Native HTML and source tests compare valid srcset and bindings with empty and literal cases |
| A directory walker omits a supported module file | A directory scan can look clean while a direct file scan reports findings | Oracle framework records compare file counts and findings for the .mjs and .cjs cases |
| A consent or overlay fixture hides site state | A scan can change the page or misclassify a real gate | Browser fixtures check opt-out scans, wall and gate refusal, evidence, and restoration |
| The extension host fails or its readiness response is ignored | A page can appear clean when no scan ran | Startup tests require one shared host, readiness before delivery, retry after failure, and visible scan errors |
| A popup accepts a partial, old-tab, or pre-navigation report | Users can copy incomplete findings or findings from another page | Popup tests gate copy on scan completion and active URL, then invalidate state on navigation and failure |
| An account callback accepts the wrong state, issuer, or callback host | A token exchange can attach to the wrong sign-in or server | Account tests reject repeated or mismatched callback fields and unpaired metadata |
| An extension message or walk hand-off comes from an untrusted sender | A page or foreign extension can start work or reach account actions | Account, agent, and walk tests require the extension origin, approved page, and paired socket |
| A DevTools advisory is treated as a repair command | The panel can prescribe an unsafe or unwanted change | Panel and shared-report tests keep advisory presentation separate from mapped repair suggestions |
| A motion capture accepts an unsupported timing or leaves an animation running | Report frames can describe incomplete or unbounded motion | Motion tests bound wall duration and require all animations to settle |
| Reduced-motion capture samples a sub-millisecond transition | A clean reduced-motion page can produce false motion evidence | Capture tests treat sub-millisecond motion as instant and check zero duration |
| A live preview request fails while the original page still renders | The test can report success without showing the generated variant | Browser checks reject live-tree request failures and verify rendered style |
| A static page loses its accepted style after the first session | Users can see the wrong source state or a later session can affect accepted content | Static-page browser checks inspect immediate visibility, scoped style, tuned values, and a second interaction |
| A live-server source path follows a symlink outside the workspace | A live session can expose a file that the project does not own | Security checks allow in-workspace targets and reject outside or broken links |
| A test process leaves a live server running | Later suites can attach to a stale port or session | Run markers, the reaper, and the runner's leak check kill and report survivors |
| A question round loses its recorded root after the caller changes directory | Wait or stop can report a dead round or act on the wrong project | Oracle cases check root pointers, missing-state diagnostics, scoped stop, and answer collection |
| A routing test is treated as a completed workflow | An agent can pass context loading without building or reviewing | Skill behavior stops at protocol checkpoints and the workflow suite owns completion claims |
| A protected page change is judged only by any file difference | An agent can claim a valid artifact while changing unrelated content | Scope checks require the named target to change and protected content to remain intact |
| An activation description fires for unrelated requests | The agent loads the wrong skill and changes task routing | Activation evaluation requires zero false activations and model-specific recall floors |
| A provider key file is readable by other users or leaks into a child | Credentials are exposed or a fixture process gains unrelated provider access | Key tests require private files, preserve environment precedence, and strip credential names from child environments |
| An explicitly selected installer provider is treated as already satisfied by another provider | A requested harness is not installed even though install reports no work | Picker tests distinguish named selection from detected-only selection |
| A Hermes project is untrusted or its launcher is missing | The plugin can run in the wrong project or silently skip design checks | Plugin tests require trusted-root selection, home fallback, and no-op behavior without a launcher |
| A hook manifest names a missing launcher or retains a legacy script path | The provider silently skips the hook or runs an obsolete entry point | Hook builders and manifest tests require the current launcher and preserve unrelated settings |
| A provider hook uses the wrong platform form or install root | Hooks work for one host or scope and silently no-op elsewhere | Build, manifest, and shell-runner tests check launcher paths, guards, Windows forms, and provider routing |
| A Windows hook path contains spaces or a launcher returns failure | The hook can fail to start or hide a blocking result | Windows shell tests run spaced paths and require nonzero launcher status to remain nonzero |
| An OpenAI plugin includes hooks or reads the Claude payload | Plugin review can reject the package or load the wrong skill | OpenAI staging tests require Codex-only content, ChatGPT metadata, and no hook directory |
| An agent-target result comes from a non-holder or arrives after timeout | The wrong tab can answer a request or start a session the agent never saw | Live target tests enforce leases, target ownership, request fencing, and delivered status |
| A launcher or npm shim uses an unverified binary | An untrusted engine can enter the cache or run as a valid install | Launcher and shim tests require a manifest digest, size-aware stamp, clean refusal, and lookup precedence |
| A download sidecar or checksum line is missing or mismatched | An unverified engine can enter the cache or release package | Launcher, shim, fetch, and engine-release tests refuse the payload and leave no cache artifact |
| A loop archive pin is overwritten by different bytes or an insecure redirect | Release verification can trust the wrong archive or weaken transport security | Loop release tests validate semver, HTTPS, signed redirects, digest equality, and no-overwrite behavior |
| A transient release service failure is treated as a missing asset | A complete release can fail verification during a short CDN outage | Engine release checks retry failed asset requests before reporting a missing asset |
| A release guard misses an embedded engine input | Engine source can change after the engine tag without blocking a dependent release | Release tests scan compiled source inclusions and compare them with guarded paths |
| A relocated package keeps a project-relative launcher path | An installed plugin can run an old project copy or fail to start | Cursor, plugin-path, and VS Code tests rewrite paths and run the launcher from spaces |
| Version sources, release notes, or artifacts disagree | A valid-looking tag can publish incomplete or stale content | Disposable release tests and pre-release guards refuse the run |
| A signing job can publish unsigned output or use broad token access | A release can bypass its trust boundary | Engine workflow tests require protected signing, ordered verification, and scoped permissions |
| A workflow action is not pinned or CI gets write access | A build can run untrusted action code or modify repository content | Workflow security tests require full commit pins and scoped permissions |
| A skill file exceeds its reviewed cap or a use loads too much context | An agent can receive an oversized or unreviewed prompt | Token budget tests check every file, common use, and real CLI output cap |
| A provider response changes copy, omits an edit, or corrupts typed source data | Live Apply can save incomplete or unsafe source changes | Agent validation requires exact coverage, source evidence, coupled-key checks, and safe partial results |
| A framework-owned wrapper is changed during recovery | Hot reload can lose app state or leave stale live markup | Browser source contracts defer structural cleanup, fence old sessions, and use reload recovery |
| A live locator picks a style-only or dynamic-text match as source content | Wrap or insert can edit the wrong file or hide an agent fallback | Oracle cases require source candidates, text disambiguation, and explicit ambiguity errors |
| A component-diff region uses only the page-wide shift | A moved page can make a stationary region look missing | Golden reports record the better per-region offset and preserve the region score |
| Pull-request history is evaluated before old comments or label events are hydrated | The sheriff can warn or close on incomplete evidence | Sheriff tests exercise full-history hydration and preserve trusted markers and handoff events |
| A stale marker from an untrusted commenter is accepted | The sheriff can close a pull request without its required warning | Sheriff tests require trusted authors for warning and close markers |
| A maintainer decision is treated as contributor work | A pull request can age or close while waiting for a maintainer | Sheriff checks keep policy decisions visible while suppressing contributor aging until responsibility returns |
Key files
| File | Why to read it |
|---|---|
| scripts/test-suites.mjs | Defines suite purpose, membership, triggers, and opt-in policy. |
| scripts/run-tests.mjs | Shows command expansion, process-group limits, timeouts, and live-server leak checks. |
| scripts/ci-test-plan.mjs | Shows how events and changed paths select CI jobs. |
| .github/…/ci.yml | Shows the test matrix, tool setup, and opt-in job gates. |
| tests/test-suites.test.mjs | Protects suite registration and change-selection coverage. |
| tests/…/README.md | Explains recorded command behavior, fixtures, normalization, and reviewed deltas. |
| tests/…/lib.mjs | Defines staged oracle cases and observable result comparison. |
| tests/…/motion.mjs | Shows browser-free motion command boundary cases. |
| tests/motion-capture.test.mjs | Shows the isolated animation timing contract. |
| tests/…/smooth.html | Shows the clean transition and reduced-motion fixture. |
| crates/…/motion.rs | Shows real capture, report, frame, and reduced-motion verification. |
| tests/…/critiques.mjs | Shows isolated signed-out preference state and multi-step CLI verification. |
| tests/…/health-design.mjs | Shows DESIGN.md sync and refresh command cases. |
| tests/…/setup.mjs | Shows local setup status and usage-error cases. |
| tests/…/context.mjs | Shows context, monorepo target, surface-brief, question, and deterministic workspace cases. |
| tests/…/hooks.mjs | Shows provider-shaped hook events, session flows, configuration, duplicate-event behavior, and Hermes coverage. |
| tests/…/live-wrap.mjs | Shows source locator, text disambiguation, wrap, insert, and generated-file behavior across project shapes. |
| tests/…/critiques-on-signed-out.json | Fixes persisted critique-sync state and signed-out user messaging. |
| tests/…/health-design-on.json | Fixes DESIGN.md sync state and guidance. |
| tests/…/setup-status-default.json | Fixes default setup-status output. |
| tests/…/hook-harness-vscode-patch-string.json | Fixes the VS Code patch-string hook response shape. |
| tests/…/live-wrap-text-picks-later-file.json | Fixes the successful later-file source selection and live wrapper contract. |
| tests/…/live-insert-text-in-no-file.json | Fixes the ambiguous dynamic-text fallback and error shape. |
| tests/…/comp-diff-json.json | Fixes structured component-diff output, including region offsets. |
| tests/…/DELTAS.md | Records reviewed changes to command output and engine behavior. |
| tests/framework-fixtures.test.mjs | Shows the static fixture contract for live injection, CSP, and source wrapping. |
| tests/…/README.md | Explains runtime fixture configuration, nested app roots, and browser scenarios. |
| tests/…/fixture.json | Shows the Vue runtime fixture's source target, variant path, and acceptance checks. |
| tests/…/dev-keys.mjs | Defines private local credential lookup and child-environment filtering. |
| tests/…/dev-keys.test.mjs | Protects credential parsing, precedence, file privacy, and child isolation. |
| tests/…/engine-bin.mjs | Defines the shared engine lookup and test environment. |
| crates/…/broken_image_source.rs | Shows source-level broken-image parsing across markup, bindings, and code literals. |
| crates/…/broken_image.rs | Shows HTML-level image source and binding rules. |
| crates/…/live_server_security.rs | Shows live-server path, event, ignore, and manual-discard security boundaries. |
| crates/…/hook_tests.rs | Shows native hook session, provider, baseline, and completion contracts. |
| tests/extension-build.test.mjs | Shows cross-browser extension startup, scan lifecycle, cancellation, recovery, and deferred checks. |
| tests/extension-account.test.mjs | Shows extension sign-in, account session recovery, journey notices, and persona access contracts. |
| tests/extension-agent.test.mjs | Shows native-agent admission, site consent, secret protection, and session cleanup. |
| tests/extension-walks.test.mjs | Shows browser walk hand-off, permissions, sign-in safety, protocol, and package variants. |
| tests/extension-popup.test.mjs | Shows active-tab popup and side-panel state, scan completion, navigation, copy, and retry behavior. |
| tests/extension-report.test.mjs | Shows the shared report format and advisory grouping used by extension copy surfaces. |
| tests/extension-devtools.test.mjs | Shows DevTools finding presentation and copy behavior. |
| tests/…/README.md | Describes manual Chrome and Firefox smoke coverage for clean, flagged, CSP, and worker-restart pages. |
| tests/live-e2e.test.mjs | Shows the user-visible live flow and its source, DOM, network, and recovery proofs. |
| tests/live-static-accept.test.mjs | Shows acceptance behavior on a static page without HMR or auto-reload. |
| tests/live-e2e-accept-cleanup-regression.test.mjs | Shows the focused provider-backed durable accept and stale-wrapper guard. |
| tests/new-work-e2e.test.mjs | Shows browser decision rounds, bounded question failures, and deterministic image checks. |
| tests/…/agent.mjs | Shows the deterministic agent contract used to verify generated variants and edits. |
| tests/…/llm-agent.mjs | Shows provider selection and validation of model-produced variants and source edits. |
| tests/…/session.mjs | Shows fixture staging, server boot, browser setup, and teardown. |
| tests/live-browser-source.test.mjs | Shows source-level contracts for generation, manual edit staging, agent targets, wrapper recovery, and accepted-session cleanup. |
| tests/live-reference.test.mjs | Shows authoring contracts for live routing, polling, provider blocks, CSS guidance, action vocabulary, and manual-edit handoff. |
| tests/live-agent-target.test.mjs | Shows CLI-to-browser targeting, overlay claims, lease recovery, bounded waits, and late-result fencing. |
| tests/live-svelte-adapter-deepseek.test.mjs | Shows the full provider-backed Svelte live workflow and source cleanliness checks. |
| tests/hermes-plugin.test.mjs | Shows Hermes hook registration, trust boundaries, event translation, and Stop decisions. |
| tests/build.test.js | Shows cross-provider skill, agent, fallback, launcher-payload, placeholder, and line-ending generation contracts. |
| tests/…/factory.test.js | Shows provider output metadata, references, placeholders, and script marker contracts. |
| tests/…/activation.eval.mjs | Shows model-based skill activation scoring and false-activation gates. |
| tests/…/harness.mjs | Shows staged skill behavior workspaces, bounded tools, trace capture, and turn limits. |
| tests/…/providers.mjs | Shows provider selection, key lookup, model defaults, and skip behavior. |
| tests/skill-reference.test.mjs | Shows authoring contracts that keep reference flows aligned with engine behavior. |
| tests/skill-workflow-scope.test.mjs | Shows target-change and protected-content checks for workflow evidence. |
| tests/token-budget.test.mjs | Shows skill-file, use, and real CLI output budget guarantees. |
| scripts/…/token-budget.js | Defines file and use budget enforcement. |
| scripts/token-budget.json | Records the reviewed file, use, and CLI output caps. |
| tests/hook-build.test.mjs | Shows provider hook manifest shapes, launcher guards, platform forms, and Hermes packaging. |
| tests/openai-plugin.test.mjs | Shows OpenAI plugin staging, Codex-only content, metadata, icon, and release handoff checks. |
| tests/hook-build-windows.test.mjs | Shows Windows shell execution, spaced plugin paths, failure propagation, and missing-launcher behavior. |
| crates/…/hook_manifest_tests.rs | Shows hook merge, migration, scope, and provider-path safety contracts. |
| crates/…/install_picker_tests.rs | Shows explicit provider selection and detected-only install behavior. |
| crates/…/drift_ports_tests.rs | Shows cross-platform bundle staging, agent freshness, and install/update scope parity. |
| tests/launcher-download.test.mjs | Shows verified launcher download, cache, failure, cleanup, and retry behavior. |
| tests/cli-shim.test.mjs | Shows npm fallback lookup order, version handling, checksum-protected cache behavior, and host diagnostics. |
| tests/check-engine-release.test.mjs | Shows release-sidecar, checksum-manifest, and transient-download verification for every engine target. |
| tests/check-loop-release.test.mjs | Shows loop archive pin validation, secure redirects, digest checks, and no-overwrite behavior. |
| tests/github-sheriff.test.mjs | Shows deterministic pull-request policy, history hydration, trusted markers, and maintainer handoffs. |
| scripts/…/sheriff.mjs | Defines pull-request blocker, label, warning, and close decisions. |
| tests/cursor-plugin.test.mjs | Shows relocated Cursor plugin staging, path rewriting, hooks, links, and version checks. |
| tests/plugin-paths.test.js | Shows plugin path rewrite and safety checks. |
| tests/vscode-extension.test.mjs | Shows the VS Code chat-skill package boundary and relocated launcher behavior. |
| tests/release.test.mjs | Shows release refusal, source-inclusion, component, and dry-run checks in a disposable Git setup. |
| tests/release-engine-workflow.test.js | Shows protected engine signing, same-run artifact transfer, and publication order. |
| tests/bundle-signing.test.mjs | Shows the signed bundle and trusted-key contract. |
| tests/workflow-security.test.js | Shows action pinning and the permission boundary between CI and publishing workflows. |