AI provider integration
AI provider integration gives the design skill one provider-neutral workflow while adapting its commands, agent assets, model calls, and image tools to each supported agent client. It separates build-time translation of the shipped skill from local workflow scripts and test-only model adapters, so a provider change does not require rewriting the design process.
Provider-neutral skill delivery
The provider build turns one source skill into client-specific skill packages. A provider configuration names the client, its install directory, the markdown blocks it accepts, and any client-specific agent or hook surface. The transformer emits the package with those choices applied (scripts/…/providers.js• L1-10, scripts/…/factory.js• L233-316).
flowchart LR
Source[Shared skill source] --> Config[Provider configuration]
Config --> Transform[Provider transformer]
Transform --> Markdown[Translated skill and references]
Transform --> Scripts[Scripts with provider identity]
Transform --> Agents[Native or degraded agents]
Transform --> Hooks[Provider hook manifest]
Markdown --> Client[Agent client]
Scripts --> Client
Agents --> Client
Hooks --> Client
The source stays portable through named placeholders and explicit provider blocks. Markdown keeps only blocks whose tags match the target. Script source is not passed through broad text replacement. A small provider module carries the command prefix and provider identity through exact marker replacement (scripts/…/utils.js• L579-603, scripts/…/utils.js• L643-703, skill/…/provider.mjs• L1-5).
Provider selection model
Provider selection has two related meanings. The build selects an agent client such as Claude Code, Codex, Cursor, or GitHub Copilot. The test and live-agent helpers select a model service such as Anthropic, OpenAI, Google, or DeepSeek from an explicit setting, model name, or available key (tests/…/providers.mjs• L43-87, tests/…/llm-agent.mjs• L205-249).
| Selection need | Main input | Result |
|---|---|---|
| Skill delivery | Provider build configuration | Client-specific files, paths, syntax, and optional hooks |
| Behavior test model | Model ID or model-list setting | SDK model with matching credentials and options |
| Live editing agent | Explicit provider or available key | LLM client for variants, steering, and manual edits |
Model helpers fail early for unknown model families and skip a test when its credential is absent. This keeps provider coverage useful without making an unrelated provider key a requirement for every run (tests/…/providers.mjs• L50-62, tests/…/workflow-contract.test.mjs• L101-110).
References: scripts/…/providers.js• L12-179 scripts/…/factory.js• L233-316 tests/…/providers.mjs• L43-87 tests/…/llm-agent.mjs• L205-249
Provider-specific assets
Provider assets preserve the same agent role across clients while using each client's native file shape. The build can emit native agent files for clients that support them, degraded reference files for clients without subagents, nested Codex agents inside a skill, and provider hook manifests where the client has a supported hook surface (scripts/…/factory.js• L337-408).
Each packaged client also has a manifest for its install surface. The Claude manifest identifies the impeccable package at version 4.10.0 and points the client to the packaged skills directory. The Grok manifest uses the same package identity and version while adding a license and discovery keywords for its client catalog (plugin/…/plugin.json• L1-12, plugin/…/plugin.json• L1-20).
The provider list is a contract shared by build output and download services. Install and download paths must use the same provider-to-directory mapping. This includes aliases that share a client directory (scripts/…/providers.js• L12-179, cli/…/download-providers.js• L1-34).
| Asset kind | Purpose |
|---|---|
| Translated SKILL.md and references | Give each client the same workflow in its accepted syntax |
| Native agent files | Let a client delegate a role using its own agent format |
| Degraded references | Let a client without subagents run the role inline |
| Provider hook manifest | Connect supported client events to local design checks |
| Script provider marker | Let runtime scripts report the active client without unsafe source rewriting |
| Client plugin manifest | Describe package identity and the client-facing skill surface |
References: scripts/…/factory.js• L179-224 scripts/…/factory.js• L337-408 scripts/…/providers.js• L45-82 cli/…/download-providers.js• L6-34 plugin/…/plugin.json• L1-12 plugin/…/plugin.json• L1-20
Agent-facing model invocation
Agent-facing invocation adapts several model services to the same workflow contract. The live editing agent exposes operations for generating variants, handling a steer request, and applying manual edits. Its provider layer hides whether the request uses Anthropic directly or an OpenAI-compatible adapter (tests/…/llm-agent.mjs• L251-324).
sequenceDiagram
participant Workflow as Live workflow
participant Resolver as Provider resolver
participant Client as Model client
participant Guard as Response checks
participant Source as App source
Workflow->>Resolver: Select provider and model
Resolver->>Client: Create provider client
Workflow->>Client: Send role instructions and target context
Client-->>Workflow: Structured model response
Workflow->>Guard: Validate variants or edit plan
Guard->>Source: Apply accepted change
Source-->>Workflow: Updated source and browser state
The adapter normalizes system text, messages, output limits, timeouts, and usage data. Provider-specific controls remain at the edge. Examples include OpenAI reasoning effort and an Anthropic-compatible DeepSeek endpoint. The workflow therefore validates one response shape instead of branching on each service (tests/…/llm-agent.mjs• L251-289, tests/…/llm-agent.mjs• L296-324).
This model layer belongs to the live-agent test and end-to-end support path. It does not replace the installed skill's local scripts. The skill scripts manage context, questions, image assets, and workflow state. They do not select a text model for the agent thread.
References: tests/…/llm-agent.mjs• L251-324 tests/…/llm-agent.mjs• L292-306
Image generation
The image generation utility turns a prompt into a local image for the design workflow. It uses an OpenAI-compatible image endpoint when an API key is available and can also use a synthetic mode for local or test runs without a network call (crates/…/generate_image.rs• L320-404).
Generation without references sends a normal image-generation request. Generation with local or remote references sends an image-edit request so the model can use those images as input. The utility records the selected model, prompt, size, quality, background choice, and references with the output (crates/…/generate_image.rs• L405-487, crates/…/generate_image.rs• L512-547).
Remote references are treated as image inputs only after download checks pass. The utility follows a bounded number of redirects, limits the response size, and checks the bytes for a supported image signature instead of trusting a URL suffix or response header (crates/…/generate_image.rs• L248-310, crates/…/generate_image.rs• L431-465).
Image output provenance
Image output provenance keeps the generation intent beside the asset. The utility tries to embed the prompt in the image and writes a JSON sidecar with the prompt and generation details. The shared prompt utility also lets later workflow steps read embedded prompt data or find raster assets that have no recorded prompt (crates/…/generate_image.rs• L504-547, skill/…/embed-prompt.mjs• L1-16, skill/…/embed-prompt.mjs• L28-65).
Metadata can be removed by image conversion or optimization. The sidecar and the original asset therefore remain the useful record when embedded data is not preserved (skill/…/embed-prompt.mjs• L14-16, skill/…/visualize.md• L46-50).
References: crates/…/generate_image.rs• L320-404 crates/…/generate_image.rs• L405-487 crates/…/generate_image.rs• L504-547 skill/…/embed-prompt.mjs• L21-65 skill/…/embed-prompt.mjs• L123-175 skill/…/visualize.md• L46-50
Local workflow boundary
Local workflow scripts are provider-aware only where the installed client changes command syntax or runtime identity. They own files in the target project and coordinate browser questions, context, image assets, and live sessions. Agent-facing model adapters and provider build transformers sit outside that runtime boundary. One calls models for test or live-agent behavior. The other prepares assets for installation (skill/…/provider.mjs• L1-5, scripts/…/factory.js• L251-365).
The boundary matters during changes. A new agent client normally needs a provider configuration and translated assets. A new model service needs resolver and adapter coverage in the relevant test or live-agent helper. A new local workflow capability needs a skill script and its reference guidance. These paths may share provider names, but they do not share ownership of the same contract. See agent design workflows, live editing sessions, agent hook workflow, design context and critique, and skill commands and references for the related workflow boundaries.
Local question and live workflow interfaces
The visual question server is a local decision interface. It serves local images without uploading them and returns a structured answer on stdout or through a detached wait cycle. Its image slots can wait for a generated comp to land after the page is served (skill/…/serve-question.mjs• L1-16, skill/…/serve-question.mjs• L78-105, skill/…/serve-question.mjs• L407-458).
Live generation preflight is another local boundary. It prepares a source scaffold for an agent without writing a premature source change. It resolves the target asynchronously and caches only the source-file discovery. The live server remains responsive while the agent receives current context (skill/…/generation-preflight.mjs• L33-59, skill/…/generation-preflight.mjs• L62-115).
References: skill/…/serve-question.mjs• L78-105 skill/…/serve-question.mjs• L407-458 skill/…/generation-preflight.mjs• L33-59 skill/…/generation-preflight.mjs• L62-115
Change checklist
When these files or sections are changed, remember to consider these:
- Keep
PROVIDERS, download-provider directories, installer aliases, and provider package manifests aligned. A missing entry can build a package that the download or install path cannot find. - When adding a provider block or placeholder, update the known tag and replacement contract together. Otherwise source text can silently remain in a generated package.
- When changing provider agent formats or names, check native agent output, degraded references, and any provider allow-list in the source agent metadata.
- When changing model selection, keep credential detection, model-family detection, endpoint setup, and provider-specific options consistent.
- When changing image generation inputs, keep the generation and edit request shapes aligned with the recorded sidecar fields.
- When changing remote image references, preserve byte-type checks and the rule that a failed reference produces no generation request.
- When changing question payload image paths, preserve the serve-first behavior so a slow render does not block the user's decision page.
- When changing a client plugin manifest, keep its package version and declared skill or hook metadata aligned with the generated package.
Important failure modes
| Trigger | Consequence | Guard |
|---|---|---|
| A model ID has no recognized family | Invocation fails instead of using the wrong service | Keep model-family detection explicit and add the matching provider mapping |
| A required provider key is missing | The related behavior test skips or image generation exits without a result | Check credentials before client creation and prefer a native image tool when available |
| A provider block tag is not in the known tag set | Provider-specific guidance can remain in source or compile away incorrectly | Validate every configured provider tag against the compiler's known tags |
| A remote reference returns HTML or an unsupported image | The edit request is not sent and no output is written | Check image bytes, response size, status, and redirect limits before upload |
| A prompt is lost during image conversion | A later engineer cannot recover the asset's generation intent | Keep embedded metadata or the JSON sidecar and scan shipped raster directories |
| Image generation runs before the decision page is served | The user waits on generation and may see no live comp slot | Serve the question first, then fill declared image slots |
| A client-specific script is processed with broad placeholder replacement | JavaScript paths or regular expressions can be changed accidentally | Replace only the explicit provider markers in the provider module |
| A provider package manifest does not match its shipped skill or hook surface | The client can install a package with incomplete or misleading capabilities | Keep manifest metadata aligned with the packaged skill and hooks |
Key files
| File | Why to read it |
|---|---|
| scripts/…/providers.js | Defines the supported agent clients and their output contracts. |
| scripts/…/factory.js | Shows how one skill becomes client-specific skills, agents, scripts, and hooks. |
| scripts/…/utils.js | Defines provider block compilation, placeholder replacement, and safe script marker replacement. |
| plugin/…/plugin.json | Defines the Claude package identity and its packaged skill entry point. |
| plugin/…/plugin.json | Defines the Grok package identity and client-facing metadata. |
| tests/…/llm-agent.mjs | Shows provider selection and the common live-agent model interface. |
| tests/…/providers.mjs | Shows model-family routing, credential checks, and test model options. |
| crates/…/generate_image.rs | Defines image generation, reference-image handling, synthetic fallback, and output provenance. |
| skill/…/context.mjs | Reports image fallback availability and selects local project context. |
| skill/…/embed-prompt.mjs | Defines prompt provenance for generated and existing raster assets. |
| skill/…/serve-question.mjs | Defines the local visual decision interface and delayed image slots. |
| skill/…/generation-preflight.mjs | Defines the local source-preparation boundary for live agent edits. |