Accounts, models, and reasoning policy
The request configuration layer turns a public model name into a usable provider target. It combines provider access, model references, discovered model facts, fallback choices, and reasoning intent before execution starts. The parent page, Application execution and routing, describes where this work fits in the full request path. Execution orchestration describes what happens after the target is ready.
Account availability
Connected-account state
A connected account is provider access owned by an interactive sign-in flow instead of a static API setting. The application exposes a small, credential-free status contract so the admin layer can show connection state and guide browser or device login without handling secrets. The current bootstrap path wires this account boundary to the OpenAI provider.
| State | Meaning for request configuration |
|---|---|
disconnected | The provider does not contribute connected-account access. |
connecting | Login is in progress. Temporary login details may be shown to the user. |
connected | The provider can be made available to the runtime through the account owner. |
error | The last account operation failed and access is not usable. |
The account manager also reports a revision. When that revision changes, the application synchronizes provider model availability. Disconnect removes that provider's cached models, while a new connection makes the provider eligible for a fresh model refresh.
References: src/…/connected_accounts.py• L10-43 src/…/bootstrap.py• L41-62 src/…/auth.py• L182-213 src/…/application.py• L258-300
Provider availability
A provider becomes usable through either validated configuration or a connected account. The provider catalog records which access kind applies. Model discovery uses this access state to decide which provider metadata can remain in the application catalog.
This means a model reference can be syntactically valid while its provider is not currently usable. The route still identifies the intended provider and model, but execution can only resolve that provider when its configuration or account access is present.
References: src/…/provider_catalog.py• L90-122 src/…/config.py• L28-49 src/…/discovery.py• L47-75 src/…/provider_manager.py• L220-227
Model references and metadata
Provider model references
A provider model reference is the stable provider/model form that joins a provider ID to the provider's model ID. Settings require this form for the primary model, family-specific models, and fallbacks. The same reference lets routing preserve the provider choice while changing the public model name used by a client.
The application gathers unique references from the primary model, the optional Fable, Opus, Sonnet, and Haiku routes, and the ordered fallback list. A family route is an input mapping, not a separate provider: when a public name matches a family label, its configured reference becomes the primary target. If no family override exists, the primary model is used.
References: src/…/model_refs.py• L7-14 src/…/model_refs.py• L25-31 src/…/model_refs.py• L46-71 src/…/settings.py• L351-376 src/…/routing.py• L136-143 src/…/routing.py• L183-211
Model metadata catalog
Model metadata is provider-reported information used to describe models before a request runs. The catalog can retain thinking support, accepted input types, context size, and maximum output size. A missing value means that the provider did not report that fact.
The runtime warms metadata for configured providers before clients ask for models. It can then refresh other usable providers in the background. Each refresh reports which providers succeeded and which failed. The cache is scoped to configured and connected providers, so stale entries do not keep a removed account or invalid provider visible.
The application model inventory is the client-neutral layer between cached facts and client views. It merges configured chat model references with other cached provider models, removes duplicate references, and sorts the result by provider and model name. It also marks the configured default with the wire ID that clients must use. This keeps selection independent of list order while allowing each client view to apply its own response shape.
The client model views use this metadata to advertise a normal model ID and, when needed, a no-thinking variant. Unknown thinking support is treated as potentially supporting thinking. Other metadata helps describe model limits and input support. This catalog describes capability to clients; it does not replace route resolution or provider access checks.
References: src/…/model_metadata.py• L6-24 src/…/model_catalog.py• L22-83 src/…/discovery.py• L40-115 src/…/model_cache.py• L19-84 src/…/model_catalog.py• L164-249 src/…/gateway_model_ids.py• L20-50
Route selection
Public model mapping
The model router resolves each incoming public model name to one canonical ProviderModelTarget. A target contains the provider ID, the provider's model ID, and the complete provider model reference. This target is the contract passed to the execution layer.
The router accepts three useful forms:
- A gateway model ID that encodes a provider and model.
- A direct
provider/modelreference. - A Claude-compatible family name such as an Opus or Sonnet name.
Gateway IDs can also mark a target as no-thinking. Family names select the matching configured model and reasoning override by name. All other names use the primary configured model and root reasoning preference. Configured provider IDs must exist in the supported provider catalog. An unsupported direct provider form does not become an execution target.
The same route model supports Claude Messages requests, OpenAI Responses requests, and token-count requests. For requests that already carry an application-owned reasoning policy, routing changes the target model without resolving client reasoning intent again.
References: src/…/routing.py• L34-65 src/…/routing.py• L73-120 src/…/routing.py• L122-181 src/…/routing.py• L183-211 src/…/routing.py• L213-274 src/…/gateway_model_ids.py• L13-50
Fallback targets
Fallback targets are an ordered continuation of the primary route. The router converts each configured fallback reference into the same canonical target shape and removes a duplicate of the primary target. Execution tries these targets only when the current attempt fails in a way that permits a retry before output is committed.
Fallbacks keep the same request-level reasoning policy as the primary target. They change the provider and model destination, not the client's configured intent.
Fallback reasoning is resolved from the original request intent for each candidate. A candidate can apply its own route preference, but resolution never starts from the policy produced by a previous fallback. An explicit no-reasoning policy stays off. inherit keeps the original policy, while client uses the parsed client policy.
When a fixed route preference replaces client controls, the fallback request removes those controls before execution. This prevents old thinking or effort fields from overriding the policy selected for the new candidate.
References: src/…/routing.py• L136-143 src/…/execution.py• L153-160 src/…/execution.py• L214-243 src/…/reasoning.py• L42-80 src/…/settings.py• L373-376
Reasoning policy
Policy precedence
A reasoning policy is the provider-neutral statement of whether the request may use reasoning and which effort or token budget the client wants. The application resolves this policy once, after it has selected the route and before it calls the provider.
The effective preference follows this order:
- A gateway model variant can force reasoning off.
- A family route can use its own reasoning preference when it is not
inherit. - The root application preference applies when no family override exists.
- The client request supplies the final thinking and effort details when the preference is
client.
The root preference cannot be inherit, because a request must have a final application-level default. A family preference may inherit that root value. The supported fixed choices are off, low, medium, high, xhigh, and max, in addition to client-controlled behavior.
References: src/…/reasoning.py• L6-24 src/…/settings.py• L537-556 src/…/settings.py• L715-722 src/…/routing.py• L79-119 src/…/routing.py• L193-211 src/…/reasoning.py• L18-54
Client reasoning intent
Client input can disable reasoning, enable it, or leave the provider to decide. It can also provide a named effort or a positive token budget. The policy keeps control and effort separate, so a client can set effort while still turning reasoning off. An explicit off control takes precedence over a positive budget, while a positive budget turns reasoning on when no off control is present.
| Policy input | Application meaning |
|---|---|
| No thinking control or effort | Leave the choice to the provider. |
Disabled thinking or none effort | Disable reasoning and its output. |
| Enabled or adaptive thinking control | Request reasoning. |
| Named effort | Record the requested effort. It does not by itself select on or off. |
| Positive token budget | Request that exact budget and enable reasoning unless thinking is explicitly disabled. |
Named efforts map to application budgets as follows:
| Effort | Budget |
|---|---|
minimal or low | 512 tokens |
medium | 1,024 tokens |
high | 2,048 tokens |
xhigh | 4,096 tokens |
max | 8,192 tokens |
The resolver supports both Claude Messages thinking fields and OpenAI Responses reasoning fields. It applies the application preference once to the parsed client intent. The resulting policy travels with the routed request to provider preflight and streaming. Fallback preparation removes controls replaced by a fixed policy, so each candidate receives the policy selected for that route.
References: src/…/reasoning.py• L18-57 src/…/reasoning.py• L60-80 src/…/reasoning.py• L83-123 src/…/reasoning.py• L126-175 src/…/reasoning.py• L7-39 src/…/reasoning.py• L42-121 src/…/routing.py• L213-246 src/…/routing.py• L258-274 src/…/execution.py• L153-180 src/…/execution.py• L228-243
Request configuration flow
The following flow shows how account state, configured references, discovered metadata, public model input, and reasoning controls meet before provider execution.
flowchart LR
A[Configured settings] --> B[Model references]
C[Connected account state] --> D[Usable providers]
D --> E[Model metadata cache]
E --> F[Application model inventory]
F --> G[Client model catalog]
B --> H[Model router]
I[Public model name] --> H
J[Client thinking and effort] --> K[Reasoning resolver]
L[Route reasoning preference] --> K
H --> M[Primary target and fallbacks]
K --> N[Resolved reasoning policy]
M --> O[Configured request]
N --> O
O --> P[Provider execution]
For an API request, the handler creates the routed request context before execution. That context carries the provider/model route and the resolved reasoning policy together. When a fallback is needed, the application prepares the next candidate with its own route policy before provider execution.
References: src/…/messages.py• L101-116 src/…/responses.py• L67-81 src/…/routing.py• L213-274 src/…/execution.py• L143-160
Change checklist
When these files or sections are changed, remember to consider these:
- Keep the
provider/modelreference format aligned across settings validation, route parsing, gateway model ID decoding, and model catalog output. - Keep connected-account revision changes aligned with provider model-cache add and remove behavior, or stale models can remain visible after account changes.
- Keep family model fields and family reasoning fields aligned with the family-name matching rules, including the
inheritfallback to the root preference. - Keep the reasoning policy contract aligned between request parsing, fallback execution, and provider preflight or stream calls.
- Keep thinking capability metadata aligned with both normal and no-thinking model IDs so clients do not receive a capability claim that routes to the wrong policy.
- Keep the application model inventory aligned with configured references, cached provider models, default model IDs, and every client-specific catalog view.
- Keep new model metadata fields aligned across provider discovery, cache state, application inventory, and client model catalog output.
- Keep Claude Messages and OpenAI Responses reasoning inputs aligned with the shared policy so one protocol does not bypass application-level controls.
Important failure modes
| Trigger | Consequence | Guard |
|---|---|---|
| A model setting omits its provider prefix or names an unsupported provider. | Settings or route resolution rejects the model before provider execution. | Validate every configured model reference and check provider IDs against the provider catalog. |
| A connected account disconnects without a model-cache scope update. | Models from the removed account remain selectable in the client catalog. | Use the account revision path and evict the provider cache on disconnect. |
| A request combines a positive budget with an explicit off control. | The off control wins, so the budget does not enable reasoning. | Treat explicit disablement as higher priority than a budget. |
| A provider fails after it starts returning output. | The request cannot safely move to a fallback because output is already committed. | Advance only on retryable failure before the first committed chunk. |
A provider reports supports_thinking as false but the normal model ID is advertised. | A client may send thinking controls to a model that cannot support them. | Publish the no-thinking model variant and mark reasoning support false. |
| A provider refresh reports failure but the catalog treats it as a successful refresh. | Clients can receive incomplete or stale model metadata without a clear provider status. | Preserve separate refreshed and failed provider outcomes. |
Key files
| File | Why to read it |
|---|---|
| src/…/routing.py | Defines the public-name to provider-target mapping and attaches route reasoning preferences. |
| src/…/reasoning.py | Shows how Claude Messages and OpenAI Responses request fields become one reasoning policy. |
| src/…/model_catalog.py | Defines the client-neutral model inventory that combines configured and cached models. |
| src/…/connected_accounts.py | Defines the safe account status and login boundary used by the application. |
| src/…/application.py | Connects account state changes to provider availability and model-cache updates. |
| src/…/provider_manager.py | Owns provider availability, model metadata access, and catalog publication. |
| src/…/discovery.py | Explains which configured or connected providers are eligible for model discovery. |
| src/…/model_cache.py | Shows the lifetime and capability lookup rules for discovered model metadata. |
| src/…/model_catalog.py | Shows how model metadata becomes client-visible model and reasoning capability variants. |
| src/…/reasoning.py | Defines the provider-neutral reasoning controls, effort values, and budget meaning. |
| src/…/model_metadata.py | Defines the shared model facts and per-provider refresh outcome used by the metadata catalog. |
| src/…/model_refs.py | Defines the shared provider/model reference used by settings and routing. |