Runtime startup and provider management
The runtime layer turns validated settings into one running service. It exposes the HTTP API through ASGI, owns process-wide resources, builds provider clients when needed, and keeps provider models available to API requests, Codex sessions, and integrations. See Configuration and runtime, Configuration lifecycle, Provider runtime and discovery, and API gateway for related views of settings, provider access, and HTTP routing.
Runtime assembly
Runtime assembly connects settings, provider access, storage, optional services, integrations, and API services into one ownership graph. The application owner starts these parts as one unit and keeps responsibility for them until shutdown.
Provider construction boundary
The provider construction boundary maps a provider ID to shared settings and one provider adapter. A provider descriptor states its credential, URL, and other required settings. The construction path turns those values into a common provider configuration and then selects either a declared provider profile, a special adapter, an injected provider, or a custom provider.
Provider clients remain lazy. The runtime loads provider construction code outside the event loop, then creates the client on the owning loop only when a request or model discovery needs it. This keeps provider setup separate from provider use and lets one runtime share a client for one provider and API key.
References: src/…/config.py• L19-109 src/…/factory.py• L265-342 src/…/runtime.py• L24-59
Provider configuration boundary
Provider configuration is the validation boundary between neutral catalog data and provider clients. It resolves the selected credential and base URL from settings, applies shared network and logging settings, and reports missing values as user-facing availability errors. Custom providers use their own definition while receiving the same shared network settings.
The catalog and construction owners must stay in one-to-one agreement. Startup checks this relationship so a catalog provider cannot silently lack a construction path or have two competing paths.
References: src/…/config.py• L40-109 src/…/config.py• L112-130 src/…/factory.py• L238-278
Durable runtime resources
The application owns shared storage for Code sessions and messaging state. It also owns optional services such as integrations, messaging, transcription, and connected accounts. Each service starts only when its settings allow it and remains attached to the application owner until its cleanup completes.
This ownership model lets startup fail without exposing a partial service. It also lets shutdown retry resources that did not close on the first attempt.
References: src/…/bootstrap.py• L53-153 src/…/application.py• L101-153 src/…/application.py• L602-641
Configuration update boundary
The application is the boundary for live configuration changes. It prepares and validates an update, checks submitted credentials, then either records settings that require a restart or asks the provider manager to build and publish a new generation under the shared configuration lock.
The saved settings and the active provider generation therefore move together for changes that can apply live. If a restart signal fails, the saved change remains marked as pending instead of being reported as active.
References: src/…/application.py• L272-374 src/…/application.py• L551-571
ASGI exposure
RuntimeASGIApp is the ASGI boundary for the server. ASGI is the interface through which the web server sends HTTP traffic and process-lifetime events. The wrapper forwards HTTP traffic to FastAPI and sends startup and shutdown events to the application owner.
The server reports startup only after the runtime starts successfully. A failed startup closes owned resources and does not expose a partial service. Shutdown reports success only after all owned resources close. The process command gives this wrapper to Uvicorn with the configured host and port.
The following flow shows how the process supervisor, ASGI adapter, application owner, and HTTP app divide responsibility.
sequenceDiagram
participant S as Process supervisor
participant U as Uvicorn
participant A as RuntimeASGIApp
participant R as ApplicationRuntime
participant F as FastAPI
S->>U: Start assembled ASGI app
U->>A: lifespan.startup
A->>R: start()
R-->>A: startup result
A-->>U: startup.complete or startup.failed
U->>A: HTTP request
A->>F: Forward request
F-->>U: HTTP response
U->>A: lifespan.shutdown
A->>R: close()
R-->>A: close result
A-->>U: shutdown.complete or shutdown.failed
Lifespan ownership
Startup prepares managed configuration, starts provider model discovery, starts enabled integrations and messaging work, and starts the Code service. The runtime marks itself started after it has handed off its owned tasks. It enters a draining state before shutdown stops new provider leases and long-lived work.
Shutdown cancels discovery and background work, quiesces messaging ingress, and then closes Code, storage, transcription, providers, and connected accounts. If startup or shutdown is cancelled, the runtime still waits for owned work to settle.
References: src/…/application.py• L169-267 src/…/application.py• L602-641 src/…/lifecycle.py• L8-37
Provider runtime generations
A provider generation is one immutable settings snapshot plus its provider clients, admission limits, and model cache. ProviderRuntimeManager publishes one current generation and gives each operation a lease to it. A lease prevents replacement or shutdown from closing provider state while that operation still uses it.
Provider clients are created at most once for each provider and API key within a generation. Concurrent requests share the same construction task. When the generation closes, pending construction stops and each created client receives cleanup.
Generation replacement
An admin change that does not require a process restart builds a candidate generation before it becomes current. The manager commits the settings only after candidate creation succeeds. It then publishes the candidate, retires the old generation, and starts discovery for the new settings.
Existing leases keep the retired generation alive until their work ends. If candidate creation or the settings commit fails, the current generation stays active. A candidate that was built but not published remains tracked for later cleanup.
References: src/…/provider_manager.py• L627-702 src/…/provider_manager.py• L773-794
| State | Request behavior | Resource behavior |
|---|---|---|
| Current | New leases are allowed | Providers can be resolved and model data can be refreshed |
| Retired | Existing leases can finish | Cleanup waits for leases to drain |
| Closing | New leases are rejected | Discovery and refresh work are canceled |
| Closed | Runtime is unavailable | Provider clients and cached state are released |
Provider discovery
Provider discovery fills the current model cache with metadata from configured providers and connected accounts. Startup covers providers referenced by configured models and providers allowed by the current settings. A caller that needs a complete catalog waits for the current pass, while the service can report discovery state while that pass runs.
Discovery is best effort across providers. A failed provider query is recorded for that provider while successful providers still publish their model data. The failure classifier follows wrapped transport and provider errors and exposes only safe messages for proxy, DNS, TLS, timeout, HTTP status, invalid response, unavailable, or unknown causes. Refresh results and catalog status carry these per-provider messages without copying upstream error text or URLs. The cache scope follows settings and account state, so disconnecting an account removes its models and connecting it starts a refresh for that provider.
References: src/…/discovery.py• L19-103 src/…/provider_manager.py• L291-342 src/…/provider_manager.py• L424-554 src/…/provider_manager.py• L556-625
Codex model catalog
The Codex model catalog is a generated inventory of routable application models in the shape expected by Codex clients. The runtime builds it from the current provider model cache and sends a frozen snapshot to the catalog publisher. Startup creates the catalog when needed, and later discovery, provider replacement, and account changes can publish a newer inventory.
Publication is serialized and marks the current generation's file as ready only after the write succeeds. The catalog must contain a usable model for Codex use. A temporary lack of routable models does not stop the main HTTP service, but a caller that requires a ready catalog receives a publication error.
The default catalog path is ~/…/codex-model-catalog.json. The runtime and Codex integration use this shared file so both sides use the same model inventory.
Codex app-server sessions
The Codex app-server session is the runtime bridge between a Code request and a native Codex process. The harness waits for a catalog snapshot, checks that the selected model is available, and starts Codex with the local proxy and the model mapping that matches that snapshot.
The session uses bidirectional JSONL messages for initialization, threads, turns, prompts, and events. It accepts input only for threads owned by the session, forwards native events to the Code service, and keeps the process reserved until it has been reaped. A failed connection or timeout reports that the request was not resent. Closing the session first waits for normal process exit and then kills the owned process tree if needed.
References: src/…/codex_app_server.py• L50-129 src/…/codex_app_server.py• L131-246 src/…/codex_app_server.py• L263-299 src/…/codex_app_server.py• L515-577 src/…/codex_app_server.py• L632-718
References: src/…/provider_manager.py• L424-433 src/…/provider_manager.py• L500-625 src/…/provider_manager.py• L627-702 src/…/codex_catalog.py• L22-67 src/…/paths.py• L59-62
Integration and messaging lifecycle
The runtime coordinates clients that consume the local model catalog and an optional messaging platform. These services use the application owner and provider manager instead of creating separate provider state.
Integration publication
Integration service keeps external client settings aligned with the current model catalog. Catalog changes trigger nonblocking refresh work, while connect and refresh operations wait for a stable current catalog before writing client settings. Integration work shares the application configuration lock with admin changes, so it cannot publish a stale settings snapshot.
Startup refreshes the integrations that can consume the local proxy and catalog. The service reports each integration as starting, ready, or failed. It stops background updates before the application releases provider and storage resources.
References: src/…/integration_service.py• L59-169 src/…/integration_service.py• L331-405 src/…/integration_service.py• L702-789
Messaging lifecycle
Messaging service owns the optional messaging graph. It prepares storage, loads the selected platform, creates its workflow, waits for HTTP readiness, and then starts delivery. Voice transcription starts only when the selected settings enable it.
Messaging startup runs as owned work that is separate from the main HTTP startup. If platform setup fails, the service records a failed state and closes its partial graph. Shutdown quiesces ingress before it closes the workflow, platform runtime, and transcriber.
References: src/…/messaging_service.py• L39-114 src/…/messaging_service.py• L125-182 src/…/messaging_service.py• L225-347
Runtime diagnostics
Runtime diagnostics produce a shareable local report without starting the application or changing its settings. The report combines safe system facts with configuration state, provider readiness, integration state, installed harness status, and model discovery observed from an already running local server.
Shareable diagnostic report
The report separates configuration availability from runtime and integration availability. It marks credentials as configured without exposing their values, reduces URLs to safe origins, and uses approved provider discovery messages. It reads the running server only through its local admin status endpoint, so it does not refresh providers or alter the catalog.
References: src/…/diagnostics.py• L135-205 src/…/diagnostics.py• L208-241 src/…/diagnostics.py• L296-333 src/…/diagnostics.py• L336-395
Provider-manager lifecycle
The provider manager is the single owner of provider generations, leases, model discovery, catalog publication, and provider cleanup. API handlers, integrations, Code harnesses, and admin operations use these shared capabilities instead of managing provider clients directly. It also keeps provider-level discovery failures visible through catalog status and refresh results.
Connected-account changes update only the affected provider's cache. A disconnect removes that provider's models. A new connection adds the provider to the cache scope and starts a refresh under the current settings.
On shutdown, the manager rejects new leases, cancels discovery and publication work, waits for active leases to drain, and closes current and retired generations. It also retries cleanup for runtimes that were built but never published. Shutdown fails until every owned provider runtime closes.
The main lifecycle relationship is:
flowchart LR
Settings[Validated settings] --> Current[Current provider generation]
Current --> Lease[Request lease]
Lease --> Request[API request or Code operation]
Current --> Refresh[Model discovery]
Refresh --> Cache[Provider model cache]
Cache --> Catalog[Codex model catalog]
Settings --> Replace[Configuration replacement]
Replace --> New[New generation]
Current --> Retired[Retired generation]
Retired --> Drain[Lease drain]
Drain --> Cleanup[Provider cleanup]
References: src/…/provider_manager.py• L223-289 src/…/provider_manager.py• L291-342 src/…/provider_manager.py• L627-702 src/…/provider_manager.py• L704-880
Change checklist
When these files or sections are changed, remember to consider these:
- When provider IDs or catalog descriptors change, keep catalog ownership, provider configuration, factory selection, and construction-time checks aligned.
- When credentials, base URLs, or required settings change, keep user-facing validation errors and custom-provider configuration behavior aligned.
- When provider construction changes, preserve worker-side module loading and loop-owned client construction.
- When generation replacement changes, preserve candidate construction before settings commit, current-generation publication, lease drain, and failed-candidate cleanup.
- When model discovery or account changes change, keep cache scope, per-provider failure reporting, safe diagnostic classification, catalog publication, and catalog status aligned.
- When Codex catalog publication changes, keep the generation check, serialized publication, ready-file state, and retry error contract aligned.
- When Codex app-server selection changes, keep the selected catalog snapshot, model fingerprints, local proxy settings, and native process lifecycle aligned.
- When integration publication changes, keep catalog revision checks and configuration-lock ordering aligned with live admin changes.
- When provider shutdown changes, preserve cancellation of discovery, pending construction, publication work, active lease draining, and unpublished-runtime cleanup.
- When ASGI lifespan behavior changes, keep Uvicorn result events aligned with application startup and cleanup status.
- When diagnostics changes, keep secret redaction, local-server limits, and safe provider failure messages aligned with the report contract.
Important failure modes
| Trigger | Consequence | Guard |
|---|---|---|
| A provider lacks a matching construction owner | The provider cannot be built or may use the wrong adapter | Validate that catalog IDs are covered exactly once by profiles, special adapters, or injected providers |
| A required credential or URL is missing | Provider use fails with an unavailable configuration state | Resolve and validate provider settings before client construction |
| Several requests resolve one uncached provider | Duplicate clients or admission state could be created | Share one construction task per provider and API key |
| Candidate construction or settings commit fails | The old generation remains active while the candidate is cleaned up | Build before publication and retain failed unpublished runtimes for retry |
| A request holds a lease while its generation is retired | Cleanup could close provider state still in use | Wait for active leases to drain before generation cleanup |
| A provider model query fails | That provider has no fresh metadata while other providers can update, and callers need a safe reason to diagnose the failure | Classify wrapped causes into approved messages and carry them in refresh results and catalog status |
| A connected provider is removed | Its old models could remain selectable | Remove the provider from the cache and restart catalog publication |
| Catalog publication fails | Codex cannot use a fresh catalog while HTTP can continue | Keep catalog readiness separate from HTTP startup and report publication failure |
| A Codex process loses its pipe or does not acknowledge a request | The session becomes unavailable and input is not resent | Fail pending calls safely, keep the process reserved until reaping, and close the owned process tree |
| An integration writes from an old catalog revision | An external client can receive stale model settings | Recheck the generation and catalog revision while holding the shared publication order |
| Diagnostics reads a non-local server or unsafe error text | A report could expose remote state or sensitive provider details | Limit discovery reads to loopback and use redacted URL and failure summaries |
| Shutdown cleanup fails | Resources remain owned and the runtime cannot claim a clean close | Keep failed generations attached and retry cleanup |
Key files
| File | Why to read it |
|---|---|
| src/…/bootstrap.py | Shows the production composition root and the links between settings, providers, storage, services, and the runtime owner. |
| src/…/application.py | Defines process-wide startup, live configuration changes, service ownership, and shutdown. |
| src/…/asgi.py | Defines the ASGI boundary and lifespan result behavior. |
| src/…/config.py | Defines the boundary from provider descriptors and settings to shared provider configuration. |
| src/…/factory.py | Shows how catalog provider IDs map to provider construction owners. |
| src/…/runtime.py | Defines lazy client creation, shared construction, and cleanup for one settings snapshot. |
| src/…/discovery.py | Defines provider selection for discovery and the safe messages used for model-list failures. |
| src/…/provider_manager.py | Defines generations, leases, discovery, catalog publication, replacement, failure status, and cleanup. |
| src/…/codex_app_server.py | Shows how a catalog-selected model becomes an owned Codex process and Code session. |
| src/…/integration_service.py | Shows how the current model catalog reaches external client configuration. |
| src/…/messaging_service.py | Shows optional messaging startup, readiness gates, delivery ownership, and ordered cleanup. |
| src/…/codex_catalog.py | Connects the provider model inventory to the generated Codex catalog file. |
| src/…/diagnostics.py | Defines the safe local report for configuration, providers, integrations, harnesses, and discovery state. |
| src/…/paths.py | Defines the shared default path used for the Codex model catalog. |