Introducing a new way to review your PRs.

Read the announcement

alishahryar1 / free-claude-code

Use Claude Code, Codex, Antigravity, VSCode, Pi, and OpenCode (and 6 other harnesses) for free (1.3B+ free tokens) from your terminal, app, IDE, or phone, and now from the browser with native browser sessions (multi-harness + multi-model) like OpenClaw (voice supported + ToS friendly)

View on GitHub

28 documents · Updated Nov 20, 2026

docs / runtime-startup-and-provider-management

Runtime startup and provider management

The runtime layer turns validated settings into one running service. It exposes the HTTP API through ASGI, owns process-wide resources, builds provider clients when needed, and keeps provider models available to API requests, Codex sessions, and integrations. See Configuration and runtime, Configuration lifecycle, Provider runtime and discovery, and API gateway for related views of settings, provider access, and HTTP routing.

Runtime assembly

Runtime assembly connects settings, provider access, storage, optional services, integrations, and API services into one ownership graph. The application owner starts these parts as one unit and keeps responsibility for them until shutdown.

Provider construction boundary

The provider construction boundary maps a provider ID to shared settings and one provider adapter. A provider descriptor states its credential, URL, and other required settings. The construction path turns those values into a common provider configuration and then selects either a declared provider profile, a special adapter, an injected provider, or a custom provider.

Provider clients remain lazy. The runtime loads provider construction code outside the event loop, then creates the client on the owning loop only when a request or model discovery needs it. This keeps provider setup separate from provider use and lets one runtime share a client for one provider and API key.

References: src/…/config.py• L19-109 src/…/factory.py• L265-342 src/…/runtime.py• L24-59

Provider configuration boundary

Provider configuration is the validation boundary between neutral catalog data and provider clients. It resolves the selected credential and base URL from settings, applies shared network and logging settings, and reports missing values as user-facing availability errors. Custom providers use their own definition while receiving the same shared network settings.

The catalog and construction owners must stay in one-to-one agreement. Startup checks this relationship so a catalog provider cannot silently lack a construction path or have two competing paths.

References: src/…/config.py• L40-109 src/…/config.py• L112-130 src/…/factory.py• L238-278

Durable runtime resources

The application owns shared storage for Code sessions and messaging state. It also owns optional services such as integrations, messaging, transcription, and connected accounts. Each service starts only when its settings allow it and remains attached to the application owner until its cleanup completes.

This ownership model lets startup fail without exposing a partial service. It also lets shutdown retry resources that did not close on the first attempt.

References: src/…/bootstrap.py• L53-153 src/…/application.py• L101-153 src/…/application.py• L602-641

Configuration update boundary

The application is the boundary for live configuration changes. It prepares and validates an update, checks submitted credentials, then either records settings that require a restart or asks the provider manager to build and publish a new generation under the shared configuration lock.

The saved settings and the active provider generation therefore move together for changes that can apply live. If a restart signal fails, the saved change remains marked as pending instead of being reported as active.

References: src/…/application.py• L272-374 src/…/application.py• L551-571

ASGI exposure

RuntimeASGIApp is the ASGI boundary for the server. ASGI is the interface through which the web server sends HTTP traffic and process-lifetime events. The wrapper forwards HTTP traffic to FastAPI and sends startup and shutdown events to the application owner.

The server reports startup only after the runtime starts successfully. A failed startup closes owned resources and does not expose a partial service. Shutdown reports success only after all owned resources close. The process command gives this wrapper to Uvicorn with the configured host and port.

The following flow shows how the process supervisor, ASGI adapter, application owner, and HTTP app divide responsibility.

sequenceDiagram
    participant S as Process supervisor
    participant U as Uvicorn
    participant A as RuntimeASGIApp
    participant R as ApplicationRuntime
    participant F as FastAPI

    S->>U: Start assembled ASGI app
    U->>A: lifespan.startup
    A->>R: start()
    R-->>A: startup result
    A-->>U: startup.complete or startup.failed
    U->>A: HTTP request
    A->>F: Forward request
    F-->>U: HTTP response
    U->>A: lifespan.shutdown
    A->>R: close()
    R-->>A: close result
    A-->>U: shutdown.complete or shutdown.failed

Lifespan ownership

Startup prepares managed configuration, starts provider model discovery, starts enabled integrations and messaging work, and starts the Code service. The runtime marks itself started after it has handed off its owned tasks. It enters a draining state before shutdown stops new provider leases and long-lived work.

Shutdown cancels discovery and background work, quiesces messaging ingress, and then closes Code, storage, transcription, providers, and connected accounts. If startup or shutdown is cancelled, the runtime still waits for owned work to settle.

References: src/…/application.py• L169-267 src/…/application.py• L602-641 src/…/lifecycle.py• L8-37

Provider runtime generations

A provider generation is one immutable settings snapshot plus its provider clients, admission limits, and model cache. ProviderRuntimeManager publishes one current generation and gives each operation a lease to it. A lease prevents replacement or shutdown from closing provider state while that operation still uses it.

Provider clients are created at most once for each provider and API key within a generation. Concurrent requests share the same construction task. When the generation closes, pending construction stops and each created client receives cleanup.

Generation replacement

An admin change that does not require a process restart builds a candidate generation before it becomes current. The manager commits the settings only after candidate creation succeeds. It then publishes the candidate, retires the old generation, and starts discovery for the new settings.

Existing leases keep the retired generation alive until their work ends. If candidate creation or the settings commit fails, the current generation stays active. A candidate that was built but not published remains tracked for later cleanup.

References: src/…/provider_manager.py• L627-702 src/…/provider_manager.py• L773-794

StateRequest behaviorResource behavior
CurrentNew leases are allowedProviders can be resolved and model data can be refreshed
RetiredExisting leases can finishCleanup waits for leases to drain
ClosingNew leases are rejectedDiscovery and refresh work are canceled
ClosedRuntime is unavailableProvider clients and cached state are released

Provider discovery

Provider discovery fills the current model cache with metadata from configured providers and connected accounts. Startup covers providers referenced by configured models and providers allowed by the current settings. A caller that needs a complete catalog waits for the current pass, while the service can report discovery state while that pass runs.

Discovery is best effort across providers. A failed provider query is recorded for that provider while successful providers still publish their model data. The failure classifier follows wrapped transport and provider errors and exposes only safe messages for proxy, DNS, TLS, timeout, HTTP status, invalid response, unavailable, or unknown causes. Refresh results and catalog status carry these per-provider messages without copying upstream error text or URLs. The cache scope follows settings and account state, so disconnecting an account removes its models and connecting it starts a refresh for that provider.

References: src/…/discovery.py• L19-103 src/…/provider_manager.py• L291-342 src/…/provider_manager.py• L424-554 src/…/provider_manager.py• L556-625

Codex model catalog

The Codex model catalog is a generated inventory of routable application models in the shape expected by Codex clients. The runtime builds it from the current provider model cache and sends a frozen snapshot to the catalog publisher. Startup creates the catalog when needed, and later discovery, provider replacement, and account changes can publish a newer inventory.

Publication is serialized and marks the current generation's file as ready only after the write succeeds. The catalog must contain a usable model for Codex use. A temporary lack of routable models does not stop the main HTTP service, but a caller that requires a ready catalog receives a publication error.

The default catalog path is ~/…/codex-model-catalog.json. The runtime and Codex integration use this shared file so both sides use the same model inventory.

Codex app-server sessions

The Codex app-server session is the runtime bridge between a Code request and a native Codex process. The harness waits for a catalog snapshot, checks that the selected model is available, and starts Codex with the local proxy and the model mapping that matches that snapshot.

The session uses bidirectional JSONL messages for initialization, threads, turns, prompts, and events. It accepts input only for threads owned by the session, forwards native events to the Code service, and keeps the process reserved until it has been reaped. A failed connection or timeout reports that the request was not resent. Closing the session first waits for normal process exit and then kills the owned process tree if needed.

References: src/…/codex_app_server.py• L50-129 src/…/codex_app_server.py• L131-246 src/…/codex_app_server.py• L263-299 src/…/codex_app_server.py• L515-577 src/…/codex_app_server.py• L632-718

References: src/…/provider_manager.py• L424-433 src/…/provider_manager.py• L500-625 src/…/provider_manager.py• L627-702 src/…/codex_catalog.py• L22-67 src/…/paths.py• L59-62

Integration and messaging lifecycle

The runtime coordinates clients that consume the local model catalog and an optional messaging platform. These services use the application owner and provider manager instead of creating separate provider state.

Integration publication

Integration service keeps external client settings aligned with the current model catalog. Catalog changes trigger nonblocking refresh work, while connect and refresh operations wait for a stable current catalog before writing client settings. Integration work shares the application configuration lock with admin changes, so it cannot publish a stale settings snapshot.

Startup refreshes the integrations that can consume the local proxy and catalog. The service reports each integration as starting, ready, or failed. It stops background updates before the application releases provider and storage resources.

References: src/…/integration_service.py• L59-169 src/…/integration_service.py• L331-405 src/…/integration_service.py• L702-789

Messaging lifecycle

Messaging service owns the optional messaging graph. It prepares storage, loads the selected platform, creates its workflow, waits for HTTP readiness, and then starts delivery. Voice transcription starts only when the selected settings enable it.

Messaging startup runs as owned work that is separate from the main HTTP startup. If platform setup fails, the service records a failed state and closes its partial graph. Shutdown quiesces ingress before it closes the workflow, platform runtime, and transcriber.

References: src/…/messaging_service.py• L39-114 src/…/messaging_service.py• L125-182 src/…/messaging_service.py• L225-347

Runtime diagnostics

Runtime diagnostics produce a shareable local report without starting the application or changing its settings. The report combines safe system facts with configuration state, provider readiness, integration state, installed harness status, and model discovery observed from an already running local server.

Shareable diagnostic report

The report separates configuration availability from runtime and integration availability. It marks credentials as configured without exposing their values, reduces URLs to safe origins, and uses approved provider discovery messages. It reads the running server only through its local admin status endpoint, so it does not refresh providers or alter the catalog.

References: src/…/diagnostics.py• L135-205 src/…/diagnostics.py• L208-241 src/…/diagnostics.py• L296-333 src/…/diagnostics.py• L336-395

Provider-manager lifecycle

The provider manager is the single owner of provider generations, leases, model discovery, catalog publication, and provider cleanup. API handlers, integrations, Code harnesses, and admin operations use these shared capabilities instead of managing provider clients directly. It also keeps provider-level discovery failures visible through catalog status and refresh results.

Connected-account changes update only the affected provider's cache. A disconnect removes that provider's models. A new connection adds the provider to the cache scope and starts a refresh under the current settings.

On shutdown, the manager rejects new leases, cancels discovery and publication work, waits for active leases to drain, and closes current and retired generations. It also retries cleanup for runtimes that were built but never published. Shutdown fails until every owned provider runtime closes.

The main lifecycle relationship is:

flowchart LR
    Settings[Validated settings] --> Current[Current provider generation]
    Current --> Lease[Request lease]
    Lease --> Request[API request or Code operation]
    Current --> Refresh[Model discovery]
    Refresh --> Cache[Provider model cache]
    Cache --> Catalog[Codex model catalog]
    Settings --> Replace[Configuration replacement]
    Replace --> New[New generation]
    Current --> Retired[Retired generation]
    Retired --> Drain[Lease drain]
    Drain --> Cleanup[Provider cleanup]

References: src/…/provider_manager.py• L223-289 src/…/provider_manager.py• L291-342 src/…/provider_manager.py• L627-702 src/…/provider_manager.py• L704-880

Change checklist

When these files or sections are changed, remember to consider these:

  • When provider IDs or catalog descriptors change, keep catalog ownership, provider configuration, factory selection, and construction-time checks aligned.
  • When credentials, base URLs, or required settings change, keep user-facing validation errors and custom-provider configuration behavior aligned.
  • When provider construction changes, preserve worker-side module loading and loop-owned client construction.
  • When generation replacement changes, preserve candidate construction before settings commit, current-generation publication, lease drain, and failed-candidate cleanup.
  • When model discovery or account changes change, keep cache scope, per-provider failure reporting, safe diagnostic classification, catalog publication, and catalog status aligned.
  • When Codex catalog publication changes, keep the generation check, serialized publication, ready-file state, and retry error contract aligned.
  • When Codex app-server selection changes, keep the selected catalog snapshot, model fingerprints, local proxy settings, and native process lifecycle aligned.
  • When integration publication changes, keep catalog revision checks and configuration-lock ordering aligned with live admin changes.
  • When provider shutdown changes, preserve cancellation of discovery, pending construction, publication work, active lease draining, and unpublished-runtime cleanup.
  • When ASGI lifespan behavior changes, keep Uvicorn result events aligned with application startup and cleanup status.
  • When diagnostics changes, keep secret redaction, local-server limits, and safe provider failure messages aligned with the report contract.

Important failure modes

TriggerConsequenceGuard
A provider lacks a matching construction ownerThe provider cannot be built or may use the wrong adapterValidate that catalog IDs are covered exactly once by profiles, special adapters, or injected providers
A required credential or URL is missingProvider use fails with an unavailable configuration stateResolve and validate provider settings before client construction
Several requests resolve one uncached providerDuplicate clients or admission state could be createdShare one construction task per provider and API key
Candidate construction or settings commit failsThe old generation remains active while the candidate is cleaned upBuild before publication and retain failed unpublished runtimes for retry
A request holds a lease while its generation is retiredCleanup could close provider state still in useWait for active leases to drain before generation cleanup
A provider model query failsThat provider has no fresh metadata while other providers can update, and callers need a safe reason to diagnose the failureClassify wrapped causes into approved messages and carry them in refresh results and catalog status
A connected provider is removedIts old models could remain selectableRemove the provider from the cache and restart catalog publication
Catalog publication failsCodex cannot use a fresh catalog while HTTP can continueKeep catalog readiness separate from HTTP startup and report publication failure
A Codex process loses its pipe or does not acknowledge a requestThe session becomes unavailable and input is not resentFail pending calls safely, keep the process reserved until reaping, and close the owned process tree
An integration writes from an old catalog revisionAn external client can receive stale model settingsRecheck the generation and catalog revision while holding the shared publication order
Diagnostics reads a non-local server or unsafe error textA report could expose remote state or sensitive provider detailsLimit discovery reads to loopback and use redacted URL and failure summaries
Shutdown cleanup failsResources remain owned and the runtime cannot claim a clean closeKeep failed generations attached and retry cleanup

Key files

FileWhy to read it
src/…/bootstrap.pyShows the production composition root and the links between settings, providers, storage, services, and the runtime owner.
src/…/application.pyDefines process-wide startup, live configuration changes, service ownership, and shutdown.
src/…/asgi.pyDefines the ASGI boundary and lifespan result behavior.
src/…/config.pyDefines the boundary from provider descriptors and settings to shared provider configuration.
src/…/factory.pyShows how catalog provider IDs map to provider construction owners.
src/…/runtime.pyDefines lazy client creation, shared construction, and cleanup for one settings snapshot.
src/…/discovery.pyDefines provider selection for discovery and the safe messages used for model-list failures.
src/…/provider_manager.pyDefines generations, leases, discovery, catalog publication, replacement, failure status, and cleanup.
src/…/codex_app_server.pyShows how a catalog-selected model becomes an owned Codex process and Code session.
src/…/integration_service.pyShows how the current model catalog reaches external client configuration.
src/…/messaging_service.pyShows optional messaging startup, readiness gates, delivery ownership, and ordered cleanup.
src/…/codex_catalog.pyConnects the provider model inventory to the generated Codex catalog file.
src/…/diagnostics.pyDefines the safe local report for configuration, providers, integrations, harnesses, and discovery state.
src/…/paths.pyDefines the shared default path used for the Codex model catalog.