HTTP API gateway
The HTTP API gateway is the ASGI-facing boundary for Free Claude Code. It accepts compatible client requests, checks access and input shape, selects a configured provider path, and returns a complete result or a live event stream while it protects resources across the request lifetime.
Gateway composition
The gateway is assembled as one ASGI application. The production composition root supplies shared runtime services to a FastAPI app that joins public inference with local administration, chat, code-session, and web-tool capabilities. A runtime wrapper owns process startup and shutdown so route handlers do not construct or close process-wide services.
flowchart LR
Client[HTTP client] --> ASGI[ASGI boundary]
ASGI --> Lifetime[Request lifetime controls]
Lifetime --> FastAPI[FastAPI gateway]
FastAPI --> Public[Public protocol routes]
FastAPI --> Local[Local capability routes]
Public --> Services[Explicit runtime services]
Local --> Services
Services --> Providers[Provider runtime]
The public generation surfaces are Anthropic Messages, OpenAI Responses, OpenAI Chat Completions, and Gemini-compatible generation. The Anthropic-compatible request handling page explains the Claude-facing paths. The administrative and web-tool API capabilities page explains local control and tool surfaces. The protocol adapters page explains request and response translation.
ASGI lifecycle
The ASGI wrapper sends normal traffic to FastAPI and sends startup and shutdown events to the application runtime. Provider resources, messaging resources, account managers, and other process-owned services therefore start and close as one graph. A startup failure prevents readiness. An incomplete shutdown remains a failure so the owner can retry cleanup.
References: src/…/asgi.py• L9-63 src/…/bootstrap.py• L30-68 src/…/application.py• L97-175 src/…/app.py• L51-62
Service boundary
Routes receive an ApiServices boundary from application state. This boundary exposes request-runtime access, administrative actions, task control, web-tool access, and optional code-session access without exposing provider or messaging objects directly to HTTP code. The administrative side also owns configuration changes, provider checks, model refresh, connected-account login, and client-integration actions.
The boundary keeps optional local capabilities explicit. A deployment can omit code-session support without changing the public inference contracts, while all route groups still use the same service boundary.
References: src/…/ports.py• L20-120 src/…/dependencies.py• L15-43 src/…/bootstrap.py• L57-68
Request routing
The route layer maps HTTP methods and paths to product flows. Access checks and typed input parsing run before the flow handler. The handler then selects a provider path or a local capability, and the response uses the protocol named by the URL.
The shared generation-path classifier keeps authentication, request lifetime, request outcomes, and error selection aligned. It recognizes Gemini actions under /v1beta/models/... and also recognizes SDK calls under /v1/models/... when the URL ends in a Gemini generation action, without confusing ordinary OpenAI model paths with Gemini requests.
| Surface | Main purpose | Access boundary |
|---|---|---|
POST /v1/messages | Anthropic-compatible message generation | Proxy token auth when enabled |
POST /v1/responses | OpenAI Responses-compatible generation | Proxy bearer auth when enabled |
POST /v1/chat/completions | OpenAI Chat Completions-compatible generation | Proxy bearer auth when enabled |
POST /v1beta/models/{model}:generateContent | Gemini-compatible JSON generation | Gemini credential auth when enabled |
POST /v1beta/models/{model}:streamGenerateContent | Gemini-compatible streamed generation | Gemini credential auth when enabled |
POST /v1/messages/count_tokens | Anthropic-compatible token count | Proxy token auth when enabled |
GET /v1/models and GET /muse-code/models | Advertise model choices for client types | Proxy bearer auth when enabled |
GET / and /health | Report service state or health | Root is protected; health is public |
POST /stop | Stop managed work | Proxy bearer auth when enabled |
/admin and /admin/api/* | Local configuration, provider checks, account login, model refresh, and client integrations | Loopback client and local origin |
| Additional local route groups | Chat, code-session, and web-tool capabilities | Local application error policy |
The gateway answers HEAD and OPTIONS probes for the main public paths. Model lists are views of the same routable inventory, so a client sees model names and capabilities that the gateway can advertise for its protocol. Gemini resource paths place the model and generation action in the URL. The request body supplies the generation settings but cannot replace that URL model.
The local admin surface presents provider status, custom provider configuration, API-key editing, model refresh, connected-account login, and setup actions for supported client integrations. Its browser UI also tracks provider startup, model discovery, local endpoint reachability, integration updates, code-session availability, and server reconnect state. These controls change local configuration or integration files rather than public provider routing. Versioned admin assets are served with fixed media types so browsers do not depend on the host MIME registry.
Model selection
The model router turns a public model name into a provider and provider-model target. It supports configured aliases, explicit provider/model references, gateway model identifiers, and ordered fallback targets. The routed request keeps the public model identity for the response while using the provider model for execution.
The model catalog provides client-specific views of the same routable inventory. Claude views include stable shortcuts and gateway model IDs. Messages views can expose native provider identifiers. Responses views expose provider references and capability data for reasoning, input types, context, output, and retry behavior. The Muse view reuses the Responses-facing routable IDs and adds native picker metadata for context limits, output limits, and reasoning support. Gemini requests use the model named in the resource path and then enter the same provider routing system. The admin model picker uses the shared discovered inventory and reports providers that could not refresh.
The router also carries the reasoning preference for the selected route. Generation-owned model metadata stays with the retained lease, so execution uses the model information selected for that request even if later settings change.
References: src/…/routing.py• L29-58 src/…/routing.py• L61-108 src/…/routing.py• L124-180 src/…/model_catalog.py• L30-168 src/…/model_catalog.py• L171-277 src/…/admin_routes.py• L502-527 src/…/routes.py• L69-130
Request dependencies
Proxy authentication is an optional token gate controlled by current settings. When enabled, most public routes require a bearer token. The Anthropic Messages path also accepts the token in x-api-key. Gemini accepts the supported Google-style header or query credential forms and rejects repeated key credentials.
Administrative access uses a separate boundary. The client address and browser origin must both be local. Provider lookup uses the retained request lease, which keeps the provider choice aligned with the settings and model metadata used by the active request.
References: src/…/dependencies.py• L20-157 src/…/admin_routes.py• L111-125 src/…/ports.py• L38-65
Local capability surfaces
The local admin surface joins server control with client setup. It lets an operator view provider and model state, edit provider access, test local endpoints, manage connected accounts, refresh models, and update supported editor or desktop integrations. Provider checks distinguish model discovery from local endpoint reachability, so an available server and a usable model catalog remain separate observations.
The browser groups these actions into provider, model configuration, messaging, integration, and code-session views. It polls startup and integration state, refreshes model choices after account or provider changes, and waits for a new server instance after a listening-address change. Integration actions report the files or settings they will change and tell the operator when the target client must reload or restart.
References: src/…/admin_routes.py• L111-183 src/…/admin_routes.py• L196-527 src/…/index.html• L30-239 src/…/admin.js• L122-242 src/…/admin.js• L637-753 src/…/admin.js• L1568-1791 src/…/admin.js• L1986-2391
Request lifetime
The request lifetime is the period from accepted input through complete response delivery or client disconnect. For client-owned operations, the gateway treats the HTTP connection as the owner of the work. A disconnect cancels the application task instead of allowing provider work or another long operation to continue without a consumer.
sequenceDiagram
participant C as Client
participant G as Gateway
participant L as Runtime lease
participant P as Provider
C->>G: Send request
G->>L: Acquire matching runtime generation
G->>P: Start routed operation
P-->>G: Result or stream chunks
G-->>C: JSON or SSE response
alt Operation completes
G->>L: Release lease
else Client disconnects
G-->>P: Cancel application work
G->>L: Release lease
end
Client disconnects
The lifetime middleware watches ASGI receive events while client-owned work runs. It covers public generation requests, token counting, model-list reads, and the admin folder picker. When it sees http.disconnect, it cancels the application task and waits for cleanup. When the application finishes first, it cancels the receive watcher so no background task remains.
The middleware preserves cancellation during cleanup. This keeps a disconnected request from becoming a false success and lets downstream providers stop with the request that owns them.
References: src/…/request_lifetime.py• L7-98 src/…/request_lifetime.py• L101-135
Runtime lease ownership
An inference route acquires one request-runtime lease before it creates its handler. The lease fixes the settings, provider generation, and model information used by that request. A complete response releases the lease before the route returns. A stream binds release to response cleanup, so the provider generation remains alive until the body is consumed or closed.
References: src/…/routes.py• L108-162 src/…/routes.py• L170-278 src/…/response_streams.py• L169-190
Request outcomes
Inference outcome tracking observes the full HTTP delivery path without changing the response body. It records success, failure, cancellation, or an incomplete response after delivery and cleanup. It also reads protocol failure events from streams so a request that returns an error event is not recorded as a success.
Each request receives an opaque ID at ingress. The gateway places request-id on responses and adds x-request-id for Responses, Chat Completions, model-list, and Gemini paths. The same ID links route logs, provider traces, and terminal stream decisions without exposing request content.
References: src/…/request_outcomes.py• L20-134 src/…/request_ids.py• L12-102
Validation and error handling
Validation and error handling preserve the wire contract selected by the URL. Typed request models validate ordinary body shapes at ingress. Protocol handlers add checks for rules that schemas cannot express. Known failures become protocol-shaped errors, while unexpected failures are logged with safe detail and become server errors.
Input validation
Typed request models reject malformed body shapes before provider execution. Native provider paths validate their compatible request shape before execution. Product flows add business checks such as requiring at least one message, rejecting unsupported server tools, rejecting duplicate JSON fields, and enforcing Gemini URL actions and query options.
The compatible Messages path rejects a system role placed inside the messages list. It returns a stable error code and an update message so current Claude Code clients can retry with the system content in the top-level system field. Token counting applies the same boundary. Known Claude auto-mode classifier requests are recognized by their request shape so the gateway can provide the stop hint expected by that classifier.
Validation logs record request shape metadata such as locations, roles, content kinds, block types, block keys, and tool names. They do not record message text or credential values. Gemini and Chat Completions validation use their own error payloads while admin configuration keeps FastAPI's structured validation details.
References: src/…/app.py• L104-151 src/…/request_errors.py• L33-66 src/…/routes.py• L60-109 src/…/routes.py• L170-243 src/…/routes.py• L327-370 src/…/inline_system_turns.py• L8-41 src/…/detection.py• L9-34 src/…/validation_log.py• L6-46
Protocol errors
The gateway distinguishes an ordinary application error from a terminal provider-execution error. Ordinary errors use the selected protocol's normal status and payload. Terminal execution errors also carry a no-retry signal because the provider attempt has reached a final decision.
The shared error mapper selects the public error shape from the classified route. Messages uses Anthropic errors. Responses and Chat Completions use OpenAI-style errors. Gemini uses Gemini errors. Unexpected API failures return a server error and log only the exception type unless trace logging is explicitly enabled; safe exception text is still used for the client-facing detail.
Code-session failures use a local JSON shape with status values that reflect missing, conflicting, invalid, unavailable, or failed sessions. A general exception handler applies the same path-based choice when an error escapes the product flow.
| Code-session error | HTTP status |
|---|---|
| Not found | 404 |
| Conflict | 409 |
| Validation | 400 |
| Unavailable | 503 |
| Other code error | 500 |
References: src/…/app.py• L64-202 src/…/request_errors.py• L38-113 src/…/response_streams.py• L193-233 src/…/chat.py• L87-116 src/…/gemini.py• L82-112
Response streaming
Streaming is the gateway's live delivery path for provider output. The internal execution path produces an asynchronous event stream. The gateway reads the first chunk before it starts the response, which allows it to return a normal HTTP error when execution fails before delivery begins. After delivery starts, it sends protocol events and translates later failures into the selected wire format.
Streaming response ownership
Managed streaming responses close the body iterator and release the retained runtime resource exactly once. Cleanup runs when the stream ends, the client disconnects, or the response is cancelled. Cleanup failures become trace events, while cancellation remains control flow so cleanup does not hide the request cancellation.
References: src/…/response_streams.py• L52-190
Wire-format streams
The Messages API emits Anthropic server-sent events and can aggregate the same internal stream into one JSON message for non-streaming clients. The Responses API emits OpenAI Responses-style server-sent events and requires streaming. Chat Completions emits OpenAI Chat Completions-style server-sent events or aggregates them into JSON for non-streaming clients. Gemini emits Gemini-compatible server-sent events or aggregates them into JSON for non-streaming generation.
A stream that ends before its first chunk becomes a regular error response. A failure after streaming starts becomes a terminal event for that wire format. The public stream also keeps protocol identity and continuation replay data stable when provider recovery or replacement changes the internal attempt. If recovery ends without adding output, the gateway emits a terminal failure instead of publishing an unusable replay.
References: src/…/messages.py• L88-224 src/…/chat.py• L57-94 src/…/gemini.py• L53-80 src/…/response_streams.py• L236-493 src/…/stream_delivery.py• L42-172 src/…/replay_finalization.py• L42-99
Client tool delivery
The public stream withholds incomplete client tool calls until their complete group is safe to publish. This prevents a client from acting on partial tool arguments. If an attempt fails or ends with an unfinished call, the gateway removes that incomplete call from the public snapshot and emits the protocol's terminal failure instead.
References: src/…/stream_delivery.py• L126-166 src/…/stream_delivery.py• L302-421
Change checklist
When these files or sections are changed, remember to consider these:
- Changes to generation path classification must stay aligned across routing, authentication, lifetime ownership, request outcomes, request-ID headers, and error payload selection. Gemini action suffixes must remain distinct from ordinary OpenAI model paths.
- Changes to inference or stream paths must preserve lease release after normal completion, first-chunk failure, client cancellation, and response close.
- Changes to public model identifiers must update model-list views and model-router decoding so every advertised ID remains callable. Provider metadata must stay aligned with the selected generation.
- Changes to
ApiServicesmust be reflected in the production ASGI composition boundary and every route dependency that consumes it. Optional code-session support must remain explicit. - Changes to admin provider configuration must keep masked API-key references, credential checks, custom-provider mutations, and the config apply contract aligned.
- Changes to admin provider presentation must keep each provider card's asset reference aligned with its served asset, fixed media type, and source record. Visual assets must remain separate from provider routing identifiers.
- Changes to admin configuration or integration actions must preserve loopback access, no-store account state, startup polling, and reconnect behavior after a listening-address change.
- Changes to Gemini URL actions, query credentials, or error handling must stay aligned with the Gemini handler, protocol adapter, and shared generation-path classifier.
- Changes to inline system-turn handling must preserve the error code and the client recovery message for both Messages generation and token counting.
- Changes to Chat Completions request or response behavior must stay aligned across the route, handler, OpenAI error payloads, request-ID headers, non-stream aggregation, and complete tool-call delivery rules.
- Changes to stream recovery, replay sealing, or client tool calls must preserve public IDs, sequence values, terminal frames, released-output continuity, and incomplete-call filtering across all supported wire formats.
Important failure modes
| Trigger | Consequence | Guard |
|---|---|---|
| Missing or invalid token while proxy auth is enabled | Protected public route returns 401 before provider work starts | Protocol-specific auth dependencies check the accepted credential forms and compare tokens safely |
Repeated Gemini key query credentials | Request is rejected instead of choosing one ambiguous credential | Gemini auth rejects more than one key value |
| Non-local client or origin calls the admin surface | Admin page or API returns 403 | The loopback admin check validates both client host and origin |
| Provider generation changes during active work | The request could use resources that no longer match its settings | The route holds one runtime lease until response cleanup |
| Client disconnects during client-owned work | Provider work could continue without a client and leak request resources | Lifetime middleware cancels the application and waits for cleanup |
| Application finishes before the disconnect watcher | The watcher could remain an orphan task | The middleware cancels and waits for the watcher |
Inline system role appears inside Messages messages | The request fails or an older Claude Code client cannot recover with its current request shape | The gateway returns mid_conversation_system_not_supported and directs the caller to use top-level system or update Claude Code |
| Stream fails before its first event | The client may receive an empty or invalid event stream | The gateway prefetches the first chunk and returns a normal error |
| Stream fails after events have started | The client may wait forever or retry a completed provider attempt | The stream emits a terminal protocol event or a no-retry terminal response |
| Recovery publishes a draft continuation replay that does not match released output | A later provider turn can receive invalid history or expose the wrong signed state | The public stream seals replay data against the delivered prefix before publication |
| Recovery ends without adding output | The client receives a continuation result that cannot complete the answer | Stream finalization raises a non-retryable upstream failure |
| Validation logging records raw request content | Sensitive prompt data can enter diagnostic logs | Validation summaries record shape and names rather than content values |
| Gemini model or action is accepted from the body instead of the URL | The client can receive a response for a different resource than requested | The route validates the resource suffix and rejects a body model field |
| Incomplete client tool call reaches the public stream | A client can act on partial or invalid tool arguments | The delivery layer holds call groups until completion and filters unfinished snapshots |
| API-key rows use stale references or duplicate values | A provider credential update can reject valid edits or change the wrong entry | The editor sends credential revisions and row references and rejects duplicate entries |
| Provider logo reference or source record is missing | The admin provider card loses its visual identity or attribution | The asset path, fixed media type, and source record change together |
| Admin settings change the listening address without a reconnect path | The browser reports a failed apply even though settings were saved | The admin UI waits for a new server instance and offers reconnect or navigation |
| Connected-account login completes without model refresh | The admin picker and downstream clients show stale model choices | Account completion refreshes model options and shared startup status |
| A code-session event belongs to an old epoch or revision | The UI can show stale output or submit a turn against changed state | The code-session client checks epoch, revision, cursor, and operation identifiers |
| Admin config conflicts with a newer saved revision | A save is rejected instead of silently overwriting current settings | The config endpoint returns 409 and the UI reloads before another save |
| Admin asset version or allow-listed filename is invalid | The browser receives 404 and cannot load the matching UI asset | The asset route checks the package version and served asset set |
Key files
| File | Why to read it |
|---|---|
| src/…/bootstrap.py | Shows the production composition of runtime owners and API services. |
| src/…/asgi.py | Shows how HTTP traffic and process lifespan share one ASGI boundary. |
| src/…/app.py | Defines the FastAPI application, middleware, route inclusion, and top-level error policy. |
| src/…/routes.py | Maps public paths to request flows and binds runtime leases to responses. |
| src/…/generation_paths.py | Defines the shared classification of public generation protocols. |
| src/…/inline_system_turns.py | Defines the client-facing boundary for system turns inside Messages requests. |
| src/…/detection.py | Defines request-shape detection for known Claude auto-mode classifier calls. |
| src/…/dependencies.py | Defines settings, retained-generation provider resolution, service, and public auth boundaries. |
| src/…/routing.py | Explains how public model names become provider execution targets. |
| src/…/model_catalog.py | Shows how the shared model inventory becomes protocol-specific and native client choices. |
| src/…/messages.py | Shows Anthropic-compatible request routing, shared execution, JSON aggregation, and web-tool integration. |
| src/…/chat.py | Shows Chat Completions request routing, shared execution, JSON aggregation, and OpenAI error delivery. |
| src/…/gemini.py | Shows Gemini generation through the shared provider execution and stream boundary. |
| src/…/request_lifetime.py | Defines cancellation ownership for client-owned operations and coordinated task cleanup. |
| src/…/request_outcomes.py | Defines final inference outcome recording after HTTP delivery. |
| src/…/request_ids.py | Defines ingress correlation and protocol-specific response ID headers. |
| src/…/request_errors.py | Defines shared application error mapping and safe API failure logging. |
| src/…/response_streams.py | Defines first-chunk handling, stream cleanup, and terminal response behavior. |
| src/…/stream_delivery.py | Defines public event projection, recovery continuity, replay sealing, and wire termination. |
| src/…/replay_finalization.py | Defines how continuation replay data is matched to output already released to the client. |
| src/…/admin_routes.py | Defines the local-only administrative surface, provider checks, asset delivery, model refresh, account actions, and client integrations. |
| src/…/index.html | Shows the admin views for providers, configuration, messaging, integrations, and code sessions. |
| src/…/admin.js | Shows how the admin UI loads state, edits provider credentials, tracks startup, refreshes models, manages integrations, and reconnects after a restart. |
| src/…/code_sessions.js | Shows the local code-session library, live event feed, turn controls, prompts, and session state checks. |