Protocol adaptation and streaming
The protocol adaptation layer lets clients use Anthropic Messages, Gemini GenerateContent, OpenAI Responses, or Chat Completions through one provider system. It keeps shared meaning for content, tools, reasoning, IDs, errors, and streams while rejecting data that has no safe form at the next protocol boundary.
Protocol boundary
Public protocol roles
The API exposes four client-facing contracts. Anthropic Messages, Gemini GenerateContent, OpenAI Responses, and Chat Completions each keep their own request validation, response shape, event names, and error format at the public edge. They share routing and provider execution after ingress. The Anthropic protocol adapter, Gemini protocol adapter, and Chat Completions protocol adapter describe the request and stream contracts for those protocols.
The provider choice is independent from the client protocol. A request can enter as Gemini content, Chat messages, Anthropic messages, or Responses input and leave through a different provider path. The selected adapter presents the result again in the client protocol, so clients do not need to know which upstream wire handled the request.
The following diagram shows the main shared and protocol-specific boundaries.
flowchart LR
A[Anthropic client] --> AM[Shared message meaning]
G[Gemini client] --> AM
R[Responses client] --> AM
C[Chat Completions client] --> AM
AM --> P[Provider execution]
P --> AM
AM --> A
AM --> G
AM --> R
AM --> C
References: src/…/transport.py• L190-337 src/…/client.py• L141-286 src/…/transport.py• L927-1139 src/…/transport.py• L142-297
Shared message meaning
The shared message meaning is the hand-off shape between public protocols and provider paths. It represents user and assistant text, images, function calls, function results, visible reasoning, and provider-owned reasoning data before an adapter renders them for a client or provider.
Conversion preserves meaning rather than copying field names. It keeps tool results tied to their call IDs, keeps content order where the destination supports it, and stops when the destination would lose a document, image, tool, reasoning value, or request control. Provider-managed tool history can use a protocol-specific replay form when the destination supports it; otherwise the adapter keeps only safe history context or rejects the conversion.
References: src/…/models.py• L14-163 src/…/models.py• L37-263 src/…/input.py• L50-102 src/…/provider_input.py• L41-104
Request conversion
Anthropic request conversion
Anthropic Messages requests can stay in their native form or move to a Chat, Gemini, or Responses provider request. System text becomes the destination's system or instruction field. User and assistant content becomes ordered destination items. Tool calls and results retain their relationship across the change in shape.
Native preparation applies provider controls and requires replayable provider-owned thinking history when that history must continue. Converted paths carry only controls with a safe target form. Tool constraints such as strict execution or deferred loading are rejected when a plain upstream function cannot enforce them. Unsupported fields, content blocks, or history are rejected before provider execution.
References: src/…/models.py• L81-139 src/…/native.py• L43-168 src/…/passthrough.py• L21-82 src/…/conversion.py• L40-101 src/…/provider_input.py• L44-151
Gemini request conversion
Gemini GenerateContent requests use contents with user or model turns. The adapter converts those turns into the selected Chat, Messages, or Responses request form. System instruction becomes system text or instructions. Inline images become the destination's supported image input. Function calls and function responses become the destination's tool call and tool result forms.
Gemini generation settings become shared sampling, token, tool, reasoning, and structured-output controls when the destination supports them. A setting with no safe equivalent produces a conversion error instead of a silent downgrade. Native Gemini also requires images to be inline base64 data and rejects controls that its wire cannot enforce.
Completed Anthropic WebSearch turns have a special replay path. When a signed Gemini record is present, the adapter restores the hidden function call and its matching results in the original order. It checks the search name, query, IDs, and result count before it restores provider history. Without a valid signed record, the web-tool blocks become plain history context instead of untrusted native function state.
References: src/…/native_request.py• L34-166 src/…/native_request.py• L319-485 src/…/native_request.py• L488-633 src/…/web_search_replay.py• L17-144
Responses request conversion
OpenAI Responses input is converted into the shared message meaning before model routing. Instructions become system text. Input message items become user or assistant content. Function calls and function outputs become tool-use and tool-result blocks. Reasoning records remain separate from ordinary text.
The provider request uses a stateless streamed form. It disables provider storage and asks for encrypted reasoning content so later history can preserve provider-owned state. The adapter encodes tool names for the Responses rules and carries model, limits, sampling, metadata, tools, tool choice, and reasoning effort only when the provider path can represent them. A request with no usable input or with unsupported fields stops before provider execution.
Anthropic history sent to a Responses provider uses function-call items, function outputs, and reasoning items so the provider can continue the same turn structure. Provider-managed web-tool blocks become bounded history context rather than ordinary function calls.
References: src/…/input.py• L50-189 src/…/provider_input.py• L44-107 src/…/provider_input.py• L110-160 src/…/provider_input.py• L171-315 src/…/provider_input.py• L318-375
Chat Completions request conversion
Chat Completions messages are a provider-facing form for text, images, tool calls, tool results, and reasoning controls. The adapter can accept native Chat input or translate Anthropic, Gemini, and Responses requests into a Chat request when the selected provider supports that form.
Provider profiles apply the controls that the target can represent. Tool names and provider extensions stay in the request boundary. Unsupported controls are rejected instead of being silently removed. Anthropic tool turns keep the assistant call before its matching tool results. Image-bearing tool results use a safe text marker plus a user image message when the Chat wire cannot place images in a tool message.
References: src/…/conversion.py• L224-323 src/…/conversion.py• L326-463 src/…/conversion.py• L466-504 src/…/conversion.py• L718-869
Content and tool semantics
Content block mapping
Text is the most portable content type. Images cross the boundary as URL or base64 input when the target supports that source. Documents, server-managed tools, annotations, and other native extensions remain protocol-specific unless the destination has a defined equivalent.
Anthropic server tools use a server_tool_use block and a matching web-search or web-fetch result block. Their local completion stream keeps the Anthropic message lifecycle intact and records the tool request in usage. Other provider paths either restore a validated native Gemini WebSearch turn or carry a bounded text description of the tool history.
Tool calls use a name, call ID, and structured input. Tool results use the same call ID and may carry text or images. The adapter preserves order and rejects a block when moving it would change the conversation or make the result impossible to use.
References: src/…/models.py• L14-74 src/…/server_tool_types.py• L1-10 src/…/server_tool_sse.py• L19-240 src/…/web_search_replay.py• L28-144 src/…/native_request.py• L338-412
Tool identity mapping
Tool identity is a cross-protocol contract. OpenAI-style targets have stricter name rules than Anthropic or Gemini. A shared name codec creates stable provider-facing aliases and restores the client name when output returns. The same mapping covers declarations, forced choices, earlier calls, and streamed results.
Chat streaming resolves fragments within each choice by tool index or ID. It buffers names, arguments, and extension data until the source choice ends. If the signals identify more than one call, or if a native envelope hides which call owns a fragment, the adapter fails instead of assigning the fragment to the wrong tool.
References: src/…/openai_tool_names.py• L11-123 src/…/tool_calls.py• L17-95 src/…/tool_calls.py• L192-487 src/…/tool_calls.py• L589-641
Reasoning and identity
Reasoning continuity
Reasoning has two forms in the shared layer: readable reasoning text and opaque provider-owned continuation data. The adapter keeps them separate. Readable text can become visible thought content. Opaque data can continue a provider exchange only when its origin is accepted.
Gemini exposes this continuity through thought parts and an opaque thoughtSignature. Anthropic thinking blocks and Responses reasoning items carry equivalent information on their own wires. Native conversion rejects foreign protected state, while gateway-owned replay data can restore the original provider state after its origin passes validation.
References: src/…/reasoning.py• L7-121 src/…/history_replay.py• L14-172 src/…/native_request.py• L92-107 src/…/provider_input.py• L205-280 src/…/passthrough.py• L50-82
Identifier continuity
IDs let clients match a response, output item, reasoning item, content block, and tool call over time. Each public protocol keeps its own response and event IDs. Tool call IDs remain linked through conversion, while adapters create new output IDs when the destination needs a different object type.
Model IDs can also encode the provider route. This lets the gateway advertise a client-safe model name and recover the selected provider without exposing provider-specific routing rules in the request body.
References: src/…/ids.py• L6-19 src/…/messages_stream.py• L104-130 src/…/gateway_model_ids.py• L5-51 src/…/response.py• L54-93
Server-sent event streams
Shared stream framing
The stream layer turns arbitrary network chunks into complete server-sent events before protocol logic reads them. Anthropic Messages and native Gemini streams use the same incremental framing rule. OpenAI SDK streams are read as typed provider events and then pass through the same recovery and presentation boundary.
The first chunk is read before a streaming response is returned. A failure before public output becomes an HTTP error. A failure after output starts becomes the active protocol's terminal failure form. The response owner also closes the remaining iterator when the stream ends, fails, or is cancelled. Anthropic emitters provide the terminal error event and ledger state needed to keep this boundary explicit.
References: src/…/decoder.py• L1-68 src/…/stream_contracts.py• L50-102 src/…/emitter.py• L12-62 src/…/response_streams.py• L42-213 src/…/wire.py• L29-59 src/…/transport.py• L56-80
Anthropic stream contract
Anthropic SSE is an indexed sequence of message, content-block, delta, and terminal events. Its validator checks event order, block indexes, block types, and terminal state. A ledger keeps text, thinking, and tool arguments separate until each block closes.
The ledger also tracks open blocks and accumulated output so it can close unfinished content safely and estimate usage when the upstream does not provide it. The same stream can be folded into one non-streaming Messages body. Incomplete tool JSON is not published as a successful tool call, and a truncated stream keeps only the visible state that can be represented safely. Native passthrough rewrites only the public model and preserves provider-owned event payloads after it checks the lifecycle.
References: src/…/stream_contracts.py• L18-174 src/…/ledger.py• L18-440 src/…/emitter.py• L18-62 src/…/sse_aggregation.py• L23-154 src/…/passthrough.py• L85-115
Gemini stream contract
Gemini output is a sequence of candidate parts followed by a finish reason. The Gemini adapter can render shared Chat output, validated Anthropic Messages events, or OpenAI Responses events as Gemini text, thought, function-call, and final candidate parts.
The adapter joins adjacent text with the same thought state, validates function arguments as JSON objects, maps provider stop states to Gemini finish reasons, and normalizes token usage. It emits an opaque replay signature at completion when later turns need provider-owned reasoning or tool metadata. Gemini non-streaming responses use the same public stream, then aggregate the events after completion.
References: src/…/response.py• L17-207 src/…/messages_stream.py• L14-158 src/…/responses_stream.py• L14-309 src/…/stream.py• L64-311
Chat Completions stream contract
Chat Completions chunks are normalized into shared text, reasoning, tool, usage, and completion state before another public protocol renders them. Native Chat requests can keep the provider's Chat wire, while translated requests use a protocol-specific output presenter.
The stream assembler buffers partial tool names and arguments until it can produce a valid call. It tracks identity by choice and by the strongest available tool signal. It maps provider finish reasons and usage into the selected public protocol. A native Chat stream retains opaque provider fields only when it can separate completed calls from the surrounding envelope. Otherwise it stops rather than emit an unsafe association.
References: src/…/tool_calls.py• L192-311 src/…/tool_calls.py• L313-487 src/…/tool_calls.py• L528-586 src/…/transport.py• L207-538 src/…/transport.py• L1187-1314
Responses stream contract
The Responses adapter builds an ordered public lifecycle from shared or Anthropic stream state. It opens output items for text, reasoning, and tools, sends deltas as content arrives, and closes each item before the final response event. A native Responses provider can instead relay its own lifecycle while rewriting public model metadata.
Function arguments are committed only when their JSON is complete. Responses reasoning and refusal events become the corresponding Anthropic content state. Provider encrypted reasoning is wrapped as gateway-owned replay data at item completion. Failed or incomplete upstream state becomes an incomplete or failed response rather than a successful item that clients cannot safely replay.
References: src/…/messages_stream.py• L104-392 src/…/provider_stream.py• L54-415 src/…/provider_stream.py• L418-437 src/…/native.py• L23-162 src/…/assembler.py• L32-243
Continuation recovery
Continuation recovery lets a retry extend a response after a provider failure without repeating text already released to the client. Bounded delivery evidence records the visible prefix, IDs, closed blocks, reasoning, tool calls, and usage. The recovery layer chooses restart, continuation, handoff, or stop from that evidence.
For eligible text output, overlap checking removes only the repeated prefix and keeps the public response identity. Structured output, opaque content, released tool calls, changed prefixes, or evidence beyond the retention limit stop recovery rather than create a synthetic result.
References: src/…/delivered_response.py• L16-186 src/…/stream_recovery.py• L10-106 src/…/continuation_overlap.py• L42-335 src/…/continuation_stream.py• L16-146
Error boundaries
Failure normalization
Provider execution uses one protocol-neutral failure model with a stable kind, status, message, and retry hint. Each public adapter maps it to its own error vocabulary. Anthropic uses an error envelope, Responses uses an error object or failed response event, and Gemini uses a Google-shaped error object with a mapped status name.
Diagnostics add bounded upstream detail only after they redact recognizable credentials and cap the displayed body. The boundary preserves the source failure for recovery and post-start terminal events. A provider failure is never presented as a clean stream end.
References: src/…/failures.py• L7-51 src/…/diagnostics.py• L11-19 src/…/diagnostics.py• L72-191 src/…/errors.py• L8-123 src/…/emitter.py• L38-49
Conversion rejection
Conversion errors mean that a requested feature has no safe target form. The API reports them as invalid requests before provider streaming begins. Typical causes include unsupported image sources, tool-name collisions, unresolved tool history, incompatible reasoning replay, unsupported generation controls, and provider-native events with no Gemini or Responses equivalent.
This distinction protects clients from receiving a successful-looking response that omitted content or changed a tool or reasoning contract. Native paths apply the same rule when a wire control cannot be enforced, such as native Gemini image detail or strict tool behavior. Signed WebSearch history is also rejected when its calls, IDs, query, or results do not match the visible turn.
References: src/…/conversion.py• L68-101 src/…/provider_input.py• L110-160 src/…/native_request.py• L34-77 src/…/native_request.py• L338-429 src/…/web_search_replay.py• L36-116
Change checklist
When these files or sections are changed, remember to consider these:
- When a content or tool type changes, update every request conversion direction and every stream writer. An ingress type needs a safe completion form on egress.
- When a tool-name rule changes, check declarations, forced choices, history, provider input, and output decoding as one identity contract.
- When reasoning fields change, preserve the split between readable text and provider-owned replay data. Check Anthropic thinking, Responses reasoning, and Gemini thought signatures together.
- When replay metadata changes, keep provenance checks aligned with the destination provider and model. For WebSearch, also keep hidden names, query values, call IDs, and result blocks aligned.
- When a generation control changes, check Gemini validation, all destination request forms, provider capability limits, and the public conversion error.
- When a stream event changes, check chunk framing, event order, block closure, finish reasons, sequence or response IDs, aggregation, and cleanup for both SSE and SDK streams.
- When function arguments change, check partial JSON, completed snapshots, declared-tool checks, call-ID reuse, and all public output forms.
- When failure mapping changes, update pre-start errors, post-start terminal events, recovery context, redaction, and the public status mapping.
- When continuation recovery changes, check delivered-prefix capture, overlap removal, response identity, replacement IDs, handoff behavior, and retention limits.
- When token accounting changes, check streamed usage, aggregated Gemini output, reasoning-token subtraction, cached input, and tool-result images.
Important failure modes
| Trigger | Consequence | Guard |
|---|---|---|
| A request contains data with no safe destination form | Conversion stops before provider execution | Target-specific adapters reject unsupported content and controls |
| A tool name collides after aliasing or namespace flattening | Provider output cannot be matched to the client tool | The shared codec and tool adapter reject ambiguous mappings |
| Gemini history has an orphaned function response or foreign thought signature | The next request may use the wrong call or provider state | History normalization checks call links and accepts only gateway replay bundles |
| A signed Gemini WebSearch turn has changed calls, query, IDs, or result count | Native history may execute or replay the wrong search | WebSearch restoration checks the signed call sequence before lifting it into Gemini parts |
| A provider returns an undeclared function or invalid JSON arguments | The public client receives an unsafe tool call | Stream adapters check declared names and require a JSON object |
| A native stream emits an unsupported block, event, or stop reason | The public protocol cannot represent the result | The source validator and destination adapter reject the stream |
| A wire response is not an SSE stream or ends without its terminal event | The client receives an incomplete or invalid response | Each transport checks the media type and requires a protocol stop event |
| A wire chunk ends inside an SSE event | The adapter may lose or split a protocol event | The incremental decoder buffers until an event boundary and flushes at finish |
| A stream fails before its first output | The client cannot receive a valid partial response | The API returns a pre-start HTTP error |
| A stream fails after output starts | An HTTP error cannot replace sent SSE data | The active protocol emits its terminal failure form |
| A Gemini stream returns grounding data, citations, or unsupported native parts | The selected client protocol cannot represent the result | The Gemini stream rejects data without a client-safe equivalent |
| A continuation changes already released text | The public response contains conflicting prefixes | Overlap checking stops recovery on a changed prefix |
| A provider-owned reasoning record comes from an incompatible origin | The next provider may receive invalid continuation state | Replay provenance must match the destination before restoration |
| An upstream error body contains credentials or exceeds the display cap | Diagnostics can leak secrets or overwhelm the client | Shared diagnostics redact known credential forms and cap bodies at 16 KiB |
| A Chat tool fragment has no usable index or ID, or matches more than one call | Partial arguments can be assigned to the wrong tool | The identity resolver raises an upstream failure and stops assembly |
| A native Chat envelope contains opaque fields while calls finish separately | The adapter cannot prove which fragments belong to completed calls | Native grouping rejects the envelope instead of guessing |
Key files
| File | Why to read it |
|---|---|
| src/…/transport.py | Shows how one Messages upstream supports Anthropic, Gemini, Chat, and Responses output contracts. |
| src/…/passthrough.py | Shows native Messages validation, SSE relay, and recovery when the upstream already uses the public protocol. |
| src/…/wire.py | Shows Messages SSE framing and upstream error classification. |
| src/…/client.py | Shows how each public request style becomes a native Gemini request and selects its output presenter. |
| src/…/transport.py | Shows native Gemini stream ownership, history replay, retry, and post-start failure handling. |
| src/…/stream.py | Shows how Gemini parts, tool calls, thought data, finish reasons, and usage become public output. |
| src/…/transport.py | Shows Chat request conversion, tool identity handling, typed chunk assembly, and output selection. |
| src/…/transport.py | Shows Responses request preparation, lifecycle presentation, failure mapping, and continuation. |
| src/…/models.py | Defines the shared Anthropic-facing content, tool, reasoning, request, and response vocabulary. |
| src/…/conversion.py | Shows how Anthropic turns become Chat messages while preserving tool dependencies, reasoning, and multimodal results. |
| src/…/server_tool_sse.py | Shows how completed local web-tool operations become valid Anthropic message and SSE blocks. |
| src/…/emitter.py | Shows common Anthropic SSE serialization, stop-reason mapping, and terminal error frames. |
| src/…/ledger.py | Shows how Anthropic blocks stay ordered, accumulated, and complete during streaming. |
| src/…/provider_input.py | Shows how Anthropic history becomes a Responses provider request with reasoning, tool items, and safe field checks. |
| src/…/provider_stream.py | Shows how Responses output items, refusals, reasoning, tool arguments, usage, and failures become Anthropic SSE. |
| src/…/native_request.py | Shows native Gemini request construction from Chat, Messages, and Responses inputs. |
| src/…/web_search_replay.py | Shows validated restoration of signed Anthropic WebSearch history into Gemini function turns. |
| src/…/tool_calls.py | Shows choice-scoped Chat tool fragment identity, buffering, sealing, native grouping, and client-name restoration. |
| src/…/diagnostics.py | Defines credential-safe, bounded upstream error detail for public diagnostics. |
| src/…/errors.py | Maps neutral failures to Anthropic error types, status codes, and envelopes. |
| src/…/decoder.py | Shows how split wire chunks become complete SSE events. |
| src/…/history_replay.py | Defines provenance-aware reasoning replay across protocols. |
| src/…/failures.py | Defines failure categories shared by execution, recovery, and wire adapters. |
| src/…/response_streams.py | Shows first-chunk handling, stream ownership, cleanup, and post-start failures. |
| src/…/stream_recovery.py | Defines safe restart, continuation, handoff, and stop decisions. |