Introducing a new way to review your PRs.

Read the announcement

alishahryar1 / free-claude-code

Use Claude Code, Codex, Antigravity, VSCode, Pi, and OpenCode (and 6 other harnesses) for free (1.3B+ free tokens) from your terminal, app, IDE, or phone, and now from the browser with native browser sessions (multi-harness + multi-model) like OpenClaw (voice supported + ToS friendly)

View on GitHub

28 documents · Updated Nov 20, 2026

docs / core-request-and-stream-resilience

Core request and stream resilience

The core resilience layer keeps one client request coherent when an upstream rejects saved state, stops a stream, or returns output again after a retry. It carries safe native history forward, records only what the client received, removes repeated recovery output, and stops when the result can no longer be trusted. It serves the shared protocol adapters and provider integrations rather than changing the meaning of any one public API. See protocol adapters, Anthropic protocol adapter, Chat Completions protocol adapter, and provider integrations for the surrounding boundaries.

Continuity model

The system treats request continuity as a contract between request history, public delivery, and upstream recovery. A retry is safe only when the next upstream request can represent the delivered prefix and the public response can keep one valid identity.

This layer separates three kinds of state:

  • Request history carries conversation input and provider-owned replay records.
  • Delivered response state records output that the client actually received.
  • Recovery state controls hidden retries, visible continuation, tool handoff, or final failure.

The separation prevents a provider retry from silently replacing public output. It also lets each protocol adapter project the same delivered prefix into its own request and event format.

Continuity evidence

Continuity evidence is request-local state about output already released to the client. It records readable text, thinking, completed tool calls, response headers, closed blocks, progress, and a protocol-shaped continuation seed.

The seed keeps the delivered items in the client protocol. The next request can then turn that seed into the selected upstream format without rewriting the saved client conversation. This separation lets the gateway recover a response while the normal history remains owned by the client and harness.

References: src/…/stream_recovery.py• L12-60 src/…/delivered_response.py• L49-68 src/…/delivered_response.py• L462-517

Recovery boundaries

Recovery has two boundaries. Before public content is released, a retry can replace the hidden attempt. After content is released, a retry must continue or hand off from the recorded prefix; it cannot silently replace what the client saw.

The decision also depends on output safety. Plain text and readable thinking can continue when the native prefix is available. Structured output, opaque provider state, unsafe events, and released tool calls can require a stop or a tool handoff instead.

References: src/…/stream_recovery.py• L63-145 src/…/delivered_response.py• L28-46 src/…/delivered_response.py• L530-542

History replay finalization

History replay carries provider-owned state across turns without treating every provider signature as portable. The core stores the native record with its source provenance, then projects it for the destination protocol; continuation records add the public echo only after the client-visible response is known.

Native replay records

Native replay records preserve the upstream item and its origin. The destination can restore the native item when the provider, protocol, endpoint, and connection match. Otherwise it uses readable context when that is safe, and it does not send foreign opaque state as if it were native.

History preparation works on a fresh request copy. It also keeps source paths so a later provider correction can target the original history item after projection filters or reshapes the request. Native Messages passthrough follows the same rule: it restores only FCC-owned replay state and rejects a record from a different provider or model.

References: src/…/history_replay.py• L38-74 src/…/history_replay.py• L620-745 src/…/passthrough.py• L49-81

Replay carrier finalization

A continuation replay carrier starts as a draft with native context but no public echo. The stream finalizer seals that draft against the delivered response before it is published, so the next turn can compare the saved native output with the exact public assistant parts.

Finalization follows this sequence:

  1. Find replay carriers only in known reasoning fields.
  2. Read the delivered continuation seed.
  3. Convert the seed into the carrier's native Gemini context when required.
  4. Store the public echo in a sealed carrier and replace the draft before publication.

If the public prefix cannot be retained, finalization fails instead of creating unusable history. Sealed values are reused within the response so one carrier is not encoded more than once.

References: src/…/replay_finalization.py• L20-47 src/…/replay_finalization.py• L49-98 src/…/continuation_request.py• L221-236

Association resolution

Some Messages output arrives as separate readable and redacted blocks. Association resolution joins the anchor and final metadata into one replay record at the anchor's position. It rejects duplicate, conflicting, or reversed parts because those states could attach provider data to the wrong assistant content.

The result is stable on repeated preparation. Unpaired data remains usable as the available record rather than being guessed into a complete association.

References: src/…/history_replay.py• L352-396 tests/…/test_history_replay.py• L230-328

Delivered response state

Delivered response state is the bounded record of the public stream, not a second conversation history. It lets recovery reason from what the client actually received even when the upstream sent cumulative snapshots, split deltas, or protocol-specific terminal events.

Public prefix tracking

The tracker folds each public event into protocol-neutral evidence while retaining enough wire shape to rebuild a continuation. It joins text and thinking deltas, tracks item and block closure, keeps tool calls separate from ordinary text, and preserves response metadata needed by the public lifecycle.

The snapshot exposes four key results:

ResultPurpose
Readable text and thinkingDefine the visible prefix and overlap target
Continuation seedRebuild the delivered assistant turn for a new upstream
Call and closure stateDecide between continuation, handoff, and stop
Safety reason and progressReject unsafe recovery and detect a no-op retry

Each supported public wire has its own event shape, but the recovery decision uses the same evidence. Responses text contracts such as structured output and logprobs remain unsafe when synthetic continuation could change their meaning.

References: src/…/delivered_response.py• L94-143 src/…/delivered_response.py• L145-234 src/…/delivered_response.py• L264-517

Retention limit

Recovery evidence has a shared 1 MiB retention limit. When the limit is reached, the tracker drops its reconstructable items and marks the prefix unsafe. The healthy stream still reaches the client, but a later failure cannot pretend that a complete final snapshot or safe continuation exists.

The same limit bounds replay carriers and unpublished overlap frames. Tool-call buffers use a separate 16 MiB pending-output and 65,536-record limit because they hold source events until a call reaches an authoritative completion point. A limit failure is terminal rather than a reason to emit a partial call.

References: src/…/recovery_limits.py• L1-3 src/…/delivered_response.py• L70-87 src/…/continuation_overlap.py• L449-470 src/…/tool_call_buffer.py• L10-49

Stream protocol state

Stream protocol state protects the public lifecycle before recovery logic sees an event. It validates event order, preserves public identity, and keeps incomplete tool data private until the source declares a valid completion.

Native Messages lifecycle

The native Messages state machine requires one message start, valid content block lifecycles, a supported stop reason, and one message stop. It preserves provider event payloads while assigning dense public block indexes after source filtering. The same validation supports native passthrough and conversion into the Responses lifecycle.

Tool input remains private while JSON fragments arrive. A completed tool block receives either its streamed JSON or its eager start input, and malformed or mixed input becomes an upstream failure instead of a client-visible call.

References: src/…/native_stream.py• L31-229 src/…/native_stream.py• L232-294 src/…/messages_stream.py• L172-228

Tool call completion

Tool call completion is source-owned evidence. The public presenter may format a call, but it does not decide that the call is complete from a display event alone. Buffers retain call fragments in source order and release them only when the source stop event or authoritative final snapshot validates the identity and arguments.

The shared rule covers several wire shapes:

  • Messages waits for a valid stop reason and a closed tool block.
  • Chat choices resolve fragments by choice plus index or call ID, then seal calls only for successful choice termination.
  • Responses items wait for an authoritative completed item and reject changed identities, indexes, or arguments.

Unfinished calls are discarded during continuation. Completed calls can instead cause a safe handoff so the client receives one usable tool turn without another upstream request.

References: src/…/tool_calls.py• L22-136 src/…/tool_calls.py• L47-177 src/…/tool_calls.py• L43-145 src/…/stream_output.py• L11-25

Responses output lifecycle

The Responses stream uses an output ledger to reserve output positions, track active text, reasoning, and tool blocks, and commit complete items in order. Its event builder gives every event a rising sequence number, while its relay keeps the response ID and terminal status stable across provider output and synthesized failure.

Native Messages output can use this lifecycle without going through a Chat presenter. Native blocks map to Responses message, reasoning, or client tool items. Unsupported provider extensions fail before they can create a misleading public item.

References: src/…/ledger.py• L17-89 src/…/completion.py• L12-124 src/…/event_builders.py• L11-50 src/…/native.py• L68-158

Stream recovery

Stream recovery changes an upstream attempt without changing the public response contract. It uses delivery state to choose a hidden restart, a visible continuation, a completed-tool handoff, or a terminal error.

This diagram shows how an upstream failure moves through the shared evidence and recovery decisions.

flowchart LR
    A[Upstream stream] --> B{Public content released?}
    B -->|No| C[Hidden restart]
    B -->|Yes| D[Delivered prefix]
    D --> E{Prefix safe and native?}
    E -->|Yes| F[Continuation request]
    E -->|Completed calls| G[Tool handoff]
    E -->|No| H[Terminal failure]
    F --> I[Overlap filtering]
    I --> J[One public response]
    G --> J

Recovery assessment

The recovery assessment is independent of the attempt budget. A normal upstream stop ends recovery. A failure before content can restart only when retryable budget remains. A released tool call can hand off only when the call is complete and the source and prefix both allow handoff.

For text or thinking continuation, the assessment requires a non-empty safe prefix and a continuation seed. A structured contract, an unsafe reason, missing native prefix, or an empty retry becomes a final stop.

References: src/…/stream_recovery.py• L85-145 src/…/delivered_response.py• L505-517

Continuation request

The continuation request presents the delivered prefix as assistant history and adds an instruction to continue without repeating it. It keeps the original model and tool contract, while discarding unfinished tool calls from the interrupted attempt because they were not delivered or run.

The request can cross the client and upstream protocol boundary. It preserves exact delivered text where a normal converter might insert separators. It also captures the sent Gemini context for later replay sealing. Native tool and reasoning forms are projected into Chat, Messages, Responses, or Gemini history without changing the client-side conversation.

References: src/…/continuation_request.py• L37-86 src/…/continuation_request.py• L88-219 src/…/request.py• L106-171

Overlap handling

Overlap handling removes only output that the new upstream repeats from the delivered prefix. It compares text and thinking by logical channel and keeps the new suffix, including output split across many events or represented as cumulative snapshots.

The overlap state stays private until it can decide whether the candidate matches the expected prefix. A changed prefix, inconsistent event order, terminal snapshot, or retained-data overflow stops the continuation instead of emitting duplicated or conflicting output.

References: src/…/continuation_overlap.py• L43-161 src/…/continuation_overlap.py• L304-470 tests/…/test_delivered_stream_recovery.py• L322-436

Public response continuity

The public stream maps continuation events into the response that is already open. It closes the delivered prefix where the wire format needs closure, removes repeated start frames, keeps the original response identity, offsets later item indexes, and maintains increasing sequence numbers.

The stream publishes a terminal completion only after the continuation adds progress. A repeated prefix with no new output is a failure, not a successful recovery. A tool handoff instead closes the response with the already delivered complete call and does not make another upstream request.

Native Responses and Messages relays also synthesize one safe terminal failure when a public stream ends before its normal terminal event. The failure keeps the response identity and does not invent completion for an active tool.

References: src/…/continuation_stream.py• L21-171 src/…/continuation_stream.py• L173-270 src/…/native.py• L121-158 src/…/messages_stream.py• L373-392

Attempt holdback

Private stream consumers can hold early upstream events for a short window so an early cutoff can retry without exposing a failed attempt. Public streams do not add this delay: their delivery state is the commit source, and readable content is released as soon as it is ready.

The holdback is bounded by time and bytes. The provider reserves the remaining attempt budget for midstream recovery when generated output already exists, then discards uncommitted events for an early retry or flushes them only when a salvageable completed call requires it.

References: src/…/stream_recovery.py• L15-18 src/…/stream_recovery.py• L47-103 src/…/stream_recovery.py• L156-201 tests/…/test_stream_recovery.py• L16-93

Wire safety boundaries

Wire safety boundaries keep shared failure meaning and tool identity stable while each adapter uses its own public format. They prevent provider-specific details from leaking into recovery state or error handling.

Error translation

The core uses neutral execution failures for recovery decisions. The Anthropic boundary maps those failures to Messages error types and status codes, redacts sensitive message text, and preserves in-band upstream failure categories. This keeps the recovery layer independent of one provider's error vocabulary.

References: src/…/errors.py• L8-31 src/…/errors.py• L34-51 src/…/errors.py• L100-122

Tool identity translation

Tool identity must survive protocol conversion because a call ID and a tool name link model output to later tool results. The core creates deterministic portable names for OpenAI-compatible boundaries, retains the original name in request-local scope, and restores it when output returns. Responses tool adaptation applies the same rule to custom tools, namespaces, and client tool search.

References: src/…/openai_tool_names.py• L17-62 src/…/tool_adaptation.py• L55-157 src/…/tool_adaptation.py• L419-484

Stream diagnostics

Stream diagnostics record completion, interruption, cancellation, byte count, and chunk count without changing the stream result. Trace values redact credential-shaped fields, and cleanup warnings preserve the original stream failure when input closure also fails.

References: src/…/trace.py• L24-69 src/…/trace.py• L72-117 src/…/trace.py• L135-225

Change checklist

When these files or sections are changed, remember to consider these:

  • Keep replay carrier fields aligned with stream presenters, history projectors, and finalization fields for all four upstream protocols.
  • Keep provenance checks aligned with provider, protocol, endpoint, connection identity, and model handling. Do not expose credentials in replay state.
  • Keep delivered-prefix parsing aligned with every public event shape, block-close rule, tool-call rule, and terminal snapshot.
  • Keep overlap rules aligned with text and thinking channels, cumulative snapshots, event indexes, item IDs, and sequence numbers.
  • Keep continuation requests aligned with the client wire protocol and the upstream tool contract. Unfinished calls must not enter the next turn.
  • Keep the 1 MiB recovery limit separate from the 16 MiB pending tool-output limit. Both limits must still produce an unsafe or terminal result rather than partial replay.
  • Keep native stream validators aligned with public relays so a valid source lifecycle cannot become an invalid destination lifecycle.
  • Keep source tool completion evidence separate from presentation events. A presenter must not release an unfinished call.
  • Keep public commit state separate from private holdback state so recovery does not delay or duplicate visible output.
  • Keep recovery decisions inside the common attempt budget and preserve the no-op continuation failure rule.

Important failure modes

TriggerConsequenceGuard
A continuation has no retained native prefixThe next upstream cannot continue the delivered response safelyRequire a continuation seed and stop when it is missing
The retry repeats the complete delivered prefix without new outputThe client sees a false successful completionRequire continuation progress before accepting a terminal frame
The retry changes an already observed prefixThe public response contains conflicting text or thinkingBuffer overlap evidence and fail on inconsistent content or order
Retained recovery evidence exceeds 1 MiBA partial final snapshot could be presented as completeClear reconstructable state and mark recovery unsafe
Pending tool output exceeds its byte or record limitAn unresolved call can consume unbounded memory or leak as a partial callFail with a non-retryable pending-output limit error
A released tool call is incompleteThe client may act on a call that the provider did not finishStop recovery unless the call is complete and handoff is safe
A native event arrives out of lifecycle orderThe public stream can contain invalid blocks or terminal stateValidate source lifecycle before relay or conversion
A draft replay carrier is published before its public echo is knownThe next turn cannot validate or rebuild the native continuationFinalize carriers against delivered response state before publication
A stream failure occurs after public content is committedA fresh request could duplicate visible outputContinue, hand off, or emit a terminal failure instead of restarting blindly
A private holdback consumes the last attempt before recoveryThe partial result cannot be completedReserve the last attempt when generated output needs midstream recovery
Tool names collide after conversion or call IDs are reusedTool results can attach to the wrong toolPreserve request-local identity mappings and reject ambiguous identities

Key files

FileWhy to read it
src/…/history_replay.pyDefines provenance-aware native history, readable projections, and replay association rules.
src/…/replay_finalization.pySeals draft continuation carriers against output actually delivered to the client.
src/…/delivered_response.pyCaptures bounded public output evidence and produces recovery snapshots.
src/…/stream_recovery.pyDefines protocol-neutral delivered prefixes and safe recovery actions.
src/…/continuation_request.pyBuilds an upstream request from the delivered client-side prefix.
src/…/continuation_overlap.pyRemoves repeated continuation output and rejects changed prefixes.
src/…/continuation_stream.pyMaps a recovery attempt into one continuous public response.
src/…/native_stream.pyValidates native Messages lifecycle and preserves provider event identity.
src/…/tool_calls.pyHolds Messages tool events until source termination proves completion.
src/…/tool_calls.pyResolves and seals Chat tool fragments by choice and call identity.
src/…/tool_calls.pyValidates authoritative Responses tool completion and preserves source order.
src/…/messages_stream.pyConverts native Messages blocks into one Responses lifecycle.
src/…/ledger.pyTracks Responses output slots, active blocks, and usage.
src/…/tool_call_buffer.pyProvides bounded source-order retention for unresolved calls.
src/…/stream_output.pyCarries source-established completed-call evidence to presentation code.
src/…/stream_recovery.pyApplies holdback, attempt limits, and provider stream failure decisions.
src/…/trace.pyDefines sanitized stream diagnostics and cleanup behavior.
tests/…/test_delivered_stream_recovery.pyShows text continuation, overlap handling, tool handoff, retention limits, and public identity guarantees.
tests/…/test_stream_recovery.pyDefines hidden retry, commit, holdback, budget, and cancellation behavior.